-
Notifications
You must be signed in to change notification settings - Fork 212
feat: add instrument_asyncio() method for asyncio instrumentation #1745
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
36b4c86
e32b399
7ed5246
1b57e4f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| --- | ||
| title: "Logfire Asyncio Integration: Monitor Async Operations" | ||
| description: Learn how to use logfire.instrument_asyncio() to trace and monitor asyncio-based operations. | ||
| integration: otel | ||
| --- | ||
| # Asyncio | ||
|
|
||
| The [`logfire.instrument_asyncio()`][logfire.Logfire.instrument_asyncio] method can be used to instrument | ||
| `asyncio`-based operations with **Logfire**, including tracing coroutines, futures, and detecting slow | ||
| event loop callbacks. | ||
|
|
||
| ## Installation | ||
|
|
||
| Install `logfire` with the `asyncio` extra: | ||
|
|
||
| {{ install_logfire(extras=['asyncio']) }} | ||
|
|
||
| ## Usage | ||
|
|
||
| ```py title="main.py" skip-run="true" skip-reason="external-connection" | ||
| import asyncio | ||
| import os | ||
|
|
||
| import logfire | ||
|
|
||
| logfire.configure() | ||
| os.environ['OTEL_PYTHON_ASYNCIO_COROUTINE_NAMES_TO_TRACE'] = 'my_coro' | ||
|
|
||
| logfire.instrument_asyncio() | ||
|
|
||
|
|
||
| async def my_coro(): | ||
| await asyncio.sleep(0.1) | ||
|
|
||
|
|
||
| async def main(): | ||
| await asyncio.create_task(my_coro()) | ||
|
|
||
|
|
||
| asyncio.run(main()) | ||
| ``` | ||
|
|
||
| The `OTEL_PYTHON_ASYNCIO_COROUTINE_NAMES_TO_TRACE` environment variable specifies which coroutine names | ||
| to trace. Set it to a comma-separated list of coroutine function names. | ||
|
|
||
| You can find more configuration options in the | ||
| [OpenTelemetry asyncio instrumentation documentation][opentelemetry-asyncio]. | ||
|
|
||
| ## Slow Callback Detection | ||
|
|
||
| [`logfire.instrument_asyncio()`][logfire.Logfire.instrument_asyncio] also includes Logfire's built-in slow | ||
| callback detection. It logs a warning whenever a function running in the asyncio event loop blocks for | ||
| longer than `slow_duration` seconds (default: 0.1s): | ||
|
|
||
| ```py title="main.py" skip-run="true" skip-reason="external-connection" | ||
| import logfire | ||
|
|
||
| logfire.configure() | ||
| logfire.instrument_asyncio(slow_duration=0.5) | ||
| ``` | ||
|
|
||
| [`logfire.instrument_asyncio()`][logfire.Logfire.instrument_asyncio] uses the | ||
| **OpenTelemetry Asyncio Instrumentation** package, | ||
| which you can find more information about [here][opentelemetry-asyncio]. | ||
|
|
||
| [opentelemetry-asyncio]: https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/asyncio/asyncio.html | ||
| [opentelemetry]: https://opentelemetry.io/ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from contextlib import AbstractContextManager | ||
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| if TYPE_CHECKING: | ||
| from ..main import Logfire | ||
|
|
||
| try: | ||
| from opentelemetry.instrumentation.asyncio import AsyncioInstrumentor | ||
| except ImportError: # pragma: no cover | ||
| raise RuntimeError( | ||
| '`logfire.instrument_asyncio()` requires the `opentelemetry-instrumentation-asyncio` package.\n' | ||
| 'You can install this with:\n' | ||
| " pip install 'logfire[asyncio]'" | ||
| ) | ||
|
|
||
|
|
||
| def instrument_asyncio( | ||
| logfire_instance: Logfire, | ||
| slow_duration: float = 0.1, | ||
| **kwargs: Any, | ||
| ) -> AbstractContextManager[None]: | ||
| """Instrument asyncio to trace coroutines, futures, and detect slow callbacks. | ||
|
|
||
| See the `Logfire.instrument_asyncio` method for details. | ||
| """ | ||
| from ..async_ import log_slow_callbacks | ||
|
|
||
| AsyncioInstrumentor().instrument(**kwargs) | ||
| return log_slow_callbacks(logfire_instance, slow_duration) |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -892,6 +892,41 @@ def force_flush(self, timeout_millis: int = 3_000) -> bool: # pragma: no cover | |
| """ | ||
| return self._config.force_flush(timeout_millis) | ||
|
|
||
| def instrument_asyncio( | ||
| self, | ||
| slow_duration: float = 0.1, | ||
| **kwargs: Any, | ||
| ) -> AbstractContextManager[None]: | ||
| """Instrument asyncio to trace coroutines, futures, and detect slow event loop callbacks. | ||
|
|
||
| This combines the OpenTelemetry asyncio instrumentation (for tracing coroutines, futures, | ||
| and `to_thread` calls) with Logfire's slow callback detection. | ||
|
|
||
| Args: | ||
| slow_duration: the threshold in seconds for when an event loop callback is considered slow. | ||
| **kwargs: Additional keyword arguments to pass to the OpenTelemetry `instrument` methods, | ||
| for future compatibility. | ||
|
|
||
| Returns: | ||
| A context manager that will revert the slow callback patch when exited. | ||
| This context manager doesn't take into account threads or other concurrency. | ||
| Calling this method will immediately apply the instrumentation | ||
| without waiting for the context manager to be opened, | ||
| i.e. it's not necessary to use this as a context manager. | ||
| """ | ||
| from .integrations.asyncio_ import instrument_asyncio | ||
|
|
||
| self._warn_if_not_initialized_for_instrumentation() | ||
| return instrument_asyncio( | ||
| self, | ||
| slow_duration=slow_duration, | ||
| **{ | ||
| 'tracer_provider': self._config.get_tracer_provider(), | ||
| 'meter_provider': self._config.get_meter_provider(), | ||
| **kwargs, | ||
| }, | ||
| ) | ||
|
Comment on lines
+895
to
+928
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚩 Context manager return only covers slow callback patch, not OTel uninstrumentation The Was this helpful? React with 👍 or 👎 to provide feedback. |
||
|
|
||
| def log_slow_async_callbacks(self, slow_duration: float = 0.1) -> AbstractContextManager[None]: | ||
| """Log a warning whenever a function running in the asyncio event loop blocks for too long. | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -64,6 +64,7 @@ wsgi = ["opentelemetry-instrumentation-wsgi >= 0.42b0"] | |
| aiohttp = ["opentelemetry-instrumentation-aiohttp-client >= 0.42b0"] | ||
| aiohttp-client = ["opentelemetry-instrumentation-aiohttp-client >= 0.42b0"] | ||
| aiohttp-server = ["opentelemetry-instrumentation-aiohttp-server >= 0.55b0"] | ||
| asyncio = ["opentelemetry-instrumentation-asyncio >= 0.42b0"] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🚩 uv.lock not updated with opentelemetry-instrumentation-asyncio dependency The Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| celery = ["opentelemetry-instrumentation-celery >= 0.42b0"] | ||
| django = ["opentelemetry-instrumentation-django >= 0.42b0", "opentelemetry-instrumentation-asgi >= 0.42b0"] | ||
| fastapi = ["opentelemetry-instrumentation-fastapi >= 0.42b0"] | ||
|
|
@@ -124,6 +125,7 @@ dev = [ | |
| "opentelemetry-instrumentation-aiohttp-client>=0.42b0", | ||
| "opentelemetry-instrumentation-aiohttp-server>=0.55b0", | ||
| "opentelemetry-instrumentation-asgi>=0.42b0", | ||
| "opentelemetry-instrumentation-asyncio>=0.42b0", | ||
| "opentelemetry-instrumentation-wsgi>=0.42b0", | ||
| "opentelemetry-instrumentation-fastapi>=0.42b0", | ||
| "opentelemetry-instrumentation-starlette>=0.42b0", | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| from __future__ import annotations | ||
|
|
||
| from asyncio.events import Handle | ||
|
|
||
| import logfire | ||
| from logfire.testing import TestExporter | ||
|
|
||
|
|
||
| def test_instrument_asyncio(exporter: TestExporter) -> None: | ||
| """Test that instrument_asyncio patches Handle._run and can be reverted.""" | ||
| from opentelemetry.instrumentation.asyncio import AsyncioInstrumentor | ||
|
|
||
| assert Handle._run.__qualname__ == 'Handle._run' | ||
|
|
||
| try: | ||
| with logfire.instrument_asyncio(slow_duration=100): | ||
| # Check that the slow callback patching is in effect | ||
| assert Handle._run.__qualname__ == 'log_slow_callbacks.<locals>.patched_run' | ||
|
|
||
| # Check that the patching is reverted after exiting the context manager | ||
| assert Handle._run.__qualname__ == 'Handle._run' | ||
| finally: | ||
| # Clean up OTel instrumentation (context manager only reverts slow callback patch) | ||
| AsyncioInstrumentor().uninstrument() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather please add a
logfire.instrument_asynciomethod, similar to #1744, which uses both AsyncioInstrumentor and log_slow_async_callbacks which can be deprecatedThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done —
logfire.instrument_asyncio()is now implemented, combiningAsyncioInstrumentorwithlog_slow_callbacks. Also fixed the CI failure:test_logfire_api.pyhad a catch-all loop that calledinstrument_asyncio()without cleanup, leavingHandle._runpatched and causingtest_slow_async_callbacksto fail.