OpenLLMetry
Open-source LLM observability traces model and vector database calls in Python apps, with Ollama support and standard OpenTelemetry output.
OpenLLMetry helps developers trace LLM applications alongside their existing database and API activity. It extends OpenTelemetry, so teams can send AI application traces to the monitoring system they already use.
- Model calls: Trace calls to Ollama, OpenAI, Anthropic, Gemini and AWS Bedrock, among other supported providers.
- Retrieval activity: Follow vector database calls in Chroma, Qdrant, Weaviate, LanceDB and Pinecone alongside model requests.
- Frameworks and agents: Trace applications built with LangChain, LangGraph, LlamaIndex, CrewAI and OpenAI Agents, plus interactions over MCP.
- Monitoring destinations: Send standard OpenTelemetry data to destinations including OpenTelemetry Collector, Grafana, SigNoz, Datadog, Honeycomb and Traceloop.
Teams with an existing OpenTelemetry setup can use individual instrumentations. The Traceloop SDK provides another way to add tracing while retaining the same standard output. Application traces go to the monitoring destination you select.
Traceloop maintains the Python project under the Apache 2.0 license. OpenLLMetry-JS provides the JavaScript and TypeScript implementation.
Logotyper
