OpenLIT
Self-hosted LLM observability and evaluation platform under Apache 2.0. Monitor Ollama, vLLM and coding agents with portable OpenTelemetry traces.
OpenLIT is a self-hosted platform for developers who need to understand how their LLM applications and AI agents behave. It connects model calls with tool activity, retrieval and agent steps, so teams can investigate errors and compare cost, latency and output quality across a workflow.
The core platform is open source under Apache 2.0 and can be self-hosted with Docker Compose or Kubernetes. Its UI and ClickHouse storage run in your infrastructure, keeping collected prompts, traces and secrets in your environment. Integrations include local LLM backends such as Ollama, GPT4All and vLLM, alongside cloud providers including OpenAI and Anthropic. Model requests still go to whichever backend your application uses.
Tracing uses OpenTelemetry, so you can retain existing instrumentation and export telemetry to Grafana, Datadog or other compatible backends. OpenLIT doesn't require a proxy for every model call. It also monitors Claude Code, Cursor and Codex sessions, with views of tool use, costs and outcomes.
Evaluations support LLM judges, heuristics and human review on experiments or production traffic. Prompt Hub versions prompts and lets teams deploy or roll them back independently of application code. OpenGround compares model responses side by side on the same inputs, while Vault stores and rotates API secrets outside app code.
For teams running inference on their own hardware, the GPU collector tracks utilization, memory, temperature and power on NVIDIA, AMD and Intel GPUs.
