OpenLIT

OpenLIT helps developers investigate failures and assess the quality of existing AI applications and coding agents. It connects model requests with the tools, retrieval and agent steps involved in producing an answer.

• Agent tracing: Inspect calls and sessions, filter by cost or latency, and monitor coding agents including Claude Code, Codex and Cursor.
• Evaluations and guardrails: Assess outputs with model judges, heuristics or human review, and detect or block prompt injection and restricted topics at runtime.
• Prompt and context management: Version and roll back prompts independently of app code, store reusable retrieval content, and apply rules based on trace attributes.
• Model comparison and secrets: OpenGround compares responses to the same inputs, while Vault stores and rotates API credentials outside application code.
• GPU monitoring: Track utilization, memory, temperature and power on NVIDIA, AMD and Intel GPUs alongside inference traces.

The core platform is open source under Apache 2.0 and free to self-host with Docker Compose or Kubernetes. Its UI and ClickHouse storage run on your infrastructure, where collected prompts, traces and secrets reside. Integrations include Ollama, GPT4All and vLLM, plus OpenAI and Anthropic; requests to cloud models still go to those providers. OpenTelemetry traces remain portable to Grafana, Datadog and compatible backends.