MLflow

MLflow helps developers and teams inspect AI applications, compare changes and track model experiments. It brings agent debugging and evaluation into a platform you can run locally or on your own servers.

• Agent tracing: OpenTelemetry-based traces let you inspect application behavior and monitor quality, latency, costs and safety, with AI analysis to identify problems in recorded runs.
• Evaluation: Compare results using built-in metrics, LLM judges or custom checks, track quality over time and detect regressions before deployment.
• Prompt management: Keep prompt versions and their history together, test and deploy them, and use automatic optimization to improve results.
• Model access and agent hosting: An OpenAI-compatible gateway handles provider routing, fallbacks and cost controls, while Agent Server hosts agents with streaming responses and built-in tracing.
• Model experiments: Record training parameters, metrics and evaluation results, manage models in a shared registry, and deploy them for batch or live predictions.

MLflow is free and open source under Apache 2.0. It supports local environments, self-hosted servers and Docker, with managed hosting also available through Databricks, Amazon SageMaker and Azure ML. Integrations include Ollama, LangChain, LangGraph, LlamaIndex and PyTorch. Self-hosting MLflow keeps the platform on your infrastructure; requests to cloud model providers such as OpenAI and Anthropic still go to those services.