Open-Source LLM Evaluation Tools
Test models, prompts, agents and RAG pipelines against your own cases with Promptfoo, DeepEval, Ragas or lm-evaluation-harness.
A local security scanner for AI apps and MCP servers that checks prompt injection and agent permissions, with offline analysis under Apache 2.0.
11Updated 23 hours agoApache-2.0
#AI red teaming#Git integration#MCP
Nox is a security scanner for developers building LLM applications, RAG systems and agents with tool access. It traces untrusted input into model calls and model output into dangerous operations, including paths across functions and files.
Guardrails, Security and SandboxingLLM Evaluation and TestingMCP Servers and Agent Skills+0
Local AI agent builder runs on desktop or self-hosted servers, supports local and cloud models, and automates evaluation and tuning under Apache 2.0.
3.2KUpdated 1 day agoApache-2.0
macOS · Windows · Linux · Docker · Web#Agent Skills#Human approval#LLM tracing
PenguinHarness builds runnable AI agent applications from plain-language requirements on your computer or server. It's for developers who want agents to handle app creation, evaluation, tuning and deployment.
Agent FrameworksLLM Evaluation and TestingLLM Observability and TracingPrompt-to-App Builders+0
A self-hosted data labeling and AI evaluation platform with custom review interfaces, model-assisted annotation and an Apache 2.0 license.
28.7KUpdated 24 hours agoApache-2.0
macOS · Windows · Docker · Web#Human approval#Multi-user access#Multimodal input
Label Studio is a platform for teams preparing training datasets or reviewing AI outputs. It brings human annotation and evaluation into a browser interface that can run locally or on your own server.
Data Labeling and AnnotationLLM Evaluation and Testing+0
LLM evaluation CLI and library that runs locally, compares Ollama and cloud models, and tests AI applications for vulnerabilities. MIT licensed.
27.2KUpdated 19 hours agoMIT
#AI red teaming#Git integration#MCP
Promptfoo is an open source CLI and library for developers and security teams evaluating AI applications on their own machine. It supports local models through Ollama alongside cloud providers. Evaluations run locally, but tests against cloud models send requests to those providers, and most require an API key.
Guardrails, Security and SandboxingLLM Evaluation and Testing+0
A self-hosted LLM fine-tuning tool with a no-code browser interface, memory-saving training methods and support for NVIDIA GPUs on Ubuntu or Docker.
5.2KUpdated 2 days agoApache-2.0
Linux · Docker · Web#Hugging Face integration#LoRA#Quantization
H2O LLM Studio helps teams adapt language models to their own datasets through a browser interface, without writing training code. It's aimed at organizations building models for specific tasks.
Experiment Tracking and MLOpsLLM Evaluation and TestingLLM Fine-Tuning+0
Python LLM red teaming framework that runs tests on your machine, saves local reports and uses model APIs for attacks and scoring. Apache 2.0.
3.1KUpdated 3 weeks agoApache-2.0
#AI red teaming#Guardrails
DeepTeam is a Python framework that runs on your machine to test chatbots, AI agents and retrieval-augmented generation applications for security failures. It's for developers and security teams who want adversarial tests they can repeat as an application changes. Its model-powered attacks and scoring use the selected provider; calls to OpenAI or other cloud services send data off your machine.
Guardrails, Security and SandboxingLLM Evaluation and Testing+0
A local AI agent debugger records coding runs, replays saved model responses without model calls, and compares models from the same checkpoint.
668Updated 1 month agoApache-2.0
Windows · Docker · Web#Code execution#Git integration#LLM tracing
OrcaReplay is a local AI agent debugger for developers investigating failed coding runs or testing how another model handles the same task. It captures what happened beyond the chat transcript, including failed commands and changed files.
LLM Evaluation and TestingLLM Observability and Tracing+0
A local LLM training and deployment framework with LoRA, multimodal models and vLLM, SGLang or LMDeploy integration, licensed under Apache 2.0.
16.1KUpdated 1 day agoApache-2.0
Web#Distributed execution#Hugging Face integration#LoRA
ms-swift is a Python framework for developers and researchers training and deploying language and multimodal models on their own hardware. It covers model adaptation through evaluation and serving, with support for Qwen3, DeepSeek-R1, Llama4, Mistral, Qwen3-VL and InternVL3.5.
Inference Servers and ClustersLLM Evaluation and TestingLLM Fine-TuningQuantization and Model ConversionTraining Frameworks+0
An open-source LLM toolkit trains, fine-tunes and serves models on your own hardware, with quantization and multi-GPU support.
13.7KUpdated 4 days agoApache-2.0
#Hugging Face integration#LoRA#Quantization
LitGPT is a Python toolkit for developers and researchers who want to train and deploy language models on their own hardware. Its directly editable model implementations suit custom applications and research experiments.
Inference Libraries and BindingsInference Servers and ClustersLLM Evaluation and TestingLLM Fine-TuningTraining Frameworks+0
Local AI coding workflow engine with fixed specifications, automated evaluation and resumable runs. Works with Claude Code and Codex CLI. MIT licensed.
6.4KUpdated 2 days agoMIT
#Code execution#Guardrails#MCP
Ouroboros is a locally installed workflow engine for developers who want to define requirements and check an AI coding agent's output. It runs through a terminal CLI, a Claude Code plugin or an MCP server. Your chosen coding runtime supplies the agent and model access. When used with Claude through Claude Code's standard cloud connection, model requests go to that external service.
Agent FrameworksAutonomous Coding AgentsLLM Evaluation and TestingMCP Servers and Agent Skills+0
A local AI verification toolkit checks binary claims and Python or C rewrites against evidence, with a CLI and MCP server under the MIT license.
1.3KUpdated 2 months agoMIT
macOS · Windows · Linux#Code execution#Guardrails#MCP
Reverify helps developers and reverse engineers check AI-generated claims against actual binary files and code behavior. It returns verified, refuted or inconclusive verdicts with evidence, so an assistant's proposed explanation can be checked before you rely on it.
Agent MemoryGuardrails, Security and SandboxingLLM Evaluation and TestingMCP Servers and Agent Skills+0
An open-source LLM vulnerability scanner tests text, image and audio attacks against model APIs and supports automated checks in GitHub Actions.
2KUpdated 1 day agoApache-2.0
#AI red teaming#Multimodal input
Agentic Security is an LLM vulnerability scanner for developers, researchers and security teams testing AI models and agent workflows. It probes how systems respond to hostile inputs and attempts to bypass their safety controls.
Guardrails, Security and SandboxingLLM Evaluation and Testing+0
An open-source Python tool evaluates AI agents and RAG apps with step-level tracing, explained scores and storage in a database you control.
3.6KUpdated 20 hours agoMIT
#Batch processing#LLM tracing#MCP
TruLens helps developers assess AI agents and retrieval-augmented generation (RAG) apps, then trace poor results to the steps that caused them. It connects evaluation scores with execution traces so teams can compare changes in quality, latency and cost.
LLM Evaluation and TestingLLM Observability and Tracing+0
MaxText
2 videosAn open-source LLM training library using Python and JAX, with pre-training and fine-tuning on TPUs and GPUs and a mode without GCP dependencies.
2.5KUpdated 18 hours agoApache-2.0
Docker#Distributed execution#Multimodal input#Quantization
MaxText is an LLM training library for researchers and engineers building models from scratch or adapting existing ones. It provides a reference implementation teams can modify for their own model designs, with training that scales from a single host to large clusters.
LLM Evaluation and TestingLLM Fine-TuningTraining Frameworks+0
An open-source LLM evaluation toolkit tests local models and compatible APIs across six trustworthiness dimensions, with HTML and JSON scores.
636Updated 1 month agoMIT
#AI red teaming#Hugging Face integration#OpenAI-compatible API
TrustLLM is a research toolkit for teams comparing language models beyond answer quality. It evaluates local LLMs and models accessed through APIs for behaviors such as hallucination, harmful responses, bias, and information leakage.
LLM Evaluation and Testing+0
A self-hosted AI agent platform for testing prompts, evaluating outputs, and tracing execution, with OpenAI and Volcengine Ark integrations.
5.8KUpdated 5 days agoApache-2.0
Docker · Web#LLM tracing#Prompt versioning
Coze Loop helps developers test and monitor AI agents, with tools for investigating failed runs and comparing prompt changes. Its self-hosted edition brings prompt development, evaluation, and execution traces into one browser interface.
LLM Evaluation and TestingLLM Observability and TracingPrompt Management and Playgrounds+0
An open-source LLM evaluation framework that compares local models and cloud APIs on shared benchmarks, with support for multi-GPU evaluation.
7.6KUpdated 3 days agoApache-2.0
#Distributed execution#Hugging Face integration#OpenAI-compatible API
OpenCompass is an LLM evaluation framework for researchers and developers comparing models for their applications. It lets you assess models on your own hardware alongside cloud APIs in the same experiment.
LLM Evaluation and Testing+0
A self-hosted AI observability platform traces and evaluates LLM apps, with Ollama integration, experiment comparisons and OpenTelemetry support.
12.3KUpdated 1 day ago
Docker · Web#LLM tracing#MCP#Ollama integration
Arize Phoenix helps developers investigate failures in AI agents and LLM applications and compare proposed fixes. It connects recorded application activity with evaluations and repeatable experiments.
Experiment Tracking and MLOpsLLM Evaluation and TestingLLM Observability and TracingMCP Servers and Agent SkillsPrompt Management and Playgrounds+0
A self-hosted LLM monitoring tool that tracks costs, latency, and prompt traces on macOS, Windows, Linux, or Docker, with Ollama and cloud API support.
20Updated 3 months ago
macOS · Windows · Linux · Docker · Web · Browser Extension#Batch processing#LLM tracing#MCP
Torrix is a self-hosted LLM monitoring tool for teams investigating AI spending and failed requests. It keeps request records in your infrastructure and brings usage analytics and individual traces into one dashboard.
Guardrails, Security and SandboxingLLM Evaluation and TestingLLM Gateways and RoutersLLM Observability and TracingPrompt Management and Playgrounds+0
An open-source Python framework evaluates LLMs, RAG and predictive models with custom metrics and a self-hosted monitoring dashboard.
8.1KUpdated 2 months agoApache-2.0
Web
Evidently helps AI teams check output quality and catch regressions in chatbots, RAG applications, AI agents and predictive models. Its Python library handles text and tabular data for development evaluations and production monitoring.
LLM Evaluation and TestingLLM Observability and Tracing+0
A coding agent workflow analyzer that turns project and session evidence into prioritized findings, with support for Claude Code, Codex and Qoder.
2.5KUpdated 3 days agoMIT
#Agent Skills#Git integration#Source citations
Better Harness helps developers and teams assess how coding agents carry out work. It examines the workflow around a code change to find unclear goals, missing validation and delivery checks that a diff review can miss.
Coding Assistants and AgentsLLM Evaluation and TestingLLM Observability and TracingMCP Servers and Agent Skills+0
A Python library that reduces LLM cache memory use with Hugging Face Transformers, multiple compression methods and support for cache quantization.
1.3KUpdated 4 days agoApache-2.0
#Context compression#Distributed execution#Hugging Face integration
kvpress is a Python library for developers and researchers reducing the memory needed for long-context LLM inference. It compresses the key-value cache that grows as a model processes longer inputs.
Inference Libraries and BindingsLLM Evaluation and Testing+0
An AI plugin for self-hosted Discourse forums adds chatbots, semantic search and moderation, with a choice of providers and self-hostable AI API services.
48.1KUpdated 21 hours agoGPL-2.0
Linux · Web#LLM tracing#Multi-user access#Structured output
Discourse AI adds community assistants, search and moderation to a Discourse forum you can host on your own server. It's for forum operators who want AI features inside their community site. Forum hosting and AI processing are separate choices: the plugin supports a choice of providers, and its AI API services have a self-hosting option.
AI Bots for Chat AppsLLM Evaluation and TestingWebsite and Support Chatbots+0
An open-source Python LLM evaluation framework with local test runs, explainable scores and custom criteria for agents, RAG pipelines and chatbots.
19.5KUpdated 2 days agoApache-2.0
#LLM tracing#Multimodal input
DeepEval is an LLM evaluation framework for developers testing AI agents, RAG pipelines and chatbots. It helps compare prompts and models and catch quality regressions within Python tests or CI/CD.
Datasets and Synthetic DataLLM Evaluation and TestingLLM Observability and Tracing+0
A self-hosted AI agent observability platform with OpenTelemetry tracing, evaluations and guardrails, free to run under Apache 2.0.
3.1KUpdated 4 days agoApache-2.0
Windows · Linux · Docker · Web#LLM tracing#Ollama integration#Prompt versioning
OpenLIT helps developers investigate failures and assess the quality of existing AI applications and coding agents. It connects model requests with the tools, retrieval and agent steps involved in producing an answer.
GPU Management and MonitoringLLM Evaluation and TestingLLM Observability and TracingPrompt Management and Playgrounds+0
A self-hostable LLM observability platform with tracing, evaluations and prompt management, available under MIT or as a hosted cloud service.
37.4KUpdated 1 day ago
Docker#Agent Skills#LLM tracing#MCP
Langfuse helps teams debug LLM applications and AI agents, then test changes against production examples. It brings traces, evaluations and prompt revisions into one place.
LLM Evaluation and TestingLLM Observability and TracingPrompt Management and Playgrounds+0
An open-source AI app framework connects to Ollama, Gemini and OpenAI, with self-hosted application logic, tool calling and structured output.
6.6KUpdated 1 day agoApache-2.0
Web#LLM tracing#Multimodal input#Ollama integration
Genkit is Google's framework for developers building AI agents and chatbots into web and mobile apps. It brings model connections, application workflows and testing tools into one framework.
Agent FrameworksLLM Evaluation and TestingLLM Libraries and SDKsPrompt Management and PlaygroundsRAG Frameworks and Pipelines+0
A self-hosted LLM gateway with Ollama support, response evaluation and adaptive A/B testing, licensed under Apache 2.0 and no longer maintained.
11.7KUpdated 5 months agoApache-2.0
Docker · Web#Batch processing#LLM tracing#Multimodal input
TensorZero is a self-hosted platform for developers who need to compare and improve models in LLM applications. The project is archived and no longer maintained.
LLM Evaluation and TestingLLM Fine-TuningLLM Gateways and RoutersLLM Observability and TracingPrompt Management and Playgrounds+0
A TypeScript test runner for AI apps lets you inspect results on localhost, compare LLM outputs, and enforce score thresholds in CI.
1.7KUpdated 5 days agoMIT
Web#LLM tracing
Evalite is a TypeScript evaluation tool for developers testing LLM-powered apps. It pairs code-based tests with a local interface for reviewing how an app responds.
LLM Evaluation and Testing+0
LogotyperGot an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your projectLogotyperGot an idea for an AI tool? I'm Eugene, the designer and developer behind Localhosted. I'll help you build it, from the first sketch to launch.
Start your project
