Evalite
A TypeScript test runner for AI apps lets you inspect results on localhost, compare LLM outputs, and enforce score thresholds in CI.
Evalite is a TypeScript evaluation tool for developers testing LLM-powered apps. It pairs code-based tests with a local interface for reviewing how an app responds.
- Local results UI: Inspect outputs, traces, and logs in an interactive interface on localhost.
- Vitest-based tests: Use familiar testing patterns, mocks, and fixtures to evaluate app behavior across different scenarios.
- Model comparison: Test outputs from any LLM against each other without committing your evaluations to one provider.
- CI checks: Fail builds when evaluation scores fall below your thresholds and export results as static HTML bundles for review.
Evalite is open source under the MIT license. Its test runner and results interface run locally; the LLM you choose determines where model requests go.
Logotyper
