KADATH

KADATH is a self-hosted AI agent runtime for developers and researchers comparing agents against a measurable goal. It runs successive generations of agents and selects stronger approaches, but model calls require OpenAI's cloud API, an API key and internet access.

• Framework evolution: Agents can change their system prompts, Python implementation, tools, dependency declarations and supporting files between evaluation rounds.
• Fixed benchmarks: A model proposes a benchmark for your approval before a run, and a separate control layer keeps the objective and scoring rules fixed throughout the competition.
• Selection and descent: The runtime grades agents independently, retains stronger approaches and creates modified descendants that inherit a parent's persistent workspace and progress.
• Inherited and shared memory: Agents retain ancestral experience with its original provenance and can publish records for peers; memory search weighs relevance, evidence, peer ratings and the source agent's verified performance.

KADATH runs in Docker and is open source under Apache 2.0. Initial agents build on Hugging Face's smolagents framework. Run state and agent frameworks stay in your infrastructure, while OpenAI supplies the models used for agent work, benchmark design and grading.

PostgreSQL stores production run state and uses row-level security to separate agents' knowledge. Agents access memory through scoped tokens rather than database credentials, and they can't directly edit scores, objectives or scheduler tables. Their framework repositories remain read-only during an evaluation round.