txtai

txtai is a Python framework for developers building search, chat with their data, and AI agents on their own hardware. It combines a searchable knowledge store with model workflows, so applications can retrieve context and act on it.

• Semantic and multimedia search: Search by meaning across text, documents, audio, images and video, including images and text in a shared search space.
• Knowledge queries: Combine sparse and dense vector search with SQL queries, graph analysis and topic modeling to explore relationships in your data.
• Model workflows: Connect question answering, summarization, transcription, translation and speech generation, using specialized smaller models alongside LLMs.
• Agents and RAG: Build retrieval augmented generation applications and agents based on smolagents that use the database, model pipelines, workflows and other agents.
• Application connections: Expose applications through web and Model Context Protocol (MCP) APIs, with bindings for JavaScript, Java, Rust and Go.

Local inference supports Hugging Face and llama.cpp, and models can load from local directories or the Hugging Face Hub. Local deployments can process data without sending it to remote model services. Optional LiteLLM connections to OpenAI, Claude and AWS Bedrock send requests to those providers.

The framework runs locally or in Docker and supports scaling through container orchestration. It's open source under Apache 2.0.