OGX (formerly Llama Stack)

OGX, formerly Llama Stack, is an AI application server for developers building chat apps, document search and AI agents. It brings model access and the services those applications need into a server you can host yourself.

• Model access: Use text and vision models, streaming responses and embeddings through a common API, with Ollama and vLLM as local inference backends.
• Agent tools: The Responses API coordinates tool calls on the server, connects agents to MCP tools and includes file search for answers grounded in uploaded documents.
• Document search: Store files and search them through vector stores, with backend choices including FAISS, SQLite-vec, Qdrant and PGVector.
• Client compatibility: Use OpenAI-compatible clients or the Anthropic and Google GenAI SDKs, and switch model providers without rewriting application code.
• Application support: Manage prompt templates and versioned agent skill bundles, process documents for ingestion and submit batch requests.

OGX is open source under the MIT license and can run on a laptop, in a datacenter or on cloud infrastructure. Applications connect over HTTP, so they aren't tied to a Python framework. Its Responses API follows the Open Responses specification.

Inference can run locally through Ollama or vLLM. Choosing OpenAI, Anthropic, Gemini, AWS Bedrock or Vertex AI sends inference requests to that external service. Responses guardrails also rely on an external OpenAI-compatible moderation endpoint.