Manifest setup and comparison with OpenRouter and LiteLLM

Manifest is an open source LLM router that sits between an AI agent and its model providers. The speaker reports a 70% drop in token costs over a weekend with the same agent and tasks. That result is a reported example, not a guaranteed saving.

The setup uses Docker Compose, then points an existing client's OpenAI endpoint at Manifest. The demonstration connects OpenAI and Ollama, stores provider credentials in Manifest, and uses a Manifest API key in a Python script. A dashboard tracks token usage, costs per agent and budgets.

According to the speaker, Manifest scores requests across 23 dimensions and selects the cheapest capable model. It uses deterministic scoring rather than another LLM, with routing said to take under two milliseconds. Requests can go to API providers or a local model through Ollama or Llama.cpp. The self-hosted router keeps routing decisions on the user's machine; choosing a remote provider still means sending the request to that provider.

The speaker frames OpenRouter as a cloud endpoint and LiteLLM as a unified interface with routing the user controls. This is a brief account of his workflow preferences, rather than a test of every competing routing feature. Manifest automates selection and provides fallbacks, but users still configure keys and providers. The speaker cautions that its scoring can choose cheaper models than expected. Overrides are available, so users need to understand those decisions.