CascadeFlow
An open-source AI agent runtime library that controls model and tool calls inside your app, with Ollama, vLLM and cloud provider support.
CascadeFlow is a library for developers who need to control AI agent spending and behavior during a run. It works inside the application, assessing model calls, tool calls and sub-agent handoffs before deciding how execution should continue.
- Model cascading: Try smaller models first and escalate to larger ones when responses fail quality checks, with domain routing for specialist tasks such as coding or legal review.
- Runtime controls: Apply cost, latency, quality, budget, compliance and energy policies to switch models, block tool calls, stop execution, retry, request approval, redact content or return a cached response.
- Decision records: Inspect a per-step audit trail to see which choices the runtime made during an agent run.
- Agent integrations: Work with frameworks including LangChain, OpenAI Agents SDK, CrewAI and Hermes Agent, with Python and TypeScript packages.
The open-source core uses the MIT license and runs within your application. It supports local LLM backends through Ollama and on-premises vLLM, alongside cloud providers including OpenAI, Anthropic, Groq and Together AI. Model requests go to the selected backend, so choosing a cloud provider sends those requests outside your infrastructure.
Studio is a separate managed offering with visual policy and domain builders, custom scoring dimensions, fleet spending reports and organization controls. It supports versioned policies, gradual rollouts and automatic rollback.
