RubyLLM

RubyLLM is an AI framework for Ruby and Rails developers building chat and agent features into their applications. Its shared API lets you switch between local and hosted models without writing a separate integration for each provider.

• Chat and structured responses: Build conversations with history, streaming replies and file attachments, or request responses with fields defined by a schema.
• Agents and tools: Give agents access to Ruby methods and selected MCP server tools, require human approval for chosen actions, and organize tasks into workflows.
• Document search: Generate embeddings and rerank retrieval results for answers based on your application's documents.
• Images, audio and video: Analyze images and videos, transcribe or generate speech, create images and videos, and convert documents to Markdown with OCR, depending on provider support.
• Rails integration: Save conversations with Active Record and display streaming replies through Hotwire, with background jobs, usage and cost tracking, and OpenTelemetry tracing.

RubyLLM is open source under the MIT license and runs within your application. It supports Ollama for local models, hosted providers such as Anthropic and OpenAI, and OpenAI-compatible endpoints. The backend you choose processes model requests; hosted providers receive the prompts and attachments you send them.

Capabilities vary by provider, so connecting a local model doesn't make every feature available locally. Gemini handles files in the examples, xAI handles video, Mistral provides OCR, and Cohere provides reranking.