RubyLLM
An open-source Ruby AI framework for Ruby and Rails apps, with Ollama, hosted providers and OpenAI-compatible endpoints under an MIT license.
RubyLLM is an MIT-licensed AI framework for developers building Ruby and Rails applications with local or hosted models. Its shared API lets an application switch between Ollama, cloud providers such as Anthropic and OpenAI, and OpenAI-compatible endpoints without rewriting its model integration. The framework runs in your application; model processing happens at the local or hosted backend you choose.
Chat includes conversation history, streaming replies and attachments such as images, recordings and PDFs. RubyLLM also supports structured responses defined by schemas, so applications can work with predictable fields rather than free-form text. Agents can call Ruby tools, require human approval for selected actions, and coordinate through workflows with memory and work that resumes across background jobs and deployments.
For document search, it provides embeddings and relevance reranking alongside retrieval-augmented generation. Other capabilities include speech transcription and generation, image and video analysis and generation, document OCR to Markdown, and content moderation. Feature availability depends on the provider: the documented examples use Gemini for files, xAI for video, Mistral for OCR and Cohere for reranking.
Rails integration saves conversations with Active Record, stores files through Active Storage and streams replies with Hotwire. Active Job handles background agent work. The framework also tracks token usage and costs, supports batch requests and prompt caching, and provides retries and fallback handling when requests fail.
