llmware

llmware is a Python framework for developers building applications that answer questions using their own documents or carry out multi-step AI agent workflows. It focuses on small, specialized models that can run on laptops, edge devices, and self-hosted servers.

• Document retrieval: Parse PDFs, Word documents, PowerPoint presentations, and Excel files, extract tables and text from embedded images, and organize content for text, semantic, or hybrid search.
• Answers with sources: Combine retrieved material with LLM prompts for document-based answers, using BLING and DRAGON models designed for RAG.
• Multi-model agents: Use SLIM function-calling models to build workflows that divide tasks across specialized models, including CPU-based agents.
• Model and database choice: Access models through a shared catalog and connect document collections to stores such as SQLite, Postgres, ChromaDB, Milvus, and Qdrant.

The open-source framework uses the Apache 2.0 license and runs on Windows, macOS, and Linux. It supports GGUF models, OpenVINO, ONNX Runtime, and PyTorch, with CPU-oriented BLING models and quantized versions of SLIM and DRAGON. ONNX Runtime QNN support lets compatible models run on Snapdragon NPUs on Windows Arm64.

Local deployment lets applications process documents and run models on your own hardware. AI Bloks also offers a separate hosted RAG service.