Unstract local setup with Ollama and document RAG

This sponsored first look walks through a self-hosted Unstract document extraction workflow on Ubuntu. The setup uses Docker, Ollama for a local LLM and embeddings, Qdrant for vector storage, and the Unstructured IO community text extractor. The speaker connects each component before creating a project in Prompt Studio.

The tutorial covers the connection between Docker containers and the host's Ollama server on port 11434. It also explains the resource tradeoff of a larger context window, using an increased context window for this demonstration. A project needs a default LLM profile before it accepts documents, and uploaded PDFs need indexing before retrieval.

The first test asks for the authors of Attention Is All You Need. The speaker reports roughly 28 GB of video memory use for the selected model during that query; this is an observation from the demonstrated system, not a minimum requirement. Next, the speaker disables Wi-Fi and queries a fictitious business report offline, checking inventory, credit card types and payroll calculations. An API demonstration returns answers in structured output through the terminal.

The final example switches to the online Unstract Playground and GPT-4o for an image-heavy, 263-page Voron assembly manual. The speaker also discusses LLMWhisperer OCR and a beta feature for generating extraction schemas, but does not demonstrate that beta workflow. The closing section notes PostHog usage analytics and a README setting to disable them.