Mistral OCR 4 and LM Studio: local document Q&A tutorial

This tutorial builds a document question-answering assistant with a cloud path and a local fallback. The cloud version uses Mistral OCR 4 and Ministral 3, according to the supplied description. The local version uses docTR for OCR and a Mistral language model running in LM Studio on the speaker's MacBook. A sovereign-mode setting selects the path.

The demonstration uses 20 pages about branching from the Pro Git handbook. Cloud OCR produces 171 indexed blocks, with bounding boxes that distinguish titles and paragraphs. The assistant answers a question about creating and switching branches, then cites pages and blocks for inspection. In the local run, docTR produces 20 blocks. The speaker explains that this fallback can treat an entire page as one block, so its citations are less granular.

The architecture separates PDF ingestion from question answering. Both OCR paths feed a shared block format. FastEmbed creates local embeddings, and a local vector database supplies relevant blocks as numbered sources to the chosen language model.

For readers building a local LLM document workflow, the useful comparison is the retrieval and citation behavior of each path. The speaker recommends models available both in the cloud and as open source releases, but advises documenting performance differences. LM Studio serves as the demo runtime; the speaker suggests a different runtime for production.