Unlimited-OCR deployment with vLLM and SGLang

This explainer covers Unlimited-OCR, an open source document parsing system that the speaker describes as extending DeepSeek-OCR for long documents. It explains image configuration, PDF preparation and self-hosted deployment options.

Single images have two modes. Gundam enables cropping with a 640-pixel image size and a 1024-pixel base size. Base disables cropping and uses a 1024-pixel image size. The speaker says multi-page documents and PDFs require base mode through infer_multi. The Python PDF workflow uses PyMuPDF to convert pages into temporary PNG files at 300 DPI, then passes their paths and a text prompt to the model.

The deployment discussion covers Hugging Face Transformers tested with Python 3.12.3 and CUDA 12.9 on NVIDIA GPUs. For vLLM, the speaker describes a default Docker image for CUDA 13.0 and a cu129 image for Hopper GPUs. The SGLang setup uses a uv-managed environment, a local wheel and kernels pinned to version 0.117. Its server exposes an OpenAI-compatible streaming API on port 10000, according to the explainer.

The final section explains infer.py, which starts a server and sends concurrent requests for an image directory or PDF. Input, output directory, concurrency and image mode flags control the workflow. Eight concurrent streams are an example configuration, not a measured performance result. PDF input still requires base mode.