vLLM-Omni introduction and multimodal serving setup

This office hours session introduces vLLM-Omni for self-hosted multimodal inference after reviewing vLLM v0.17.0, v0.18.0, and v0.19.0. Gao Han explains the project's scope: models that generate images, audio, or video, while models that produce only text belong in the main vLLM project.

The walkthrough covers the Omni Python entry point for offline inference and a FastAPI server with OpenAI API formats, including chat completions. For installation, Han describes installing vLLM first, then vLLM-Omni with a matching version. He identifies 0.18.0 as the formal Omni release and 0.19 RC1 as a prerelease aligned with vLLM 0.19.0.

Much of the technical discussion concerns stages. Separate engines handle autoregressive and diffusion workloads, with YAML configuration for each model's stages and arguments. Han explains how overlapping stage execution can improve throughput and how chunked audio streaming reduces the wait for the first output. These are presented as project capabilities and reported results, rather than guarantees for every model or deployment.

Feature support varies by model and configuration. The closing discussion describes llm-d orchestration integration as active work; Open WebUI integration remains a suggestion rather than an established capability.