Ollama and Open WebUI setup on NVIDIA DGX Spark

This tutorial walks through a self-hosted chat setup on NVIDIA DGX Spark using Ollama for inference and Open WebUI for browser access. The prerequisites are NVIDIA Sync and a user account added to the Docker group. The presenter creates a custom Sync application on port 12000 and uses a launch script from the linked written guide to pull a container with both applications.

The first launch can take several minutes. Creating an account establishes the Open WebUI administrator; the presenter says these credentials stay local. The guide then downloads Gemma 4 31B through the interface. Its listed download size is 20 GB, which the speaker also gives as an approximate base memory requirement before overhead and KV cache. Loading the model after a boot can take additional time. Once installed, the setup supports offline chat on the user's hardware.

The second half explains model labels through Qwen 3.5 35B-A3B-Q4KM, covering model family, version, total and active parameters, and quantization. The presenter recommends Q4 as a starting compromise and Q6 when memory allows, while noting that this advice may change. Model selection also involves commercial licensing, security review and resistance to jailbreak prompts. Hugging Face appears as another source for specialized models, with a warning to check names carefully for lookalike uploads.