DGX Spark home lab: vLLM, Ollama and Copilot CLI

This self-hosted AI setup assigns different jobs to several machines. Three NVIDIA DGX Sparks and an ASUS Ascent GB10 form the main inference cluster. The speaker describes pooling 512 GB of memory through NCCL and a MikroTik switch, with vLLM serving DeepSeek v4-0731 across all four systems. He reports about 60 tokens per second during decoding and says the model has met his development needs without frequent switches to Claude. These are personal results, not controlled benchmarks.

A 16 GB M4 Mac mini runs OpenClaw for occasional testing. The speaker also tried Hermes but found no lasting use for it. His AMD Strix Halo machine, a Beelink GTR 9 Pro, handles model experiments while the cluster stays occupied. He notes that BIOS settings can limit GPU memory allocation to 96 GB. A separate RTX Pro 4000 build with 24 GB of VRAM runs embedding models for RAG projects. The dedicated KV cache server is off during the recording because its NVMe drives were returned.

A custom dashboard monitors the machines. GitHub Copilot CLI is the main coding assistant interface, which the speaker uses to access both subscription cloud models and models in his home lab. For viewers starting to run models locally, he recommends a used Mac mini or Ollama on an existing desktop with a GPU rather than copying the full setup.