Mac vs Nvidia GPUs vs DGX Spark: local AI hardware guide

This hardware comparison explains how to choose a machine for a local LLM by checking memory capacity and bandwidth. The speaker contrasts dedicated GPU VRAM with the unified memory in Macs and Nvidia's DGX Spark. Capacity determines whether a model fits; bandwidth helps estimate generation speed once it does.

For dense models, the guide uses memory bandwidth divided by model size as an approximate decode-speed ceiling. The cited 27-billion-parameter benchmark puts the RTX 5090 at about 75 tokens per second, the RTX 4090 at 46 and the M5 Max at 31.4. These are reported baseline results, and the speaker notes that configuration and optimizations affect performance.

Larger models change the comparison. The speaker describes GPT OSS 120B as roughly 60 GB, depending on quantization, and reports better usability on machines with enough unified memory. He also explains why mixture-of-experts models can require substantial capacity while reading fewer active parameters per token.

Prompt processing gets a separate comparison: the cited RTX 4090 result is about 3,000 input tokens per second versus roughly 800 for the M5 Max. The buying advice favors a used RTX 3090 for smaller models and higher-memory systems for larger ones. The speaker cautions that prices vary by region and configuration, and recommends checking benchmarks for the models a buyer actually plans to use.