Exo on DGX Spark and Mac: demo and hardware comparison

Exo Labs previews an NVIDIA Pair backend that distributes inference requests across a DGX Spark and an M3 Ultra Mac Studio. The demo loads the same NVFP4 model on both machines and routes requests according to available resources. Alex describes a shared inference stack for NVIDIA hardware and Apple silicon; the integration is a preview, with further details and rollout still pending during the session.

The discussion explains how to run models locally across machines with different strengths. Alex proposes running compute-heavy prefill on the Spark and memory-bound decoding on the Mac, transferring the KV cache between them. He says ordinary 10 Gbit Ethernet is sufficient when transfers overlap with computation. This prefill/decode split remains planned work, and he gives no release date.

For larger workloads, the speakers compare Spark clusters with RTX Pro 6000 systems and DGX Station. They also distinguish Jetson robotics workloads from desktop LLM inference. Alex says Exo has done little testing on Jetson, so compatibility is an expectation rather than a demonstrated result.

The session introduces local.ai, which compares model task completion time and success rate on selected hardware. Its Spark results cover a single 128 GB device at the time; clustering and split inference are planned additions. Claims about cloud-equivalent speed and lower costs reflect the speakers' arguments and depend on workload, concurrency and hardware utilization.