MLX vs llama.cpp: Qwen3.6 speed on M5 Max

This local LLM comparison tests Qwen3.6-35B-A3B with MLX and llama.cpp on the same Mac. The supplied metadata identifies the hardware as a MacBook Pro M5 Max with 128GB of unified memory running macOS Tahoe. It lists mlx-vlm for MLX inference and a source build of llama.cpp with Metal GPU support.

Both runs use 4-bit quantization. The speaker selects an MLX 4-bit model and a llama.cpp quant described in the transcript as Q4KS, saying their sizes are similar. The different 4-bit distributions run sequentially on the same machine before the presenter places the results side by side. The reported generation speeds are about 98 tokens per second for MLX and 86 tokens per second for llama.cpp. These figures describe the demonstrated runs; they do not establish a speed advantage across other models or hardware.

The llama.cpp demonstration includes the prompt "why is sky blue?" and GPU readings around 90%. The speaker also questions a 4.8GB memory reading because the loaded model is around 20GB, suggesting that the smaller figure might count only active model components. That explanation remains tentative. For viewers who want to run models locally on Apple silicon, the video provides a brief inference-speed comparison rather than a full installation walkthrough or an assessment of answer quality.