oMLX vs RapidMLX: Apple Silicon benchmark tutorial

This tutorial compares oMLX with RapidMLX, MLX-VLM and vLLM-MLX on an M3 Pro with 36GB RAM. The speaker repeats a benchmark workflow from a report that tested an M5 with 64GB RAM, asking whether its long-context findings hold on his hardware. He excludes DFlashMLX and MTPLX because, he says, their narrow model requirements make the comparison less fair.

The setup uses Inference Lab to fetch engine repositories and create separate Python virtual environments. The speaker advises disabling setup entries for engines already installed. He starts an engine with engine.sh, supplies a model, then runs the benchmark against its URL. Each run takes about 15 minutes in his demonstration and produces line-delimited JSON and logs. The workflow uses plot_results.py to generate charts, followed by an HTML report requested through OpenCode.

For this local LLM comparison, the speaker reports that oMLX pulls ahead as context grows beyond 2K. He observes a modest decode throughput lead, stronger prefill results and lower time to first token at longer contexts. He also questions the size of the prefill figures and says he needs to investigate how the benchmark measures them. These are findings from the presented test, rather than proof that one engine will perform best on every Apple Silicon Mac.