Gemma 4 with MLX: setup and benchmarks on Apple silicon

This tutorial explains how to run models locally on an Apple silicon Mac using MLX and the mlx-vlm library. The supplied model link points to Unsloth's Gemma 4-E4B-it-UD-MLX-4bit quantization. The creator uses an M3 Max with 36GB of unified RAM; the results describe that setup rather than performance across all Macs.

The setup starts with a Python virtual environment and dependency installation, followed by a terminal command that loads the model from Hugging Face. The speaker says the first run downloads about 6GB. Once the interactive session starts, the demonstration covers text prompts and image questions. Dragging an image into the terminal supplies its file path, and an exit command closes the session.

A Python example then loads the model, applies a prompt template, and enables verbose output to inspect generation statistics. The speaker reports about 80 tokens per second in an initial example, then roughly 60 tokens per second in repeated text tests with a 1,000-token maximum. An image-input test also reaches around 60 tokens per second. During one run, the reported GPU utilization is 99% and RAM usage is about 74%.

The final section points to a quantization guide with 4-bit and 8-bit options for different model sizes. The speaker advises choosing according to available RAM. This gives viewers a local LLM setup walkthrough and concrete benchmark examples, without establishing a minimum memory requirement.