llama.cpp on a 4 GB PC: Gemma and Qwen performance

This llama.cpp test examines whether a fanless Windows 10 PC with 4 GB of RAM can handle a local LLM. Some memory goes to the integrated GPU, and the presenter reports that Windows uses almost half the total. The test uses CPU processing; the installed Vulkan build does not provide useful GPU acceleration in this setup.

The presenter starts a Gemma model through a batch file, then configures llama-server for network access. Opening the browser on a second computer avoids adding Chrome's memory use to the test machine. VNC provides remote desktop access, although the presenter observes that it also increases CPU use.

Gemma responds slowly with a 4,000-token context and quantized caches. The next test uses Qwen 3.5 0.8B, described as a 455 MB download with MTP included. It leaves more memory free, but prompt processing remains slow. The reported prompt-processing rates are about 0.41 tokens per second for the first test and 0.24 for the second; these are not established response-generation speeds.

The presenter concludes that this particular PC is impractical for interactive use. Memory capacity alone does not explain the result: CPU performance and memory bandwidth also limit it. Linux is suggested as a way to reduce operating-system memory use, but the video does not test that alternative.