Qwen 3.5 with llama.cpp and OpenCode: local coding setup

This tutorial builds a local LLM coding workflow around Qwen 3.5 9B, llama.cpp and OpenCode on Windows. The presenter chooses a GGUF Q4KXL model for a 12 GB RTX 3060, prioritizing a model that fits entirely in GPU memory over a larger model split into RAM. The setup includes an MTP variant and a BF16 vision encoder.

The walkthrough covers CUDA 12 binaries and libraries, folder organization and a batch file for llama-server. The presenter configures a 131,072-token context window, Q8 KV caches and two-token speculative decoding. He reports about 10.3 GB of VRAM use for the combined setup and recommends leaving at least 1 GB free for memory spikes. In his brief generation test, MTP produces roughly 67 tokens per second versus 49 without it; these results describe his hardware and settings.

OpenCode connects to the server through a custom provider at http://127.0.0.1:8080/v1. The coding assistant then creates a Bootstrap landing page with separate HTML, CSS and JavaScript files. Follow-up prompts address poor contrast and an unwanted image element. The presenter discusses documentation and browser tools but does not connect them for this exercise. A later request for many animations produces a result he considers worse, illustrating the need to review and revise generated code.