llama-server: troubleshoot Qwen and GPT-OSS prompts

This tutorial explains how to troubleshoot a local LLM served by llama-server, part of llama.cpp. The speaker starts a Qwen model on port 8012 and uses the built-in web chat to send test requests. The walkthrough also checks /v1/models through a browser and curl.

The main focus is the difference between API messages and the raw prompt that reaches the model. With --verbose enabled, the speaker extracts request and response JSON from logs, checks sampling parameters, and uses jq to read __verbose.prompt. The Qwen 2.5 Coder example shows message delimiters, a system message, and an assistant prefill. Inspecting this rendered text can help diagnose template behavior that is difficult to spot in the original messages array.

A GPT-OSS instance on port 8013 provides a second example. Its Harmony prompt format differs from Qwen's. The speaker examines the template's system message and explains how the raw prompt can confirm whether a requested reasoning level reached the model.

The service section covers systemd user units and following their logs with journalctl. The speaker warns that verbose logging adds overhead and recommends disabling it after troubleshooting. The final section demonstrates Fish shell abbreviations that strip log prefixes before parsing JSON, extract prompts, and color Harmony messages.