Hermes Agent with Qwen 3.6 27B: vLLM setup and review

This walkthrough uses Hermes Agent with Qwen 3.6 27B served through vLLM to document the speaker's Agent Plex project. The local AI setup runs inference on two RTX 4090s and two RTX 3090s. The separate Hermes instance has no GPU; the speaker allocates 16 GB of RAM and eight CPU cores, while reporting observed usage of up to 6 GB and two fully utilized cores.

The server configuration includes four-way tensor parallelism, GPU memory utilization of 0.9, chunked prefill, prefix caching, 8,192 maximum batched tokens and four concurrent sequences. These are settings for the demonstrated hardware and workload. The speaker reports generation throughput around 102 tokens per second at one point, with lower readings later in the task.

Hermes reviews project files and conversation history, then creates an architecture diagram, development summary and timeline hosted on a web server. The speaker finds useful detail but also incorrect elapsed time, hardware information and model settings. They recommend checking timelines manually and using Git history to anchor dates.

A context mismatch also appears: Hermes reports roughly 230K despite the server using about 128K. The speaker warns that compaction could occur too late, reports hangs deep into the context window and suggests an 80K to 85K cutoff for this setup. The review ends by favoring correct answers over raw token speed.