Qwen 3.8 Flash Next + Hermes Agent: vLLM setup

The speaker walks through a Qwen 3.8 Flash Next setup with Hermes Agent and vLLM, then tests it on a browser-based alien abduction game. The local LLM recipe targets four RTX 3090 GPUs with 96 GB of combined VRAM. He recommends 128 GB of system RAM and shows roughly 103 GB in use in a Proxmox LXC environment. The build assumes installed drivers plus CUDA and NVCC 13.0, with 13.3 preferred.

The configuration uses a W4A16 quantization that the speaker says supports vision. Settings include a 131072-token context, two concurrent sequences, prefix caching and automatic tool choice. GPU memory utilization is set to 0.96. The recipe targets SM86 Ampere hardware; support for SM89 is suggested as a possible adaptation rather than demonstrated.

The speaker reports around 55 to 59 tokens per second during the AI agent workflow and describes the setup as roughly twice as fast as his earlier configuration. These are results from his rig, not a general benchmark. He disables async scheduling because he says the patch set otherwise eventually crashes, at a cost of about 10 tokens per second.

The game provides a practical coding assistant test: Hermes recalls the project and receives requests to reduce difficulty and fix misaligned graphics. The playable result still has missing cow sounds and a health pickup problem, so the example also shows the need for further testing and revision.