DeepSeek V4 with vLLM: launch-day consumer GPU failures

This DeepSeek V4 local LLM report covers unsuccessful launch-day attempts on consumer GPUs. The speaker tried isolated RTX 3090 and 4090 configurations, an eight-GPU rig, and two RTX 5060 Ti cards with system RAM offloading. They report crashes near the end of loading and attribute them to missing architectural support. Disabling optimization paths with enforce eager did not resolve the failures.

The troubleshooting discussion covers vLLM nightly builds and a prebuilt Docker image. The speaker says the image avoided a Transformers dependency issue, but their hardware still could not run the model. They also report failed SGLang attempts and expect CPU-only performance to be poor. Their view that the release targets data center GPUs such as H100s and B200s is an assessment of early support, not a demonstrated requirement for every future implementation.

For people trying to run models locally, the recommendation is to wait for GGUF quantizations, possibly Q4 or Q8. The speaker had not seen those or MLX versions at recording time. A reported RTX 3090 patch remains unverified. As a comparison, they describe successful Qwen 3.6 use in vLLM, with fewer tool-calling problems in their experience, while noting that full-resolution inference was slower than Q4.