llama.cpp setup on Windows 11 with CUDA 13 and RTX 50

This tutorial walks through compiling llama.cpp for a local LLM setup on Windows 11 with an RTX 50-series GPU. The presenter clones the repository, installs the CUDA Toolkit and CMake, then bootstraps vcpkg with metrics disabled. The preparation includes adding tool directories to PATH and restarting the command prompt so it picks up those changes.

The guide uses vcpkg to install zlib, curl and OpenSSL, then installs Visual Studio build tools through winget. Inside a build directory, the presenter configures a 64-bit build with CUDA enabled and Vulkan disabled. The demonstrated CUDA paths point to version 13.2. There is a material correction: the spoken configuration uses CUDA architecture 100, while the supplied description specifies -DCMAKE_CUDA_ARCHITECTURES=120 for this RTX 50-series setup.

The presenter compiles a Release build using 20 CPU cores, then opens the executables under bin/Release. A downloaded Hugging Face model supplies the final prompt demonstration. The speaker advises choosing a model size that suits the graphics card and notes that some command parameters are specific to his hardware. The example reports CUDA device detection and displays token-per-second statistics, but the transcript supplies no numerical benchmark for comparing performance.