FlashMLA installation and H100 kernel benchmark

Uploaded on February 24, 2025, this walkthrough covers FlashMLA, DeepSeek's first release in its Open Source Week. The presenter describes it as an MLA decoding kernel for NVIDIA Hopper GPUs, optimized for serving variable-length sequences. The setup shown required CUDA 12.3 or above and PyTorch 2.0 or above; no FlashMLA version number is supplied.

The demonstration uses a rented Runpod H100 SXM instance with 80 GB of memory. The presenter clones the repository, runs its installation script and starts the supplied benchmark. He reads results of up to 2,975 GB/s and 532 TFLOPS from the output, while calling his interpretation unscientific. These are kernel benchmark readings, not measurements of chat response speed.

For comparison, he runs a separate FlashAttention 2 script on the H100 and an 8 GB RTX 4060 laptop. The reported readings are 1,398 GB/s and 44 TFLOPS on the H100, versus 106 GB/s and 3 TFLOPS on the laptop. He explicitly warns that the FlashMLA and FlashAttention 2 results are not directly comparable.

For viewers who run models locally, the hardware restriction matters: the presenter says FlashMLA failed on a 4000-series GPU and did not support 3000- or 4000-series cards in this demonstration. Consumer GPU support is a hope he expresses, not a demonstrated capability. He closes with a brief look at a FlashAttention 3 comparison chart.