Mamba-3 vs Transformers: state space models explained

This architecture tutorial traces Mamba, Mamba-2 and Mamba-3 through the mechanics of state space models (SSMs). The speaker contrasts a Transformer's growing KV cache with a fixed-size hidden state that compresses input history. That distinction helps explain memory tradeoffs relevant to local LLM inference, though the video does not cover installation or hardware requirements.

The explanation begins with continuous-time equations and discretization, then compares recurrent and convolutional computation. HiPPO provides background on retaining earlier inputs. Mamba makes B, C and the step size delta depend on the current token, allowing the model to select information. The speaker explains why this breaks the fixed convolution kernel and motivates a hardware-aware parallel scan.

Mamba-2 introduces state space duality, connecting structured SSM computation with linear attention. The speaker reports training throughput gains of roughly two to eight times over Mamba. For Mamba-3, the discussion covers exponential-trapezoidal discretization, complex dynamics expressed through rotary positional embeddings, and multi-input, multi-output updates intended to improve GPU utilization during decoding.

The closing discussion addresses retrieval: compressed states can lose exact details. The speaker presents hybrids that interleave SSM and attention layers as a practical response.