DeepSeek V2's Multi-head Latent Attention (May 2024)
DeepSeek-V2 introduced Multi-head Latent Attention, which compressed the key-value cache into a low-rank latent space. The technique cut cache memory sharply and let the model serve longer contexts on the same hardware.
V2 cut inference costs and boosted throughput relative to comparable-size models of its day.
MLA was widely copied across the industry and became the template for cache-efficient attention.
V4.1-Flash's 890-byte-per-token cache is the direct descendant of that design, pushing the same obsession from training efficiency into serving efficiency.
