DeepSeek-V4.1-Flash on vLLM: 5x Agentic Throughput Since Day 0

TL;DR: In the three weeks after DeepSeek-V4.1-Flash’s release, Inferact and the vLLM community optimized the model, achieving a 1.9× speedup at low concurrency and a 5.3× throughput improvement under a 150 TPS constraint. The performance improvement comes from:

• We implemented SWA bounded replay with CUDA graphs, achieving a ~30% TTFT reduction.
• We integrated DeepSeek’s newly released kernels, including MegaAttention, Mega-mHC, Mega-Gate, and DeepSelect.
• We aggressively fused and parallelized the remaining kernels, including mHC side streams, and fused all-reduce with the preceding and following ops into a single kernel.

DeepSeek V4.1 introduces a highly efficient architecture for long-horizon agentic serving tasks: with its causal encoder-decoder (CED) architecture, the model activates 16B parameters per token during decode but only 8B parameters during prefill. The model is also extremely memory efficient. It combines several techniques to shrink the KV cache size: Compressed Sparse Attention 2 (CSA2), FP4 KV cache, and inter-layer KV cache sharing, pushing the global KV footprint to 890 bytes per token. This post shows how we combine these model-level optimizations from DeepSeek with vLLM-side system optimizations to achieve 5× throughput on the SemiAnalysis…