Encoder Disaggregation in vLLM: Scaling Multimodal Serving with EPD

vLLM details its Encoder/Prefill/Decode (EPD) disaggregation work, including metadata-only requests that let downstream instances reuse remote embeddings without repeating media preprocessing, a dedicated encoder-only Model Runner V2 path, and remote embedding transfer via NIXL and Mooncake. It also covers cross-encoder output reuse through Mooncake Store and orchestration support in vLLM Router, SMG, and llm-d, with core capabilities in vLLM v0.32.0 and DeepSeek V4.1 Flash metadata-only support on main via #62802. A DeepSeek V4.1 Flash benchmark on GB200 found EPD delivered 6.6–9.5% higher per-GPU throughput and 44–47% lower P90 TTFT than PD at concurrency 64–128, but lower per-GPU throughput at concurrency 16.