Kimi K3 vLLM decode up to 2x faster with coding agents
Why it matters
In an Oct 16 vLLM blog post, Benjamin Chislett (NVIDIA) describes a unified CuTe DSL decode path for Moonshot's open 2.8T-parameter Kimi K3 on GB300 GPUs. Per layer, stock vLLM took 130 µs and 23 kernels vs 69 µs and 13 specialized. With DSpark speculative decoding (synthetic AL=6) it reached 842 tokens/s at batch 1 on 16 GB300s (1.73x stock); without speculation, 256.4 tokens/s (1.88x). The team writes that agent-optimized inference code is outpacing review, and proposed a fast-track RFC. Caveats: GB300-specific, decode batches 1-64, gains in several tested configurations.
Read the source: vLLM Blog (opens in a new tab)
All hits from 18 Oct 2026More Open ModelsGet the daily hits by email