Xiaomi's MiMo-V2.5-Pro UltraSpeed hits 1,000 tokens/sec on a 1T model

Xiaomi, working with the TileRT systems team, claims to be the first to push decode throughput past 1,000 tokens per second on a trillion-parameter model. Unlike Cerebras or Groq, which rely on exotic silicon, the team says it hit the number on a single standard 8-GPU commodity node by co-designing the model and runtime rather than throwing custom hardware at the problem. Access is gated: an application-only API trial runs from June 9 to June 23, 2026, priced at 3x the standard MiMo-V2.5-Pro rate but promising roughly 10x the generation speed.

The speedup comes from two main tricks. On the model side, the team applies FP4 (MXFP4) quantization-aware training selectively to the MoE experts, which hold most of the parameters and tolerate low precision, while leaving the rest of the network at higher precision. On the system side, they replace conventional speculative decoding with DFlash, a block-level masked parallel prediction scheme where the draft model fills an entire masked block in one forward pass. The draft uses sliding window attention to keep per-token cost constant rather than scaling with context length, and TileRT supplies custom kernels tuned to the new quantization and decoding pipeline.

The pitch is that at 1,000 tps a 1T model stops being something developers wait on and becomes viable for parallel reasoning strategies like Best-of-N and tree search, faster coding agents, and real-time decision loops in trading, fraud detection, and clinical assistance. Whether the benchmarks hold up under independent testing remains to be seen, but the model–system codesign story is the more interesting claim than the headline number.