Our resident kernel recovers about a quarter of a GPU while slowing decode less than 2%
Shipped partitions Ordinary jobs Our resident kernel
Extra work done beside live decode, as a share of a whole GPU, against how much slower decode gets (median step, vs. running alone). Shipped partitions and ordinary jobs: Kimi K2 decode step. Our resident kernel: Llama-3.1-8B. Measured on one NVIDIA RTX PRO 6000 Blackwell (paper, Appendix A4 and A7).