Dedicated GPUs shed bursts; a pooled fleet spreads them

Burst of requests → Qwen3.5-27B

Baseline · one model per GPU served 0 · shed 0

GPU 0

Qwen3.5-27B

GPU 1

Gemma4-12B

GPU 2

Qwen3.5-9B

GPU 3

Llama-3.2-1B

OpenInfer · every model on every GPU served 0 · shed 0

GPU 0

pooled

GPU 1

pooled

GPU 2

pooled

GPU 3

pooled

Same GPUs, same burst. Pinning saturates one GPU and sheds; pooling spreads the burst across four and serves it all.