Activity — what actually ran
interactions
—
model calls served
tokens generated
—
total model output
GPU time spent
—
actual seconds on the GPU
Savings — avoided vs standard decoding, for the same output (lower is the win)
wall-clock saved
—
less time users waited
GPU compute saved
—
less GPU time than standard
work offloaded to draft
—
tokens the 0.5B draft handled
memory saved
—
smaller footprint: weights + KV