When does model distillation beat quantization for deployment cost and throughput?
Distillation can beat quantization on runtime throughput when the student is much smaller, but the break-even depends on whether the upfront training and engineering cost is amortized over enough tokens; quantization usually wins on time-to-production and capex avoidance, while distillation wins only when sustained inference volume justifies the extra training spend.