INT4, FP8, and INT8 on consumer hardware: which quantization path fits your accelerator in 2026?
vLLM’s quantization matrix now spans INT4 W4A16, INT8 W8A8, FP8 W8A8, GGUF, and quantized KV cache support — but the right choice depends on whether your accelerator actually accelerates the format, because framework support does not guarantee kernel-level speedups on every consumer GPU, laptop, or Jetson device.