LLM Quantization Formats: GGUF, AWQ, GPTQ, FP8, and NF4
GGUF, AWQ, GPTQ, FP8, and NF4 are not five interchangeable file formats. GGUF is a model container and metadata format commonly used by llama.cpp. AWQ and GPTQ are post-training quantization methods. FP8 is a numerical format used by supported accelerators and serving kernels. NF4 is a four-bit data type designed for normally distributed weights and commonly used for QLoRA training.
Choose the runtime and hardware first. Then choose a representation that the pinned runtime supports for the exact architecture. Benchmark task quality, memory, time to first token, throughput, and concurrency before accepting the smaller artifact.
Last reviewed: 2026-08-10. The comparison favors runtime compatibility, hardware kernels, training versus serving, artifact provenance, memory, latency, and measured quality loss.
Decision table
| Goal | Best starting point | Check before use |
|---|---|---|
| Local CPU, Metal, or portable llama.cpp inference | GGUF with a documented quantization | Architecture support, chat template, context behavior, quant recipe, and source revision. |
| GPU inference with calibrated weight quantization | AWQ or GPTQ | The serving engine, GPU, bit width, group size, kernels, and model architecture. |
| Supported data-center GPU serving | FP8 | Hardware capability, runtime implementation, calibration mode, and end-to-end quality. |
| Parameter-efficient training with frozen 4-bit weights | NF4 through bitsandbytes | Compute dtype, nested quantization, optimizer memory, and the final merged or adapter artifact. |
| Easiest Transformers prototype | bitsandbytes 8-bit or 4-bit | Backend support and whether the prototype path matches the production server. |
GGUF does not specify one quantization
A .gguf extension tells you how tensors and metadata are packaged, not the precision of every tensor. Names such as Q4_K_M identify llama.cpp quantization recipes, and those recipes can treat tensor groups differently. Record the original checkpoint, conversion revision, quantization command, importance matrix if used, and hashes.
AWQ and GPTQ need a matching serving path
AWQ and GPTQ artifacts usually use Safetensors plus method-specific configuration. A model can download successfully and still fall back to a slow kernel or fail on an unsupported architecture. Check the current support matrix for Transformers, vLLM, or the actual server version before selecting the artifact.
FP8 and NF4 solve different problems
FP8 is attractive when the accelerator and serving stack implement it efficiently. NF4 is primarily associated with memory-efficient adapter training through QLoRA-style workflows. Neither label guarantees the same quality, speed, or memory use across models and hardware.
Evaluation checklist
- compare the same source checkpoint and prompt set
- include long-context and concurrent requests
- measure schema and tool-call correctness alongside perplexity
- record peak device and host memory
- verify time to first token and output throughput
- inspect rare or high-cost quality slices
- keep license, source revision, converter, recipe, and hash with the artifact
Deeper reading
- Model Quantization Guide covers algorithms, calibration, kernels, and deployment trade-offs.
- Open-Weight LLM Variants separates checkpoint role, architecture, container, quantization, runtime, and hardware.
- Local LLMs on macOS applies these choices to Apple Silicon.