Quantisation converts model weights and activations from high-precision formats, such as FP32, into lower-precision formats, such as FP16, BF16, INT8, or INT4.
The number of bits determines how many values a format can represent. FP32 offers high precision and is a common training baseline. FP16 and BF16 use half as much memory. BF16 retains the eight-bit exponent of FP32, so it has a similar numerical range, but it provides lower precision because its mantissa is shorter. INT8 provides 256 integer codes, while INT4 provides only 16.
Consider a model with 18 million parameters. With FP32, each parameter requires four bytes, giving a model size of 72 megabytes. FP16 or BF16 reduces this to 36 megabytes. INT8 reduces it to 18 megabytes, and INT4 reduces it to approximately 9 megabytes.
Quantisation offers three main benefits. First, it reduces model size and memory traffic. Second, supported hardware can execute low-precision operations faster. Third, integer operations generally consume less energy than floating-point operations. These advantages make quantisation important for mobile devices, embedded systems, and edge AI.
For activation quantisation, we can use dynamic or static methods. Dynamic quantisation calculates the activation scale during inference from the current input. It adapts well when activation ranges change, but it introduces runtime overhead. Static quantisation estimates the ranges in advance using a representative calibration dataset. This enables faster integer inference, but poor calibration can lead to clipping and larger errors.
Post-training quantisation, or PTQ, quantises an already trained model. It requires little or no additional training, so it is fast and convenient. However, the model has never experienced quantisation error during training. Accuracy may therefore decrease, especially at very low precision.
Quantisation-aware training, or QAT, simulates quantisation during the forward pass. The model keeps floating-point master parameters for optimisation, but fake-quantisation operations introduce rounding and clipping effects during training. Because rounding has zero gradients in most places, QAT commonly uses the straight-through estimator during backpropagation.
At the beginning, PTQ and QAT can produce the same quantised forward result. The difference appears during training. QAT calculates gradients from the quantised simulation, updates the floating-point weights, and quantises them again during the next forward pass. The model gradually adjusts its parameters to values that remain effective after quantisation.
In summary, PTQ is simpler and faster to apply, while QAT usually preserves accuracy more effectively. The final choice should consider the target hardware, required bit width, acceptable accuracy loss, and available training resources.”
