Q-Drift: Advancing Post-Training Quantization in Diffusion Models

In the area of diffusion models, post-training quantization (PTQ) has emerged as a practical solution for deploying large models efficiently. However, the challenge remains that quantization noise tends to accumulate, particularly impacting the denoising trajectory and thus degrading the quality of generated outputs. This is where Q-Drift comes into play, offering a corrective measure that could well be a major shift for the industry.

what's Q-Drift?

Q-Drift is a novel approach designed to tackle the quantization noise problem. It effectively treats the noise as a stochastic perturbation at each denoising step. By doing so, Q-Drift introduces a drift adjustment that preserves the marginal distribution. What's particularly striking is the method's simplicity in practice. It requires as few as five paired full-precision and quantized calibration runs to estimate a variance statistic, which is used in the drift adjustment.

The paper, published in Japanese, reveals that the adjustment is plug-and-play. This means it's easily integrated with existing samplers, diffusion models, and PTQ methods. Importantly, it adds negligible overhead during inference, making it highly efficient and practical for widespread adoption.

Performance Improvements

The benchmark results speak for themselves. Q-Drift demonstrates its prowess across various text-to-image models, including DiT and U-Net. It also shows compatibility with different samplers like Euler, flow-matching, and DPM-Solver++. Notably, it enhances the FID score over the baseline quantized models in most scenarios. For instance, on PixArt-Sigma with SVDQuant W3A4, Q-Drift achieved a FID reduction of up to 4.59 points, while CLIP scores remain preserved. Compare these numbers side by side, and the argument for Q-Drift becomes even more compelling.

Why Q-Drift Matters

Western coverage has largely overlooked this innovation, but it's time to take notice. The implications of Q-Drift's adjustment can potentially lead to more efficient deployments of high-quality generative models in real-world applications, ranging from creative industries to automated content generation. The data shows that Q-Drift isn't just an incremental improvement. it's a necessary evolution for PTQ methods.

But here's the critical question: Why hasn't the broader AI community embraced such a promising advancement? It might be due to the inertia of sticking with established methods. However, as the technology landscape shifts and demands more efficient AI solutions, ignoring innovations like Q-Drift could be costly.