Revolutionizing Quantization: QAM-W Challenges the Status Quo

race to optimize model efficiency, a new player has emerged: QAM-W, or Quadrature Amplitude Modulation for Weights. It presents a novel approach to post-training quantization, notably preserving the intrinsic structure of weight rows, a feat that many scalar quantizers struggle to achieve.

Breaking Down QAM-W

QAM-W stands out by L2-normalizing each row, applying block-Hadamard rotations, and then pairing weights into 2D coordinates. These are then quantized using a Lloyd-Max codebook, tuned to the unit circular Gaussian. This isn't just technical jargon for the sake of it. The reality is, this method allows for activation-aware per-channel scaling, a major shift in maintaining model accuracy.

Here's what the benchmarks actually show: In trials across five large language models, ranging from 1.1 billion to 13 billion parameters, the activation-aware variant of QAM-W managed to maintain perplexity rates within ±0.4% of BF16 WikiText-2 benchmarks. And it did this using 32% fewer weight bits compared to the SmoothQuant W8A8 model. That's a significant reduction without sacrificing performance.

Why It Matters

Strip away the marketing, and you get a quantization method that genuinely innovates. The joint 2D coding approach outperforms traditional polar coding, offering an improvement of 2 to 15 percentage points in perplexity at the same bitrate. Moreover, the correlation between the Kullback-Leibler divergence and perplexity change remains nearly perfect, with a Spearman correlation of 0.99 across 37 method-model pairs.

So, why should this matter to you? At a more aggressive 3.5 bits per weight (bpw) setting, QAM-W proves competitive on architectures that tolerate quantization. Even under strict constraints of 4 bpw, it's clear there's competition, as the rotated-codebook approach, QTIP, outperforms QAM-W. Yet, QAM-W shines in the 5-6 bpw range, maintaining quality while conserving bandwidth.

The Road Ahead

Frankly, the architecture matters more than the parameter count. As models grow, the demand for efficiency grows with them. QAM-W presents a compelling case for adopting smarter, more efficient quantization methods that don't compromise on accuracy. The question is no longer whether we can reduce model sizes, but how intelligently we can do it without losing performance. QAM-W might just be the key to unlocking that balance.