Model-Preserving Adaptive Rounding

Researchers introduce YAQA, an adaptive quantization algorithm for large language models that uses Kronecker-factored Hessian approximations to better preserve the original model's output distribution. YAQA achieves approximately 30% reduction in KL divergence and state-of-the-art performance across various models and tasks.