The most inspiring discoveries in int4
AutoRound is a quantization toolkit for LLMs and VLMs, optimizing performance with high accuracy at low bit widths.