Google has unveiled TurboQuant, a breakthrough compression algorithm that reduces AI key-value cache memory by 6x and delivers up to 8x speed gains on H100 GPUs — all without retraining or fine-tuning existing models. The technique quantizes KV cache to just 3 bits with zero accuracy loss, potentially cutting enterprise AI infrastructure costs by more than 50%.
TurboQuant works in two stages: PolarQuant randomly rotates data vectors and converts them to polar coordinates, eliminating expensive normalization. Then QJL applies 1-bit error correction, reducing each value to a single sign bit with negligible overhead. Google tested it across Gemma, Mistral and Llama-3.1-8B models, achieving perfect scores on needle-in-haystack benchmarks and superior recall in vector search compared to existing methods.
The announcement immediately rattled semiconductor markets. Stocks of major memory suppliers including Micron, SK Hynix, Samsung, and Western Digital fell as analysts reassessed demand projections for AI memory chips. Google plans to present the findings at ICLR 2026 and AISTATS 2026.
