Researchers break 1.58-bit barrier for ternary LLM compression
New packing method reduces storage footprint of three-symbol neural networks below theoretical limit.
What to know
- Researchers achieved sub-1.585-bit storage for ternary LLMs, breaking the previous information-theoretic ceiling via improved packing methods.
- The work improves upon the prevailing five-trit packing standard (1.625 bits per weight) used in practical deployments.
- The result addresses a long-standing compression bottleneck in quantized model deployment.
“Ternary Large Language Models (LLM) store every weight as one of three symbols {−1,0,+1}, so the cost of a ternary model is conventionally referenced to the information-theoretic log₂3 ≈ 1.585 bits per weight.”
Evangelos Georganas, Alexander Heinecke, Pradeep Dubey, Paper authors · arXiv preprint 2609.16338 ↗ · Sep 16, 12:00 AM
Evangelos Georganas Researcher, paper authorAlexander Heinecke Researcher, paper authorPradeep Dubey Researcher, paper author
How it unfolded 2 developments, newest first · click a bar or a number to jump articlesposts
-
2
Paper surfaces on Hacker News frontpage
The arXiv preprint reaches Hacker News, gaining 141 points and sparking discussion among the community about implications for model deployment and compression techniques.
-
sounds like a perfect fit for ASIC-optimized models (where matrix ops could be supported directly in BITCOS format, potentially) & achieving record power efficiency for on-device inference.And it looks like per [0], a model needs only ~30% more weights to be at comparable quality, if quantization-aware training is done...0…
2 more of the top 3 · 8 posts in this stretch
-
> We measure the actual symbol distribution of 29 ternary LLM models and find that zeros account for up to 51.5% of all weights. Motivated by this finding, we introduce BITCOS, a simple distribution-adaptive layoutI honestly assumed that's how they already work. I have to admit that I even explained it like that to a friend. Why on earth wouldn't…
-
So they get down from 1.58 to 1.48 bits per weight by exploiting the fact that actual weights in practice are 0 51% of the time. Neat.If ternary llms work out and are baked into hardware as custom silicon I bet they'll be shockingly efficient.
-
-
1
Paper on ternary LLM compression published on arXiv
Evangelos Georganas, Alexander Heinecke, and Pradeep Dubey post a preprint describing a method to achieve sub-1.585-bit storage for ternary models, improving over the conventional 1.625 bits per weight using five-trit packing.
Also covered reported alongside — the timeline has no entry for these yet
-
first by HN Best, 11d ago · also HN Frontpage, arXiv cs.AI
What people are saying 5 voices from 1 site · best of 8 · verbatim
- Sep 17
-
I'm surprised that a variable length encoding like this is usable directly as in memory format and not just as storage/transfer format.
- Sep 16
-
So this compression is only pertinent to the LLM file format? In memory it'd have to be expanded into the 1.58-bit form - 5 trits per byte.
-
Pushing past log2(3) for real. This could drastically shrink LLMs for embedded systems, making them truly portable.
-
Only a presence bitmap? If we're contemplating packing schemes I'm tempted to write a paper that uses arithmetic coding to squeeze out a few more centi-bits.
-
Ternary quantization does not make any sense. Vector quantization and trellis based methods are better in this region for PTQ.