Commenter explores compression-as-prompt concept in reverse
2 Sep 22 4:58 AM · 1d ago · 3 comments · 1 source · development 2 of 4
A Hacker News commenter extends the logic, proposing that reproducible language models could treat generated code or text as compressed versions of themselves, where the prompt (or tokens) becomes the compression.
“The prompt (or really, the tokens) would then be the compressed version of the codebase or the text.”
berkesNathan Pemberton Researcher/author
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What was reported 1 claim about this development
-
2 outlets Can gzip be a language model?
first by HN Best, 2d ago · also HN Frontpage
What people said 3 voices · verbatim
-
I've been pondering on something related: can an LLM be a chat?Some models are reproducible, in that the same prompt will generate the same output. Say that we could wire up such a model to generate some code.In that case, we could create a prompt that generates, say, an entire codebase, or a large piece of text. The prompt (or really, the tokens)…
-
3blue1brown did a series on this topic: https://www.youtube.com/watch?v=l6DKRf-fAAM https://www.youtube.com/watch?v=GlYgs6v2YfU (i think one more is yet to release)
-
This is fun, but historically people have gone a bit overboard with saying that models like this, or n-gram language models, are anywhere close to large neural network models. There is certainly a connection though.
All 4 developments of Researcher demonstrates gzip as functional language model… →
Hacker NewsNewswiresMastodon