Pena demonstrates pre-appended context triples character accuracy
1 Sep 20 · 5d ago · 1 article · 2 posts · 3 sources · development 1 of 1
The single largest improvement in JevChat is the discovery that presenting symbol options as already-appended to the reply—forcing the model to judge finished strings rather than isolated options—roughly triples top-1 accuracy and doubles probability mass on correct symbols, using fewer input tokens.
“It is the single largest improvement in the project: on character alphabets it roughly triples top-1 and doubles the probability mass landing on the right symbol, for fewer input tokens than symbol options with their per-option descriptions.”
Kyle PenaKyle Pena Developer
The whole story articlesposts the bright band is this development · numbered dots are the others · click one to jump
Reported in the same hours no headline names this development itself — these 1 claim were published in its stretch
-
1 outlet I turned Jev into a (lousy) chatbot
first by HN Frontpage, 5d ago
What people said 11 voices · best of 12 · verbatim
-
Jev has taught me the same lesson three times over now.When it first came out, I thought "this weekend, I'll do a little open-source Jev based on single-token prediction and the token logit output", but of course when it came to it, there were at least 5 that had already been done between me thinking that and getting around to it.So I wrote up[0]…
-
There might be some practical applications of this sort of idea like in situations where you want/have a heavily restricted vocabulary to build from. You can already do this with LLMs but they can get very... "distressed" if you force logits, whereas this would not.Come to think of it, I'm now curious if it would do well at building SQL queries…
-
I went for a slightly different approach, described in this thread: https://bsky.app/profile/bernd.wachter.fi/post/3mvv2g4zxp22vNo code published currently, but if somebody is interested I can clean that up next weekend and throw it on github.
-
It's the digital equivalent of Morty speaking with the death crystal: https://youtu.be/YjepJlvkdKs?t=51. The crystal shows him how he will die, so he iteratively determines his speech based on whether he sees himself dying with the life he wants.
-
i think the next step is make Jev a emoji bot... The architecture and its limitations would work well in that regime imo better then human language.
-
Lousy chatbots are still surprisingly useful for prototyping. Wonder if Jev's structured data actually helped or hindered that process.
-
Sometimes I think I'm Morty speaking with the death crystal but I don't even have a death crystal and all I fear is life itself.
-
>> write me a short story> a storyI think this is the first time I’ve knowingly laughed at a model’s joke.
-
> Write me a short story>> Short storyThis looks more like a dating app simulation than a chatbot
-
Try returning a short list of word options by lookup based on the current word completion. Would save some turns.
-
Given Jev does System 1 thinking, this would be equivalent to your ADHD heavy friend.
All 1 developments of Kyle Pena builds a symbol-by-symbol chatbot using Claude as… →
MastodonNewswiresHacker News