Kyle Pena builds a symbol-by-symbol chatbot using Claude as backbone
Developer creates JevChat, a deliberately inefficient proof-of-concept that generates text one character at a time by querying an LLM on each step.
What to know
- JevChat generates text one symbol at a time by asking an LLM (Claude/Jev) to score the next character from a given alphabet, then sampling from the distribution.
- Showing the model a pre-appended option—i.e., asking it to judge the full string with each candidate already added—vastly improves accuracy over presenting options in isolation.
- The project emphasizes experimentation and humor over practical cost efficiency; it includes 158 offline tests and live generation metrics.
Kyle Pena Developer
How it unfolded 1 development · click the chart to see its coverage articlesposts
-
1
Pena demonstrates pre-appended context triples character accuracy
The single largest improvement in JevChat is the discovery that presenting symbol options as already-appended to the reply—forcing the model to judge finished strings rather than isolated options—roughly triples top-1 accuracy and doubles probability mass on correct symbols, using fewer input tokens.
“It is the single largest improvement in the project: on character alphabets it roughly triples top-1 and doubles the probability mass landing on the right symbol, for fewer input tokens than symbol options with their per-option descriptions.”
— Kyle Pena -
Jev has taught me the same lesson three times over now.When it first came out, I thought "this weekend, I'll do a little open-source Jev based on single-token prediction and the token logit output", but of course when it came to it, there were at least 5 that had already been done between me thinking that and getting around to it.So I wrote up[0]…
2 more of the top 3 · 12 posts in this stretch
-
There might be some practical applications of this sort of idea like in situations where you want/have a heavily restricted vocabulary to build from. You can already do this with LLMs but they can get very... "distressed" if you force logits, whereas this would not.Come to think of it, I'm now curious if it would do well at building SQL queries…
-
I went for a slightly different approach, described in this thread: https://bsky.app/profile/bernd.wachter.fi/post/3mvv2g4zxp22vNo code published currently, but if somebody is interested I can clean that up next weekend and throw it on github.
-
-
background
Pena releases JevChat, a symbol-by-symbol chatbot using Claude — Kyle Pena published JevChat on GitHub, a chatbot that generates responses one symbol at a time by querying an LLM decision model at each step. The tool supports multiple sampling strategies (choice, bisect) and alphabets, with a live UI showing generation speed and scoring distribution.
What people are saying 8 voices from 1 site · best of 12 · verbatim
- Sep 22
-
> Write me a short story>> Short storyThis looks more like a dating app simulation than a chatbot
- Sep 21
-
Lousy chatbots are still surprisingly useful for prototyping. Wonder if Jev's structured data actually helped or hindered that process.
- Sep 20
-
i think the next step is make Jev a emoji bot... The architecture and its limitations would work well in that regime imo better then human language.
-
Given Jev does System 1 thinking, this would be equivalent to your ADHD heavy friend.
-
Sometimes I think I'm Morty speaking with the death crystal but I don't even have a death crystal and all I fear is life itself.
-
>> write me a short story> a storyI think this is the first time I’ve knowingly laughed at a model’s joke.
-
Try returning a short list of word options by lookup based on the current word completion. Would save some turns.
-
It's the digital equivalent of Morty speaking with the death crystal: https://youtu.be/YjepJlvkdKs?t=51. The crystal shows him how he will die, so he iteratively determines his speech based on whether he sees himself dying with the life he wants.