Commenter questions scaffolding and suggests pure reasoning test
5Yesterday 7:25 PM · 17h ago · 1 post · 2 comments · 2 sources · development 5 of 6
A reviewer noted the project includes significant guidance (pathfinding, textual milestones), making it hard to assess true capability. They proposed combining Jev with standard vLLM and testing it on a model with no prior Pokémon knowledge to see its reasoning process.
“Cool project, comes with a little too much guidance in the harness though IMO (pathfinding, textual milestones etc).”
ac2u
pancomplexDeveloper, project authorJevSmall language model
The whole story articlespostscommentsthe bright band is this development · numbered dots are the others · click one to jump
Cool project, comes with a little too much guidance in the harness though IMO (pathfinding, textual milestones etc). (The author is very upfront about this in their README though)I think if it was combined with a regular vLLM it could be really interesting, especially watching the reasoning logs.Bonus points if it was one of the latest open models…
Jev is for single shot classification, not multi-step RL environments with delayed reward and explore/exploit. My guess is it would go through the door with high confidence every time unless you change the input to add the history.
H