Post spreads via Hacker News and Mastodon HN-mirror bots
2 Yesterday 5:20 AM · 23h ago · 1 post · 1 source · development 2 of 2
After being submitted to Hacker News by user bashbjorn, the story was automatically relayed by Mastodon accounts that mirror HN frontpage activity.
NobodyWho Open-source AI project, author of the parody postbashbjorn Hacker News submitter
The whole story articlesposts the bright band is this development · numbered dots are the others · click one to jump
What people said 24 voices · best of 36 · verbatim
-
I also built one, but mine uses embeddings. It classifies concepts defined by a collection of positive and negative examples. The classifier model is trained in <1 second using ridge regression. The model itself is exactly the same shape as the embedding, so it works as a concept embedding. Since I already have a dataset, I can use it to do…
-
H
Jev in 25 Lines of Python Link: https://www. nobodywho.ai/posts/jev-in-25-l ines/ Discussion: https:// news.ycombinator.com/item?id=4 9812769
-
The whole point is the quantified output. If you just ask an LLM to type out its confidence "manually", it'll make up some nonsense. The logprob numbers are more reliable.I got this technique to work extremely reliably last year. However there were a bunch of caveats: 1) Firstly, you must institute a check that the multiple choice tokens dominate…
-
N
Jev in 25 Lines of Python: https://www. nobodywho.ai/posts/jev-in-25-l ines/ Discussion: http:// news.ycombinator.com/item?id=4 9812769
-
this is such a trivial thing to do in DSPY, no one bothered to give it a name…here's 7 lines import os import dspy lm = dspy.LM("openrouter/z-ai/glm-5.3-flash", api_key=os.environ["OPENROUTER_API_KEY"]) jev = dspy.Predict('email:str -> choice:Literal["Legitimate", "Spam", "Phishing"]') email = "Payroll asks for your password on a non-company…
-
In my experience as well using logprobs to try to quantify uncertainty, LLMs are a poor fit. Neural nets in general struggle with 'calibration' --- ie. if a prediction is truly 50/50, neural nets are often prone to predicting overconfidently [0].I ran some tests using GPT-4 to do some basic classification a couple years ago. On ambiguous options…
-
If we're talking about running it locally, what about passing a partial response as part of the input?Prompt part: "What is better, toast or bread?"Incomplete answer part: "The answer to this question is "and then have the LLM finish the answer. I did this with subtitle translation using llama.cpp (with Python) and had great success. Just past 5…
-
Looking at the logprobs on tokens works for the local models, but not on the frontier ones. It's been more or less broken since GPT-4o for example. I wrote about it two years ago: https://medium.com/data-science/9-11-or-9-9-which-one-is-hig.... Also, I've done some work in estimating confidence and on rubric evals using the same method, and you…
-
I missed the hypewave so can't say a lot about Jev, but the double standards are entertaining:About Jev:> We didn't train a model with Reinforcement Learning for Calibrated Decisions (RLCD) to calibrate the decisions and probabilities (even though they are not always correct).Only 99% correctness! Borderline unusable!About their model:> It…
-
Because of masked attention in LLMs, if you put the options before the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system…
-
Any analogy works at certain abstraction level, and this works with a premise that it's a classifier that makes decisions.Still good. In practice for Jev the devils in the details. As you all know by now, it's easy to write PoC and understand with AIs (or even manually, which is now a prestious practice).That demo will get you 80% thereGetting to…
-
The one thing I can't wrap my head around with Jev is why they're trying to create that "System One" narrative.In real life, a human doesn't do classification tasks with the System One part of their brain, they use System Two. So by definition what Jev does isn't System One thinking.If anything, regular programming that automatically executes…
-
Very nice as a conceptual thing, but there's a bit more to it. I've bolted Gemma 4 onto a custom pipeline for that (https://rcarmo.github.io/projects/go-system-one/) and it's OK-ish (a bit slow on my puny 3060, but I can use it to prototype a bunch of things locally until the Jev mania settles and we have better models).
-
I am hearing about Jev for the first time here so no idea about the hype. So their(Jev) is that the thing is faster at classification than a frontier model? Because the whole type safe aspect is already fully solvable with structured output. But their example is classification but that would also be possible and faster with a classic BERT model…
-
The number of - “I did/invented Jev last year”, or, “here’s a version of Jev I vibed up last night” is getting a bit ridiculous.Especially ridiculous is how the hacker news crowd seems to be taking these at face value…There was one the other day with a compelling demo. But when you looked closely at it, it was feeding in the options with the word…
-
Haha - I did enjoy this read! And there is a point to the whole marketing-dresses-up-stuff that is certainly true. I think it's worth pointing out the other HN story earlier https://news.ycombinator.com/item?id=49765348 about an open-weight model called Laya.P.S. I am evaluating that model for a production use case where I would have used Jev
-
i love that people are trying to make OS jevs but what is the point of doing all this work and not ask your coding agent to do a little benchmarking. selfishly want an open weight model to beat jev herejust found this one
-
This, like the hype of Jev on Twitter, totally ignores accuracy and generality across domains.In my experience even structured LLM output performs poorly on classifier tasks. LLMs are trained to talk and think longer. If you don't give LLM enough space to reason it would become very dumb.I'm not saying that Jev is way better, but that people way…
-
It's fast.If you're comparing with something, you need to state 'fast' in relative terms. Jev is definitely fast, and if this Python takes the same time to get a decision then it's also fast. If it's 100* slower than Jev though, you shouldn't be calling it 'fast', because relatively speaking it's really, really slow.
-
+1, llama.cpp has a --grammar parameter which you can pass a BNF style grammar file to constrain generation. It can be used in Python llama.cpp wrapper
-
Am I missing something here:p(y = next thinking+decision token | x = question) != p(y = next decision token | x = question)The former is what LLMs are trained for, the latter is what Jev was likely trained on (likely used thinking alignment as an auxiliary loss, but not explicitly included in the probability calibration).
-
Agreed, it's a real issue, but it can probably be vastly reduced by having the schema in the system prompt and by giving the model an expectation of a fixed value: no decent modern would pick a prose ligament over a provided value.To completely squash the issue, a few cheap LoRa iterations will do the trick just fine.
-
Maybe someone can explain why RL is even needed for post training with Jev? We have supervised labels.I guess it's due to the calibrated decision part (and that's what LLMs tell me).But I figure some supervised classification post training would still improve the model.
-
to be fair a Jev architecture would be better optimized for this particular workflow than an LLM. <think>\n\n</think> but letting an LLM think would trade latency and performance for significant reliability above that of Jev.
All 2 developments of NobodyWho publishes parody "Jev in 25 Lines of Python"… →
Hacker NewsMastodonNewswires