Independent developers launch OpenJev, a browser-based comparison tool
3 Sep 18 · 5d ago · 13 posts · 38 comments · 4 sources · development 3 of 6
OpenJev let users run small open local models (e.g. MiniCPM5, Qwen3) in-browser and compare their direct-probability and generated-JSON outputs against Jev's published accuracy figures, aiming to make Jev's claims independently testable.
“jev is basically a general purpose classifier. and i think that is amazing. super curious what the training set for that looks like.”
@badlogicgames, X user · x ↗Diogo Almeida Founder, TypeSafe AInandakishor_ml Creator of Laya (open-source rival)Simon Willison Independent AI commentatorflorianstandhar Creator of JevBench
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 24 voices · best of 40 · verbatim
-
jev is basically a general purpose classifier. and i think that is amazing. super curious what the training set for that looks like.
-
The (lack of) accessibility of this website is beyond words.
-
You might also be interested in "Open-sourced jev architecture last year with model,paper and dataset"
-
- Jev is a very good, cheap, fast classifier - incorporating it lets you use more workflow, DAG, control flow primitives in your agent flows …. LangGraph
-
> see the doc https://docs.typesafe.ai/ are the docs. You can't find it from the website... > have to join a wait-list The model is available on Vercel and Openrouter without waitlist. (Waitlist wait times appear to be between a few minutes and a day.)
-
This is very interesting! Seems like a promising direction.I wonder though if it supports the same claims as Jev: answers are not impacted by other answers to the same questions, nor the existance of other questions? It seems by sharing KV cache all questions will be visible. And I think the diffusion causes the answers to attend to eachother?Also…
-
Lots of hype on this lately but I am honestly disappointed that I have to join a wait-list and still can't even see the doc to try to understand what the capabilities are and how are we supposed to use them...
-
If you want to try a _legit_ Jev implementation that matches (at least in my evals), the vLLM patch to turn DiffusionGemma into Jev is available.On my DGX Spark I get very similar latency numbers, and it matches my evals + or - a few points on each test (DG wins some, Jev wins some, both show low confidence when wrong).I ran the same evals against…
-
Thanks a million for both the insights! ... I could have probably guessed that URL in retrospect :'D
-
AI output right now is like a final exam essay response from an anxious student. Instead of being edited for focus and clarity, it's anti-edited to cram in as many details as possible. Instead of worrying that the reader might get bored or confused, it assumes that the reader has no choice but to read the whole thing, even if they get a headache…
-
I don't understand how this is different from oai "structured output" (and whatever the similar paradigm was on Sonnet ~3.7 back then) which everyone moved on from. On their gh they say:"Jev is TypeSafe's closed service for runtime-defined semantic decisions. This project reproduces that interface pattern with open models; it does not reproduce…
-
This PR is interesting but it's making the assumption that what Jev has done is based on a diffusion model or that a diffusion model is superior for this work. Which may or may not be the case.If I understand it though it does mean you can evaluate a bunch of questions simultaneously, which is an advantage.Also: While I think it's expected/normal…
-
I'm really interested in technical details behind Jev (not this), how it can work so fast and so cheap. It's probably large (must be since the performance is so good) but somehow still fast, so it must include some really non-trivial stuff. The price suggests it may be runnable locally, but who knows.If it was possible to re-create it as an…
-
I see you've edited your comment to remove the part about the vibecoded website being disrespectful towards humans. As a human I find these types of comments about the vibecoded websites, when the submission is not about the website, disrespectful.Do you have anything to say about OpenJev, which is not about the website?
-
TypeSafe also makes an adaptor available which lets you use traditional LLMs as Jev if you just want the interface without the model
-
Yeah, real Jev got really weird, no benchmarking clause. Their Terms of Use (1(v)) and MCA (2.3(f)) both prohibit users from publishing "benchmarks or performance information about the Services". No major AI has it; we are back to Oracle-style legal.Though Jev is original, it looks highly replicable.
-
I had to re-read a few times to figure out what the site was for and about. Still not sure I understand but that's the problem for them. If I'm a customer, I'm gone cause I can't figure out what it's for and I see this far too often nowadays for a lot of technical sites.
-
Jev is such a different approach where you have to be specific about what you want and which options are open. Really interesting how those things evolve in usable features for people.Also with this example the speed of new launches based on a launch is just incredible.
-
I would need proper benchmarks but in my limited testing on my Phone using Qwen 0.6b, this doesn’t work well.Between "brocoli and poop soup" or "cake", it recommends me to eat the soup.
-
Important - Jev is way too different, the greatest innovation are its speed and that it is guaranteed to NOT generate a token from a given set of tokens hence you can drive state machines intelligently.
-
It looks like a Transformer encoder post-trained on classification and regression tasks. The encoder-only model is less noticed in recent years, but this product finds a nice application for it.
-
I am not sure how this is JEV, but just a llm following the JEV api, as it is using standard LLMS. The main contribution of JEV is not the API but the model itself. Can someone please explain ?
-
Recommend partial download support and resume, otherwise this will burn through whatever mechanism is caching and serving the models if people navigate away from the page mid-download.
-
Doesn't work for me on iPad Pro: Loading…or it is just incredible slow - and I picked the smallest model…Refreshing, model still in cache, but did not help.
All 6 developments of TypeSafe AI's Jev sparks priority fight with open-source… →
Hacker NewsNewswiresMastodonXLobstersBlueskyRedditGoogle News