Commenters cite hardware benchmarks and provider economics as evidence
5 Yesterday 2:06 AM · 23h ago · 5 comments · 1 source · development 5 of 5
Other commenters counter the skepticism by pointing to independent benchmark data on cost-per-intelligence and to third-party inference providers profitably serving open-source models on commodity GPUs, arguing the efficiency gains are grounded in hardware trends rather than marketing.
“This isn't about marketing claims, this is just basic hardware benchmarking. The inference hardware is getting more efficient per watt.”
timthelionjyn.dev (blog author) Author of the essayroflmaoqwerty Lobsters commentergulbanana Lobsters commentertimthelion Lobsters commenter
The whole story postscomments the bright band is this development · numbered dots are the others · click one to jump
Reported in the same hours no headline names this development itself — these 1 claim were published in its stretch
-
1 outlet Tokens Too Cheap to Meter
first by HN Frontpage, 20h ago
What people said 24 voices · best of 39 · verbatim
-
Because there are third party inference providers serving large open source models without large investments and they seem to make s profit.
-
A few years is hardly forever and the state of the world here indicates a lot of low hanging fruit still exists.An LLM can certainly be cheaper than grep, because it’s an approximation, while a grep is deterministic and must examine every byte in what can be a relatively complex state machine for a regex based grep. There are other scales to…
-
H
Tokens Too Cheap to Meter L: https:// jyn.dev/tokens-too-cheap-to-me ter/ C: https:// news.ycombinator.com/item?id=4 9813482 posted on 2026.09.23 at 05:21:17 (c=0, p=7)
-
Check how similar the cost of Deepseek V4.1 Flash is around the world on different providers. Calculate how many users you can serve with 4xH200. And the quality is around GPT 5.6 Sol for programming and agentic tasks.
-
I just want to rant about these Artificial Analysis charts that you see everywhere:The "most attractive quadrant" is completely meaningless. The whole point of a Pareto curve is that each point on the curve is better than everything else on at least one dimension, and that you can make these comparisons without placing a value judgement on the…
-
N
Tokens too cheap to meter: https:// jyn.dev/tokens-too-cheap-to-me ter/ Discussion: http:// news.ycombinator.com/item?id=4 9813482
-
I guess my real question is how do we know the cost per intelligence isn't being heavily subsidized by the labs?
-
It's true that LLMs "want" to be be local, but they won't shift broadly to being local until there's a sufficiently large supply of VRAM or (at least) "unified" memory from the manufacturers. (I'm also assuming here that radical regulatory changes like government bans of local models aren't going to happen.) So (AFAICS—I am no expert) the future…
-
Open weight models can be self-hosted on your own hardware, with your own electricity, and you can calculate your own unsubsidised price per token. This used to be less meaningful when the frontier closed labs were far ahead of toy open models, but this year we've got large open weight models that are competitive with the best closed models from…
-
I'm always reminded on Orwell's quote about the then new atomic bomb and his prescience on how it would all work out:"Had the atomic bomb turned out to be something as cheap and easily manufactured as a bicycle or an alarm clock, it might well have plunged us back into barbarism, but it might, on the other hand, have meant the end of national…
-
This isn't what "subsidized" means. I think we can make some useful analysis about how much inference actually costs some providers, but I *don't* think it's unreasonable to say that we have very little idea how much inference costs OpenAI and Anthropic. It's entirely possible that they are asking customers for less than it costs them to serve…
-
A much deeper analysis on the falling price per task was published yesterday by Epoch AI [1]. It's a real statistical analysis and comes to more defensible and grounded conclusions. The headline takeaway is:The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art…
-
This isn't about marketing claims, this is just basic hardware benchmarking. The inference hardware is getting more efficient per watt. This is not surprising. Everything is getting more efficient per watt over time.
-
I found the OP insightful and worth a read. Thank you for sharing it on HN.The only aspect that is poorly analyzed by the OP is business model viability. All players are investing insane amounts of money in infrastructure with the expectation that their future profits will justify all that investment. The winner or winners in the AGI race, they…
-
"Computing courses got started in universities largely because IBM donated about 100 ‘free’ computers during the 1950s, with the stipulation that computer courses must be taught. This strategy made it possible for computing to get a foot in the academic door." - Donald Knuth Businesses frequently do things and operate at a loss in order to enter…
-
> Tokens become cheaper than tool callsThe author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop."…
-
Ah, I don't know the economics of it well enough to answer. A piece of evidence we can trust is some local LLM users get value, and that's with less cost-efficient software, utilisation, cooling, and power. That only speaks to inference though. I have no idea about training.
-
I agree with OP that we will continue to see improvements, but there are also some serious bottlenecks ahead of us:- Energy is not infinite, neither energy efficiency is. - Datacentres neither. - Benchmarks are an abstraction of real world problems!On top, there is an overall "economic" aspect that most of the people miss: every change carries a…
-
Of course it is, at least in the same way your laptop is heavily subsidized by IBM and Casio approximately 50 years ago.
-
4-5 orders of magnitude is huge. Assuming an order of base 10, it's 10000x-100000x. So a call to grep may return in 1s on a typical PC. That means a GPT call takes equivalent energy of 10000-100000 PCs to do the same in 1s. That's a difference that can't be equalized with scaling. It would require a revolutionary breakthrough.I also don't…
-
I don't know about the malleable software claim. Sure, people can build out their own thing. Malleable software in itself requires software that is designed to be malleable in the first place.And perhaps people will not be willing to accept the initial friction that malleable software brings (see people who complain about Emacs or Salesforce or…
-
> NVIDIA will still boomI think Nvidia is under the same pressure as Anthropic/OpenAI. Nvidia will dominate research and probably keep dominating training, but the real volume is in inference. And for inference Nvidia's lead is only a few months, similar to the lead frontier labs have over open source. Nvidia will sell a lot of Rubin CPX's, but…
-
Too Cheap to Meter reminds me of the promise of Nuclear Power in 1954"It is not too much to expect that our children will enjoy in their homes electrical energy too cheap to meter,..." Lewis Strausshttps://en.wikipedia.org/wiki/Too_cheap_to_meter#OriginsOddly enough my power bill was metered and big.
-
Well, think that statement through a bit:Grep reads through the entire file looking for patterns.An LLM scans its neural net (in ways that I don't understand) which is kinda-sorta like having a huge index.You can improve over Grep if you have an index; and the LLM has an index.Thus, it's plausible that an LLM can be more efficient at reading its…
All 5 developments of Essay "Tokens Too Cheap to Meter" sparks debate over AI… →
Hacker NewsLobstersMastodonNewswires