conv.

← The whole story
AI
Yale-led audit finds physics AI benchmarks are broken, not just hard

Co-author jsous publishes 'Is Physics Dead' explainer

3 Sep 17 12:14 AM · 10d ago · 1 post · 1 source · development 3 of 3

One of the paper's authors publishes a blog post walking through the study's motivation and findings, arguing that despite low raw scores on CritPt (32% for GPT-5.6 Sol and GPT-6 Astra) and Humanity's Last Exam (47%), broken grading may be masking how close frontier models actually are to matching expert physics judgment.

“Apparently, even a model good enough to be declared artificial general intelligence gets roughly half of graduate-level physics wrong.”

jsous

jsous Paper co-author and blog post authorArman Cohan Co-author of the benchmark re-grading studyYale physics faculty and graduate researchers Conducted expert audits of benchmark gradingOpenAI Developer of the evaluated frontier models

The whole story articlesposts the bright band is this development · numbered dots are the others · click one to jump

Peak 3 pieces in 3h at Sep 16, 12 PM; 11 pieces over 14 days (1 article · 3 posts · 7 comments) Sep 14, 3 AM — 1 piece · 1 post — Hacker News 1Sep 14, 6 AM — quietSep 14, 9 AM — quietSep 14, 12 PM — quietSep 14, 3 PM — quietSep 14, 6 PM — quietSep 14, 9 PM — quietSep 15, 12 AM — quietSep 15, 3 AM — quietSep 15, 6 AM — quietSep 15, 9 AM — quietSep 15, 12 PM — quietSep 15, 3 PM — quietSep 15, 6 PM — quietSep 15, 9 PM — quietSep 16, 12 AM — quietSep 16, 3 AM — quietSep 16, 6 AM — quietSep 16, 9 AM — quietSep 16, 12 PM — 3 pieces · 1 article · 1 post · 1 comment — Hacker News 2, Newswires 1Sep 16, 3 PM — 3 pieces · 3 comments — Hacker News 3Sep 16, 6 PM — 3 pieces · 3 comments — Hacker News 3Sep 16, 9 PM — 1 piece · 1 post — Hacker News 1Sep 17, 12 AM — quietSep 17, 3 AM — quietSep 17, 6 AM — quietSep 17, 9 AM — quietSep 17, 12 PM — quietSep 17, 3 PM — quietSep 17, 6 PM — quietSep 17, 9 PM — quietSep 18, 12 AM — quietSep 18, 3 AM — quietSep 18, 6 AM — quietSep 18, 9 AM — quietSep 18, 12 PM — quietSep 18, 3 PM — quietSep 18, 6 PM — quietSep 18, 9 PM — quietSep 19, 12 AM — quietSep 19, 3 AM — quietSep 19, 6 AM — quietSep 19, 9 AM — quietSep 19, 12 PM — quietSep 19, 3 PM — quietSep 19, 6 PM — quietSep 19, 9 PM — quietSep 20, 12 AM — quietSep 20, 3 AM — quietSep 20, 6 AM — quietSep 20, 9 AM — quietSep 20, 12 PM — quietSep 20, 3 PM — quietSep 20, 6 PM — quietSep 20, 9 PM — quietSep 21, 12 AM — quietSep 21, 3 AM — quietSep 21, 6 AM — quietSep 21, 9 AM — quietSep 21, 12 PM — quietSep 21, 3 PM — quietSep 21, 6 PM — quietSep 21, 9 PM — quietSep 22, 12 AM — quietSep 22, 3 AM — quietSep 22, 6 AM — quietSep 22, 9 AM — quietSep 22, 12 PM — quietSep 22, 3 PM — quietSep 22, 6 PM — quietSep 22, 9 PM — quietSep 23, 12 AM — quietSep 23, 3 AM — quietSep 23, 6 AM — quietSep 23, 9 AM — quietSep 23, 12 PM — quietSep 23, 3 PM — quietSep 23, 6 PM — quietSep 23, 9 PM — quietSep 24, 12 AM — quietSep 24, 3 AM — quietSep 24, 6 AM — quietSep 24, 9 AM — quietSep 24, 12 PM — quietSep 24, 3 PM — quietSep 24, 6 PM — quietSep 24, 9 PM — quietSep 25, 12 AM — quietSep 25, 3 AM — quietSep 25, 6 AM — quietSep 25, 9 AM — quietSep 25, 12 PM — quietSep 25, 3 PM — quietSep 25, 6 PM — quietSep 25, 9 PM — quietYesterday, 12 AM — quietYesterday, 3 AM — quietYesterday, 6 AM — quietYesterday, 9 AM — quietYesterday, 12 PM — quietYesterday, 3 PM — quietYesterday, 6 PM — quietYesterday, 9 PM — quietToday, 12 AM — quietToday, 3 AM — quietToday, 6 AM — quietToday, 9 AM — quietToday, 12 PM — quietToday, 3 PM — quietToday, 6 PM — quiet 123
1Researchers post physics benchmark…2Paper gains traction on Hacker News front…3Co-author jsous publishes 'Is Physics…
Sep 15Sep 16Sep 17Sep 18Sep 19Sep 20Sep 21Sep 22Sep 23Sep 24Sep 25now · 8:28 PM ET

What people said 3 voices · verbatim

All 3 developments of Yale-led audit finds physics AI benchmarks are broken, not… →

Hacker NewsNewswires