conv.

← The whole story
AI
Yale-led audit finds physics AI benchmarks are broken, not just hard

Researchers post physics benchmark re-grading study on arXiv

1 Sep 14 4:07 AM · 13d ago · 1 post · 1 source · development 1 of 3

A paper titled 'How Good Are Frontier Models at Physics? Expert Re-Grading Reveals Broken Evaluations and Near-Saturation of Leading Benchmarks' appears on arXiv, reporting that Yale physics faculty and graduate researchers manually audited how frontier models were scored on physics benchmarks.

jsous Paper co-author and blog post authorArman Cohan Co-author of the benchmark re-grading studyYale physics faculty and graduate researchers Conducted expert audits of benchmark gradingOpenAI Developer of the evaluated frontier models

The whole story articlesposts the bright band is this development · numbered dots are the others · click one to jump

Peak 3 pieces in 3h at Sep 16, 12 PM; 11 pieces over 14 days (1 article · 3 posts · 7 comments) Sep 14, 3 AM — 1 piece · 1 post — Hacker News 1Sep 14, 6 AM — quietSep 14, 9 AM — quietSep 14, 12 PM — quietSep 14, 3 PM — quietSep 14, 6 PM — quietSep 14, 9 PM — quietSep 15, 12 AM — quietSep 15, 3 AM — quietSep 15, 6 AM — quietSep 15, 9 AM — quietSep 15, 12 PM — quietSep 15, 3 PM — quietSep 15, 6 PM — quietSep 15, 9 PM — quietSep 16, 12 AM — quietSep 16, 3 AM — quietSep 16, 6 AM — quietSep 16, 9 AM — quietSep 16, 12 PM — 3 pieces · 1 article · 1 post · 1 comment — Hacker News 2, Newswires 1Sep 16, 3 PM — 3 pieces · 3 comments — Hacker News 3Sep 16, 6 PM — 3 pieces · 3 comments — Hacker News 3Sep 16, 9 PM — 1 piece · 1 post — Hacker News 1Sep 17, 12 AM — quietSep 17, 3 AM — quietSep 17, 6 AM — quietSep 17, 9 AM — quietSep 17, 12 PM — quietSep 17, 3 PM — quietSep 17, 6 PM — quietSep 17, 9 PM — quietSep 18, 12 AM — quietSep 18, 3 AM — quietSep 18, 6 AM — quietSep 18, 9 AM — quietSep 18, 12 PM — quietSep 18, 3 PM — quietSep 18, 6 PM — quietSep 18, 9 PM — quietSep 19, 12 AM — quietSep 19, 3 AM — quietSep 19, 6 AM — quietSep 19, 9 AM — quietSep 19, 12 PM — quietSep 19, 3 PM — quietSep 19, 6 PM — quietSep 19, 9 PM — quietSep 20, 12 AM — quietSep 20, 3 AM — quietSep 20, 6 AM — quietSep 20, 9 AM — quietSep 20, 12 PM — quietSep 20, 3 PM — quietSep 20, 6 PM — quietSep 20, 9 PM — quietSep 21, 12 AM — quietSep 21, 3 AM — quietSep 21, 6 AM — quietSep 21, 9 AM — quietSep 21, 12 PM — quietSep 21, 3 PM — quietSep 21, 6 PM — quietSep 21, 9 PM — quietSep 22, 12 AM — quietSep 22, 3 AM — quietSep 22, 6 AM — quietSep 22, 9 AM — quietSep 22, 12 PM — quietSep 22, 3 PM — quietSep 22, 6 PM — quietSep 22, 9 PM — quietSep 23, 12 AM — quietSep 23, 3 AM — quietSep 23, 6 AM — quietSep 23, 9 AM — quietSep 23, 12 PM — quietSep 23, 3 PM — quietSep 23, 6 PM — quietSep 23, 9 PM — quietSep 24, 12 AM — quietSep 24, 3 AM — quietSep 24, 6 AM — quietSep 24, 9 AM — quietSep 24, 12 PM — quietSep 24, 3 PM — quietSep 24, 6 PM — quietSep 24, 9 PM — quietSep 25, 12 AM — quietSep 25, 3 AM — quietSep 25, 6 AM — quietSep 25, 9 AM — quietSep 25, 12 PM — quietSep 25, 3 PM — quietSep 25, 6 PM — quietSep 25, 9 PM — quietYesterday, 12 AM — quietYesterday, 3 AM — quietYesterday, 6 AM — quietYesterday, 9 AM — quietYesterday, 12 PM — quietYesterday, 3 PM — quietYesterday, 6 PM — quietYesterday, 9 PM — quietToday, 12 AM — quietToday, 3 AM — quietToday, 6 AM — quietToday, 9 AM — quietToday, 12 PM — quietToday, 3 PM — quietToday, 6 PM — quiet 123
1Researchers post physics benchmark…2Paper gains traction on Hacker News front…3Co-author jsous publishes 'Is Physics…
Sep 15Sep 16Sep 17Sep 18Sep 19Sep 20Sep 21Sep 22Sep 23Sep 24Sep 25now · 8:27 PM ET

What was reported 1 claim about this development

  1. 1 outlet How good are frontier models at physics?

    first by HN Frontpage, 11d ago

All 3 developments of Yale-led audit finds physics AI benchmarks are broken, not… →

Hacker NewsNewswires