conv.

← The whole story
AI
Developer releases tracker showing AI model training cutoff dates and staleness

Testing reveals weak models answer outdated facts, name dead leaders

1 Sep 16 9:01 AM · 11d ago · 1 article · 1 post · 2 sources · development 1 of 2

The creator tested 16 models with web search tools over 2000 calls. Frontier models almost always correctly decided when to search, while weaker ones stated settled facts that had changed without checking. Five models named a dead man when asked who the king of Norway is.

“I gave 16 of these models a search button and asked who the king of Norway is. Five named a dead man.”

joozio

joozio Developer/tracker creator

The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump

Peak 9 pieces in 3h at Sep 16, 8 AM; 15 pieces over 12 days (1 article · 2 posts · 12 comments) Sep 16, 8 AM — 9 pieces · 1 article · 2 posts · 6 comments — Hacker News 7, Newswires 1, Mastodon 1Sep 16, 11 AM — 3 pieces · 3 comments — Hacker News 3Sep 16, 2 PM — 2 pieces · 2 comments — Hacker News 2Sep 16, 5 PM — quietSep 16, 8 PM — quietSep 16, 11 PM — quietSep 17, 2 AM — quietSep 17, 5 AM — quietSep 17, 8 AM — quietSep 17, 11 AM — quietSep 17, 2 PM — quietSep 17, 5 PM — quietSep 17, 8 PM — quietSep 17, 11 PM — quietSep 18, 2 AM — quietSep 18, 5 AM — quietSep 18, 8 AM — quietSep 18, 11 AM — 1 piece · 1 comment — Hacker News 1Sep 18, 2 PM — quietSep 18, 5 PM — quietSep 18, 8 PM — quietSep 18, 11 PM — quietSep 19, 2 AM — quietSep 19, 5 AM — quietSep 19, 8 AM — quietSep 19, 11 AM — quietSep 19, 2 PM — quietSep 19, 5 PM — quietSep 19, 8 PM — quietSep 19, 11 PM — quietSep 20, 2 AM — quietSep 20, 5 AM — quietSep 20, 8 AM — quietSep 20, 11 AM — quietSep 20, 2 PM — quietSep 20, 5 PM — quietSep 20, 8 PM — quietSep 20, 11 PM — quietSep 21, 2 AM — quietSep 21, 5 AM — quietSep 21, 8 AM — quietSep 21, 11 AM — quietSep 21, 2 PM — quietSep 21, 5 PM — quietSep 21, 8 PM — quietSep 21, 11 PM — quietSep 22, 2 AM — quietSep 22, 5 AM — quietSep 22, 8 AM — quietSep 22, 11 AM — quietSep 22, 2 PM — quietSep 22, 5 PM — quietSep 22, 8 PM — quietSep 22, 11 PM — quietSep 23, 2 AM — quietSep 23, 5 AM — quietSep 23, 8 AM — quietSep 23, 11 AM — quietSep 23, 2 PM — quietSep 23, 5 PM — quietSep 23, 8 PM — quietSep 23, 11 PM — quietSep 24, 2 AM — quietSep 24, 5 AM — quietSep 24, 8 AM — quietSep 24, 11 AM — quietSep 24, 2 PM — quietSep 24, 5 PM — quietSep 24, 8 PM — quietSep 24, 11 PM — quietSep 25, 2 AM — quietSep 25, 5 AM — quietSep 25, 8 AM — quietSep 25, 11 AM — quietSep 25, 2 PM — quietSep 25, 5 PM — quietSep 25, 8 PM — quietSep 25, 11 PM — quietYesterday, 2 AM — quietYesterday, 5 AM — quietYesterday, 8 AM — quietYesterday, 11 AM — quietYesterday, 2 PM — quietYesterday, 5 PM — quietYesterday, 8 PM — quietYesterday, 11 PM — quietToday, 2 AM — quietToday, 5 AM — quietToday, 8 AM — quietToday, 11 AM — quietToday, 2 PM — quietToday, 5 PM — quietToday, 8 PM — quiet 1–2
1–2Testing reveals weak models answer…
Sep 17Sep 18Sep 19Sep 20Sep 21Sep 22Sep 23Sep 24Sep 25yesterdaynow · 9:49 PM ET

Reported in the same hours no headline names this development itself — these 1 claim were published in its stretch

  1. 1 outlet Show HN: How Stale Is Your AI? Release age and training cutoff for 20 models

    first by HN Frontpage, 11d ago

All 2 developments of Developer releases tracker showing AI model training cutoff… →

Hacker NewsNewswiresMastodon