conv.

← The whole story
AI
GoBench benchmark measures LLM reasoning on 9x9 Go against KataGo

Roland31415 responds that Go is out-of-distribution if labs did not specifically train on it

4 Sep 16 4:39 PM · 11d ago · 1 comment · 1 source · development 4 of 4

The creator responds that Go and ARC-AGI are similar as out-of-distribution tasks, and argues GoBench can only measure general reasoning if frontier labs did not specifically train on Go. He notes Go data might exist slightly in pretraining but shouldn't affect results much.

Roland31415 responds that Go is out-of-distribution if labs did not specifically train on it
x.com

“It is true that GoBench can only measure general reasoning if the frontier labs do not specifically train on Go.”

Roland31415

Roland31415 GoBench creator

The whole story postscomments the bright band is this development · numbered dots are the others · click one to jump

Peak 3 pieces in 3h at Sep 16, 2 PM; 4 pieces over 12 days (2 posts · 2 comments) Sep 16, 11 AM — 1 piece · 1 post — X 1Sep 16, 2 PM — 3 pieces · 1 post · 2 comments — Reddit 3Sep 16, 5 PM — quietSep 16, 8 PM — quietSep 16, 11 PM — quietSep 17, 2 AM — quietSep 17, 5 AM — quietSep 17, 8 AM — quietSep 17, 11 AM — quietSep 17, 2 PM — quietSep 17, 5 PM — quietSep 17, 8 PM — quietSep 17, 11 PM — quietSep 18, 2 AM — quietSep 18, 5 AM — quietSep 18, 8 AM — quietSep 18, 11 AM — quietSep 18, 2 PM — quietSep 18, 5 PM — quietSep 18, 8 PM — quietSep 18, 11 PM — quietSep 19, 2 AM — quietSep 19, 5 AM — quietSep 19, 8 AM — quietSep 19, 11 AM — quietSep 19, 2 PM — quietSep 19, 5 PM — quietSep 19, 8 PM — quietSep 19, 11 PM — quietSep 20, 2 AM — quietSep 20, 5 AM — quietSep 20, 8 AM — quietSep 20, 11 AM — quietSep 20, 2 PM — quietSep 20, 5 PM — quietSep 20, 8 PM — quietSep 20, 11 PM — quietSep 21, 2 AM — quietSep 21, 5 AM — quietSep 21, 8 AM — quietSep 21, 11 AM — quietSep 21, 2 PM — quietSep 21, 5 PM — quietSep 21, 8 PM — quietSep 21, 11 PM — quietSep 22, 2 AM — quietSep 22, 5 AM — quietSep 22, 8 AM — quietSep 22, 11 AM — quietSep 22, 2 PM — quietSep 22, 5 PM — quietSep 22, 8 PM — quietSep 22, 11 PM — quietSep 23, 2 AM — quietSep 23, 5 AM — quietSep 23, 8 AM — quietSep 23, 11 AM — quietSep 23, 2 PM — quietSep 23, 5 PM — quietSep 23, 8 PM — quietSep 23, 11 PM — quietSep 24, 2 AM — quietSep 24, 5 AM — quietSep 24, 8 AM — quietSep 24, 11 AM — quietSep 24, 2 PM — quietSep 24, 5 PM — quietSep 24, 8 PM — quietSep 24, 11 PM — quietSep 25, 2 AM — quietSep 25, 5 AM — quietSep 25, 8 AM — quietSep 25, 11 AM — quietSep 25, 2 PM — quietSep 25, 5 PM — quietSep 25, 8 PM — quietSep 25, 11 PM — quietYesterday, 2 AM — quietYesterday, 5 AM — quietYesterday, 8 AM — quietYesterday, 11 AM — quietYesterday, 2 PM — quietYesterday, 5 PM — quietYesterday, 8 PM — quietYesterday, 11 PM — quietToday, 2 AM — quietToday, 5 AM — quietToday, 8 AM — quietToday, 11 AM — quietToday, 2 PM — quietToday, 5 PM — quietToday, 8 PM — quiet 1–4
1–4Roland31415 responds that Go is…
Sep 17Sep 18Sep 19Sep 20Sep 21Sep 22Sep 23Sep 24Sep 25yesterdaynow · 11:09 PM ET

What people said 1 voice · verbatim

All 4 developments of GoBench benchmark measures LLM reasoning on 9x9 Go against… →

RedditX