Report is mirrored and picked up beyond Hacker News
3 Sep 19 6:55 PM · 6d ago · 1 post · 2 comments · 2 sources · development 3 of 3
The same report text was republished/linked via a Mastodon bot account tracking HN frontpage stories, spreading the findings to a wider audience.
“Protoss: powerful and expensive frontier coding agents you directly micromanage for the toughest tasks”
mcteamster, HN commenter · hn ↗benswerd Creator of Brood War BenchOpenAI Codex AI agent tested in the benchmarkGrok 4.6 (xAI) AI agent tested in the benchmarkClaude Fable (Anthropic) AI agent tested in the benchmark
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What people said 24 voices · best of 31 · verbatim
-
G
Have trouble picking the right # benchmark about AI agents/models? also played # rts # games ~30 years ago? then you might find Brood War Bench fascinating 😜 https:// bw.
-
This is a very interesting benchmark, and I think it has a lot of potential to make the speed of a model quantifiable.I'm often asking myself is it better to use higher or lower effort levels, or to maybe drop down to a "dumber" but faster model. And so using a real-time based competition as a benchmark could shed some light on this, I think.In…
-
H
Brood War Bench Link: https:// bw.swerdlow.dev/report Discussion: https:// news.ycombinator.com/item?id=4 9766966
-
All the hobbyists were using proxybot because it allowed you to use more fun languages than C/C++ but proxybot lacked the features to effectively play Zerg. I really wanted to play Zerg so I built a really sweet API+DSL in Ruby around proxybot and then used that to give the other newbies a hard time with a zergling rush.Unfortunately the proxybot…
-
N
Brood War Bench: https:// bw.swerdlow.dev/report Discussion: http:// news.ycombinator.com/item?id=4 9766966
-
I don't know where else to write this, but I want to throw the idea out there. I have long wanted to take old broodwar televised matches, many of which are terrible quality 240p, and use machine learning to convert them to into perfect Broodwar Remastered frames. This seems tractable to me because you should be able to map the terrain sets to…
-
I love this. Funnily enough StarCraft has influenced how I approach AI at a meta levelProtoss: powerful and expensive frontier coding agents you directly micromanage for the toughest tasksTerran: versatile team comps of dedicated agent roles you can delegate well-defined tasks toZerg: massive swarms of specialist custom agents inside your apps…
-
“Older models tended to play the RTS as a turn-based game, leading them to get destroyed while they were thinking. ”That’s me. I’ve always struggled with real-time games because I need to pause and think. While I excel at chess and board games, I’m just no good at real-time ones. At last I can only manage by sticking to a fixed set of tactics for…
-
OSN did it on their youtube for their old OSL series, I think it can be done better but I think original still is much better to look at, at a corresponding resolution.The key expression for playlists or yt browsing is this `AI 업스케일`f.e.
-
I also entered the competition as an undergrad—I contacted Ben Weber 10 years later (2020) and interviewed him on my podcast:
-
This thing kept me sane through university. I had a shitty computer which could barely run this and no Internet so I just played against the computer, which was both frustrating and educational.I will always love this and now I'm going to play it again. Remastered and on a fancy modern machine.
-
I like this as a concept - taking a very real world task and checking whether it works.Surprised the outcomes are so poor though. I recall years ago AI was capable of beating pro level DOTA teams.I guess in one case it was specifically trained on the interface & game while here it was not?
-
It was the late 90s, my very first day at a new school, I was asked to introduce myself at the front of the class, I mentioned I like computers, one guy at the back of the class blurts out “En Taro Adun” and without skipping a beat I replied “J’tokoh zohl”, and we instantly became best friends.
-
I had a similar idea for Super Smash Bros Melee! There is a lot of very low-quality footage out there; surely you could train some sort of model to convert it to Slippi replays. A year or two ago this would have been a grad student project; a year or two from now, it will be a one-shot prompt.
-
Your minds will be blown when you realize just how much StarCraft is ingrained into South Korean culture. They literally had (or have) dedicated TV channels just for StarCraft.Brood War was the first video game to be broadcast on TV in Korea. I'm pretty sure it's still going.
-
Same and Brood War is starting to have a bit of a resurgence. Not anything huge. When Battle.net servers are actually working, tend to play on the ladder a few times a week. Get crushed, but still one of the greatest competitive games ever made.
-
Astra had an explicit medium/xhigh levels, Fable - just Fable. What reasoning level was used? Why not multiple were tested?I’ve scrolled the article, but haven’t noticed any remarks about Fable’s levels.
-
Battle.net servers was my hangout spot with friends eventually joint clans and playing ranked matches. Making maps to play online was also super fun. Brood War was perfect.
-
In the early days of SC2, I remember people using genetric programming to optimize build orders. I remember a slightly unorthodox Zerg Roach Rush which was _really_ fast.
-
Interesting that it benches down to Haiku but doesn't bench any Chinese models (which are at least between Sonnet and Opus, when they aren't beyond Opus).
-
Check out pluto, RL trained sc bw bot. Somehow it is now going rogue on the Korean ladder and flattening pros. There are some yt games by it, search for ^333^
-
I would love to create one of these benchmarks for age of empires 2, but I have no idea how to make the AIs play it. Maybe I can get Claude to do it anyway.
-
I thought that it was videos of the replays on the site. Was very impressed that it was a replay playing that you can scroll around in and select units.
-
Why was Luna Low so good?!? Better than Terra XHigh. Can't even say it's all APM because Sol Medium beat Sol Low (both beat Sol XHigh).
All 3 developments of Brood War Bench pits AI coding agents against each other in… →
Hacker NewsNewswiresMastodon