Discussion identifies limitations of the demonstration
3 Oct 31 4:32 PM · 11d ago · 4 comments · 1 source · development 3 of 3
Commenters point out that peaceful mode removes time-pressure constraints and that LLMs have access to all published Factorio guides and videos—advantages human players rarely have on first playthrough—making direct capability comparison difficult.
“Putting into "peaceful" mode makes response time a non-factor... Most humans, especially on first play, learn game systems and interactions from scratch, then develop novel strategies and plans and "hypothesis test" their way into beating the game.”
program_whizRyan Madden Researcher / bloggerGPT-5.6 Sol AI model under test
The whole story articlespostscomments the bright band is this development · numbered dots are the others · click one to jump
What was reported 1 claim about this development
-
first by HN Frontpage, 11d ago
What people said 9 voices · verbatim
-
One limitation of these type of demos (and similar with Minecraft, etc) is that: 1. Putting into "peaceful" mode makes response time a non-factor 2. These models have read and have access to every guide and video ever published on the subject. Most humans, especially on first play, learn game systems and interactions from scratch, then develop…
-
Honestly, I'm more interested in the harness. Having an LLM gardener that could expand outposts to connect to the main network, or just pop in to observe if there's any major problem to report could be a cool project for long-running multiplayer bases.I see it at https://github.com/ryanmadden/autorio and…
-
> The rocket launch was only achieved through significant ‘manual’ effort on Sol’s part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.Looking through the video, around 12m one can see the Sol trace report the cancellation of a conflicting 'courier'…
-
AI playing video games is a funny twist of faith. We call the AI-controlled characters NPCs, but now we've become NPCs watching AI play itself. How weird is that? At the same time this might help AI learn better - each game is a new system, a new set of rules to make sense of, each one has a unique tutorial aimed at human intellect to keep…
-
If you haven't been keeping an eye on LLMs playing games, GPT 6.1 Sol can new beat games 100% vision-only (without harnesses that give the agent extra information about the game). Pokemon is no longer a difficult challenge for it, even with additional difficulty restrictions such as hard mode and level caps. Recently, GPT 6.1 Sol beat Final…
-
Nick Winter has been streaming a game of Factorio with the major labs' bots for, I wanna say weeks now. They made it off Nauvis and are now roaming space. It's pretty interesting to follow, though the commentary's been a bit quieter as of late.
-
> Robots: they're just like us!Hah I dunno, I think most human players wouldn't find that very fun and would instead take that as an opportunity to refactor the factory for optimized performance (i.e. the core gameplay loop)
-
I bet it helps that Factorio attracts exactly the kind of obsessive players that'll leave a digital trail of training data documented in precise detailBit like warthunder guys can't help but leak classified info
-
I don't have the budget to do so, but if I do, it would be interesting to put different models in the same game with the same prompt to observe the behaviors that different models have when told to problem solve.
All 3 developments of GPT-5.6 Sol completes Factorio peaceful run in 13.5 hours →
Hacker NewsNewswires