GPT-5.6 Sol completes Factorio peaceful run in 13.5 hours
Researcher Ryan Madden demonstrates OpenAI's latest model autonomously playing and beating the factory-building game.
What to know
- GPT-5.6 Sol completed a full Factorio peaceful-mode playthrough in 13.5 hours, showing improvement over GPT-5.5 but revealing persistent weaknesses in late-game scaling and transport belt logic.
- The model relies on manual resource ferrying and chest-based inventory rather than optimized belt routing—the core mechanic human players prioritize—suggesting incomplete understanding of Factorio's systems.
- Commenters debate whether the achievement is directly comparable to human play, noting that peaceful mode removes time pressure and LLMs have access to all published guides and videos.
The dispute Whether Sol's ability to complete Factorio demonstrates genuine problem-solving capability or merely skilled synthesis of training data, and what hidden constraints (peaceful mode, information access) mean for fair comparison with human performance. · positions read across 15 posts and comments
The demonstration reveals genuine AI reasoning but exposes core gaps in mechanical understanding.
-
“It doesn't seem to understand the core mechanic of Factorio, which is putting outputs onto belts and routing them to other inputs.”
fasterik · Hacker News ↗
The comparison is unfair because peaceful mode and training data access eliminate constraints humans face.
-
“Putting into "peaceful" mode makes response time a non-factor... Most humans, especially on first play, learn game systems and interactions from scratch, then develop novel strategies and plans and "hypothesis test" their way into beating…”
program_whiz · Hacker News ↗
The work is valuable for understanding AI behavior and has real-world applications beyond gaming.
-
“I'm optimistic about autonomous Factorio play as a medium for building intuition about the nature of model capabilities.”
Ryan Madden · ryanmadden.net ↗
Ryan Madden Researcher / bloggerGPT-5.6 Sol AI model under test
How it unfolded 3 developments, newest first · click a bar or a number to jump articlespostscomments
-
3
Discussion identifies limitations of the demonstration
Commenters point out that peaceful mode removes time-pressure constraints and that LLMs have access to all published Factorio guides and videos—advantages human players rarely have on first playthrough—making direct capability comparison difficult.
“Putting into "peaceful" mode makes response time a non-factor... Most humans, especially on first play, learn game systems and interactions from scratch, then develop novel strategies and plans and "hypothesis test" their way into beating the game.”
— program_whiz -
first by HN Frontpage, 11d ago
-
One limitation of these type of demos (and similar with Minecraft, etc) is that: 1. Putting into "peaceful" mode makes response time a non-factor 2. These models have read and have access to every guide and video ever published on the subject. Most humans, especially on first play, learn game systems and interactions from scratch, then develop…
2 more of the top 3 · 9 posts in this stretch
-
Honestly, I'm more interested in the harness. Having an LLM gardener that could expand outposts to connect to the main network, or just pop in to observe if there's any major problem to report could be a cool project for long-running multiplayer bases.I see it at https://github.com/ryanmadden/autorio and…
-
> The rocket launch was only achieved through significant ‘manual’ effort on Sol’s part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.Looking through the video, around 12m one can see the Sol trace report the cancellation of a conflicting 'courier'…
-
-
2
Commenters highlight Sol's unconventional factory design
Hacker News discussion emerged with multiple commenters noting that Sol's approach—relying on chests and manual insertion rather than belt routing—differs markedly from typical human strategies and raises questions about whether the model truly grasps Factorio's core mechanics.
“It doesn't seem to understand the core mechanic of Factorio, which is putting outputs onto belts and routing them to other inputs. It does this in a few places, but for the most part everything is put into chests and moved around by the player.”
— fasterik · source -
It is weird and almost alien how AI designs factories. When Claude announced Fable 5 they also did a Factorio demo that was the same kind of designs and a lot of hand feeding and direct insertion. In some sense it proves to me that llms are not just copying and are actually creating a bespoke solution. But I honestly expected more from frontier…
2 more of the top 3 · 6 posts in this stretch
-
> The rocket launch was only achieved through significant ‘manual’ effort on Sol’s part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.Robots: they're just like us!
-
It doesn't seem to understand the core mechanic of Factorio, which is putting outputs onto belts and routing them to other inputs. It does this in a few places, but for the most part everything is put into chests and moved around by the player.
-
-
1
Sol shows improvement over GPT-5.5 but faces scaling limits
Madden noted that Sol surpassed GPT-5.5's performance (which failed to use transport belts) but encountered scaling issues in the blue science stage that slowed progress significantly. The final rocket launch required extensive manual effort by Sol, repeatedly ferrying supplies between disconnected factory areas rather than optimizing production systems.
“The rocket launch was only achieved through significant 'manual' effort on Sol's part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.”
— Ryan Madden -
background
Madden publishes GPT-5.6 Sol's Factorio completion — Ryan Madden released results showing GPT-5.6 Sol (running in Codex with low thinking level) successfully completing a peaceful Factorio run in 13.5 hours of game time using a Python game manager that enforces human-like constraints such as player reach and fog of war.
What people are saying 9 voices from 1 site · best of 15 · verbatim
- Nov 1
-
AI playing video games is a funny twist of faith. We call the AI-controlled characters NPCs, but now we've become NPCs watching AI play itself. How weird is that? At the same time this might help AI learn better - each game is a new system, a new set of rules to make sense of, each one has a unique tutorial aimed at human intellect to keep…
- Oct 31
-
Nick Winter has been streaming a game of Factorio with the major labs' bots for, I wanna say weeks now. They made it off Nauvis and are now roaming space. It's pretty interesting to follow, though the commentary's been a bit quieter as of late.
-
I bet it helps that Factorio attracts exactly the kind of obsessive players that'll leave a digital trail of training data documented in precise detailBit like warthunder guys can't help but leak classified info
-
I don't have the budget to do so, but if I do, it would be interesting to put different models in the same game with the same prompt to observe the behaviors that different models have when told to problem solve.
-
If you haven't been keeping an eye on LLMs playing games, GPT 6.1 Sol can new beat games 100% vision-only (without harnesses that give the agent extra information about the game). Pokemon is no longer a difficult challenge for it, even with additional difficulty restrictions such as hard mode and level caps. Recently, GPT 6.1 Sol beat Final…
-
> Robots: they're just like us!Hah I dunno, I think most human players wouldn't find that very fun and would instead take that as an opportunity to refactor the factory for optimized performance (i.e. the core gameplay loop)
-
i think its more of an effect of factorio's hidden information.youre not really sure how to set up some of the logistics/how much of what you will need where ahead of time
-
Watching AI play games is fun, but I can't wait for someone to crack having small LLMs in games to enrich them in useful and interesting ways.
-
Factorio is a much easier game without the bugs being a pain in the arse as you expand - i wonder how Sol would handle the bugs