Open-weight AI models trigger year-long security crisis
Cheap, abliterated LLMs like GLM 5.3-flash can find vulnerabilities faster than humans can patch them.
What to know
- Open-weight GLM 5.3-flash, now stripped of safety guardrails by third parties, scores 84.5% on real-world vulnerability benchmarks and is runnable on a ~$9,500 Mac; security researchers say this puts dangerous AI exploit generation in reach of any attacker.
- The industry faces a compressed deadline—estimates range from one year to already-negative (attackers are exploiting now)—to patch systemic vulnerabilities before LLM-assisted mass exploitation becomes the norm.
- Consumer operating systems lack usable sandboxing; the conversation reveals no consensus tool exists that combines both security hardening and ease of use, leaving practitioners to choose between obscure/undocumented (Apple Sandbox), difficult to configure securely (gVisor, Kubernetes), or hard to deploy (Firecracker).
- Long-term fix proposed: capability-based security models (already used in Android/iOS pickers) replacing identity-based process authorization, but adoption requires significant UI and cultural redesign.
The dispute Whether the threat timeline is one year (researchers' starting position) or already negative (some believe exploitation is underway but hidden from public view). · positions read across 57 posts and comments
Operating-system sandboxing is the critical missing layer; capability-based security should replace identity-based models.
-
“I would like to be able to run a process in a way that prevents it from digging around in my home directory and exfiltrating anything it finds to someone else over the internet.”
simonw · Lobsters ↗
The one-year timeline is already too optimistic; black-hat exploitation is underway now and not visible in public reporting.
-
“We don't even have a year. I ran a security audit using GPT-6 Astra and Claude Fable 5.1 against my main open source project (Datasette) recently and then spent a full *week* fixing vulnerabilities.”
simonw · Lobsters ↗
No existing sandboxing tool is both secure-by-default and easy to use; every option trades off security hardening against usability.
-
“My problem with sandboxes is that what I really want is an easy-to-use sandbox from a company with a dedicated security team that works on that sandbox product, and who risk millions (ideally billions) of dollars if it leaks.”
simonw · Lobsters ↗
Urgent security response is necessary and possible; LLMs finding vulnerabilities faster than humans is actually pushing the industry toward taking security seriously.
-
“We are definitely at the time in which cheap / unaligned / locally-runnable AI models are *excellent* at finding vulnerabilities and crafting exploit scripts, and they're only going to get better. That does suck in some regards, but I…”
apropos · Lobsters ↗
apropos (author of Datasette) Security researcher, post authorSimon Willison (simonw) Datasette maintainer, open-source security expertZ.ai Co. (formerly Zhipu AI) AI model developerjfred (commenter) Security architecture discussantDeAlignAI Model abliteration group
How it unfolded 5 developments, newest first · click a bar or a number to jump postscomments
-
background
Security experts debate capability-based OS architecture — Commenters propose capability-based security models (requiring processes to hold explicit authority tokens rather than inheriting user identity) as a long-term fix, though they acknowledge UI challenges. References to government compartmentalization practices (SCIF model) and emerging tools like gVisor, Firecracker, and bubblewrap suggest incremental progress.
-
5
Researcher claims threat timeline may be even shorter
A commenter argues the actual window is negative—that black-hat exploitation is already underway in the wild and simply not making headlines, suggesting the industry is not merely behind schedule but already under active attack.
“I think it's even worse than that, it's less "we don't even have a year" and more "we have negative six months". Black hats are absolutely going ham in the wild and just not making the news.”
— wareya -
I think it's even worse than that, it's less "we don't even have a year" and more "we have negative six months". Black hats are absolutely going ham in the wild and just not making the news.
2 more of the top 3 · 32 posts in this stretch
-
Attacker level of effort has been the only thing protecting anyone for the last 40 years, but if money is not an object then all other factors in "attacker level of effort" will approach *zero*. We don't need a Judgement Day scenario or genius-hacker level models for that to be disastrous.
-
"There is an exploitable security vulnerability in $TARGET. Please find it and give me a reproducible test" When doing so, $TARGET can be either a file name or a function, which allows you to loop / parallelize over the codebase. If you hunt for specific vulnerabilities, it's best to enable the LLM to check its work by providing a verifier. For…
-
-
4
Community debate over practical sandboxing solutions emerges
Thread identifies multiple real-world sandboxing approaches (gVisor donated to CNCF, Firecracker, Kata Containers, bubblewrap, Apple Sandbox) but reveals deep tensions: every tool trades security hardening against usability, and many are either undocumented (Apple Sandbox), hard to configure securely (gVisor, Kubernetes), or difficult to deploy (Firecracker). No consensus emerges on what practitioners should use now.
“My problem with sandboxes is that what I really want is an easy-to-use sandbox from a company with a dedicated security team that works on that sandbox product, and who risk millions (ideally billions) of dollars if it leaks.”
— simonw -
I've been spending quite a lot of time fixing llm-reported vulnerabilities this year, especially in the last 3 or 4 months. Hopefully at some point, chromium will just "not have bugs", and if introduced they get found quickly before they hit stable. It's a lot of work getting there though lol.
2 more of the top 3 · 11 posts in this stretch
-
UI design is definitely a challenge, yeah; capabilities have been more obscure for a while and haven't had as much UI work as more common architectures. Though for what it's worth, things like the file/photo pickers in Android and iOS are capability-style powerboxes that people use every day. I think it's possible to expand on this, but it will…
-
I really love [bubblewrap](https://github.com/containers/bubblewrap) for this on Linux. It really does make it simple to run a single command with controlled permissions. You can share and unshare specific parts of the filesystem, as well as control network access with a simple wrapper script. Linux has some seriously powerful tools to do this…
-
-
3
Community identifies sandboxing as critical missing defense
Discussion shifts to operating system sandboxing as an inadequate defense. Multiple researchers argue that consumer OSes lack usable, documented sandboxing—and that capability-based security models (used in Android and iOS pickers but rarely elsewhere) should replace the identity-based model where processes run "as you" with full authority.
“I would like to be able to run a process in a way that prevents it from digging around in my home directory and exfiltrating anything it finds to someone else over the internet. This is way harder than it should be.”
— simonw -
I would like to see a whole lot more attention paid to sandboxing. It infuriates me that consumer operating systems don't ship with clearly documented, usable sandboxing features - the sandboxes they include today may as well have signs pasted on them saying "Beware of the Leopard". I would like to be able to run a process in a way that prevents…
2 more of the top 3 · 11 posts in this stretch
-
I remain convinced that the right answer for desktop OSes in the long term (albeit not necessarily one that's easy to get to from here) will have to involve [capabilities](http://habitat-chronicles.com/2017/05/what-are-capabilities/comment-page-1/). The identity-based model of processes that run "as you" with all your authority hasn't really fit…
-
If we can figure out how to wrap that in a UI that doesn't require half a degree in cybersecurity to use safely and effectively I'm all for it!
-
-
2
Researcher reports immediate vulnerability flood from LLM audits
Datasette maintainer Simon Willison, citing a recent security audit of his project using GPT-6 Astra and Claude Fable 5.1, reports spending a full week patching extremely obscure vulnerabilities that LLMs discovered—vulnerabilities that had gone unspotted by humans for extended periods. He contradicts the one-year timeline, stating "We don't even have a year."
“I ran a security audit using GPT-6 Astra and Claude Fable 5.1 against my main open source project (Datasette) recently and then spent a full *week* fixing vulnerabilities that they found, many of which were extremely obscure, hence why nobody had spotted them before.”
— simonw -
We don't even have a year. I ran a security audit using GPT-6 Astra and Claude Fable 5.1 against my main open source project (Datasette) recently and then spent a full *week* fixing vulnerabilities that they found, many of which were extremely obscure, hence why nobody had spotted them before. An obscure vulnerability is still a vulnerability…
1 more of the top 2 · 2 posts in this stretch
-
I’m in the middle of reading a book on the stuxnet attack and one thing that really stood out to me is that at least two of the zero days had been publicly published or disclosed for a year or more.
-
-
1
Researcher publishes vulnerability timeline warning
Security researcher and author of Datasette posts analysis showing that abliterated GLM 5.3-flash is affordable to run locally (30–45 tokens/second on the upcoming M5 Mac Studio for ~$9,500) and scores 84.5% on CyberGym (real-world vulnerabilities) and 54.4% on ExploitBench (exploit generation). The post warns the industry has roughly one year to fix systemic security flaws before cheap, unaligned models enable mass exploitation.
“We have a year to fix security everywhere... We can use frontier LLMs that move faster than a human to find and fix these issues in the time we have left.”
— apropos -
I think this is a very interesting post and worthy of discussion. For my part, I agree with it, but only partially. We are definitely at the time in which cheap / unaligned / locally-runnable AI models are *excellent* at finding vulnerabilities and crafting exploit scripts, and they're only going to get better. That does suck in some regards, but…
-
-
background
Z.ai releases GLM 5.3-flash model — Z.ai (formerly Zhipu AI) releases the General Language Model 5.3-flash as an open-weight model. When hosted by Z.ai it includes task refusals required by law, but organizations like DeAlignAI immediately release abliterated versions with safety mechanisms surgically removed.
What people are saying 11 voices from 1 site · best of 57 · verbatim
- Which sandboxing tool should practitioners use right now that is both secure and practical to deploy?
- How much of the LLM exploit generation happening today is already being weaponized in the wild versus remaining theoretical?
- Sep 20
-
I used a bunch, but one that worked particularly well was "Review the security issues that have already been fixed in this project, then audit the code for similar issues". I used GPT-6 Astra Ultra for that one, which fired off a bunch of auditing subagents.
-
The main thing here is that, as an industry, we haven't taken security seriously at all, and most attempts at security these days seem to just be excuses to lock people out of hardware (& software) they are supposed to own. (Such as Google Play Integrity and similar DRM.) I guess the silver lining is, as you say, the industry might start taking…
-
what's your harness/prompt for this? Just yolo "clanker pls look for exploitable vulns. send subagents per submodule to manage the context growth"? or something more refined?
-
To be clear, gVisor is very much intended to be fully secure in the default configuration. Features like using the host networking stack (rather than the default user space stack), device passthrough, etc are all disabled by default and must be explicitly _enabled_. It is a project goal to make things difficult to accidentally configure…
- Sep 19
-
I'm fascinated by `sandbox-exec` because it's really good, almost completely undocumented, has been marked as deprecated for *years*, and is increasingly the secret sauce behind all sorts of other sandboxing tech, including that used by OpenAI and Anthropic. I'm hoping Apple eventually notice how important it has become and decide to not deprecate…
-
I wanted to offer that I've had pretty good luck with the (ahem) now deprecated [Apple Sandbox](https://keith.github.io/xcode-man-pages/sandbox.7.html) (way different from Apple Containers, which is, itself, different from `Library/Containers` and `Library/Group Containers`) I'm grateful to iTerm2's build process for turning me onto it…
-
It's worth considering the degree to which [need-to-know policies](https://en.wikipedia.org/wiki/Need_to_know), [compartmentalization](https://en.wikipedia.org/wiki/Compartmentalization_(information_security)), and [principles of least authority/access/privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) are already successfully…
-
Interesting... gVisor is entirely maintained by Google (they first released it in 2018) but I just saw that [two weeks ago](https://github.com/cncf/sandbox/issues/521) they started the process to donate it to CNCF.
-
You might find gVisor to be of interest.
-
I'd say Docker, but everyone warns against using Docker as a sandbox. I've not fully evaluated the new [Docker Sandboxes](https://docs.docker.com/ai/sandboxes/) thing yet, though it seems very focused on coding agents and I want to use it for other stuff, too. Firecracker is a LONG way from being a user-friendly piece of software to install and…
-
I haven’t read the article yet but I’ve had very similar thoughts: - we should assume that for the foreseeable future the rate of disclosure of critical vulnerabilities is going to be much higher, and we should be building systems with this in mind; and - I wonder if this is what finally causes software engineering to be regulated like other kinds…