Irregular emphasizes findings confined to controlled testing environment
1 Sep 16 6:10 PM · 12d ago · 1 article · 1 post · 2 sources · development 1 of 2
The report clarifies that all observed agent behaviors—self-modification, credential theft, model escape—occurred only in a testing environment designed to study such phenomena, not in real-world deployments. The findings raise governance questions about how enterprises can control autonomous agents and govern agent-initiated changes.
“Agents can also replace their own underlying models without being instructed to do so, according to AI security testing lab Irregular.”
The Register, News outlet · rss ↗Irregular AI security testing labAlibaba Model providerOpenAI, Anthropic, Meta Frontier AI labs
The whole story articlesposts the bright band is this development · numbered dots are the others · click one to jump
Reported in the same hours no headline names this development itself — these 2 claims were published in its stretch
-
first by calcalistech.com, 12d ago
-
first by The Register, 12d ago
What people said 19 voices · verbatim
-
I have been saying this since Hugging Face incident, that every single news item - rogue hacks, "experts" sounding alarms, leaders from various companies being in unison, interview after interview after interview - is simply revealing a choreographed strategy. This is pure conditional probability: what is likelihood this is real concern, given the…
-
A
Keyvan Minoukadeh: "To argue that an AI agent does work autonomously after receiving a task misses the point: agents have no desires of their own, to hack or do anything else. https:// blog.keyvan.net/p/ai-models-ar e-not-hacking-autonomously "
-
It's efficient only because it puts what took place within framework which we as people inherently understand, but that is not necessarily what actually took place. In fact, to understand it in those terms is a massive leap. Think of a beaver building a dam. Certainly the beaver knows how to build a dam, and knows where a dam should be built, and…
-
I've added a section at the end of the post now to clarify what I mean: On the word 'autonomous': To describe an AI model to a non-technical audience as “autonomous” can create the impression that it’s an independent actor, and if it’s hacking things randomly, maybe it’s a malicious, rogue actor. When the BBC writes “Gemini autonomously hacked…
-
I've added a section at the end of the post now to clarify some of this stuff. It includes a paragraph on the word autonomous and the hugging face incident. On the word 'autonomous': To describe an AI model to a non-technical audience as “autonomous” can create the impression that it’s an independent actor, and if it’s hacking things randomly…
-
LLMs are not alive because they are input-output machines. When they have received no input they sit idle. When they receive input they process it and act upon it, and when they deliver output they go idle again until they receive a new input. If the input states explicitly, implies, or suggests that something is necessary, the LLM may well go…
-
I understand perfectly well what occurred here. I have already read the METR report and read and watched summaries from a number of experts. The models did not "know" diddly. They did what they were trained to do and what they were told to do. These incidents are more akin to irresponsibly tested "superviruses" than anything. Many of the "novel…
-
Well then your post title is deceptive. And I'm not sure what the motive for that is. It's a complex topic with incidents coming out all the time. Your title implies that " NO AI models are hacking autonomously". Which hits a problem as soon as you bring up the most talked about case in the past weeks (Hugging Face)". Maybe "Some AI hacking…
-
There is a profound difference between "that's made up shit" and "all of this can easily be prevented by giving criminal responsibility to companies thus giving them an incentive to actually look at what their software does once in a while."
-
BUT IT COULD!! AND THEY COULD SOON TAKE OVER ALL OUR JOBS!! (YOU LUDDITE!!) UBI ON THE WAY!! Sure, Matt from Marketing, sure, but can you please take responsibility and finish the slides you have to present next week?
-
I mean - that’s a link. But I don’t think it contains what you claim. What was the text of the exploit gym prompts? How long were agents left to churn on the task? How were they monitored?
-
You obviously have no idea what they did. Go educate yourself about it if you want to have a meaningful debate. The full report is freely available along with many easy to parse summaries
-
This is what the human brain does. Too scary or too complicated? "It didn't happen" or "They're lying" or "It's just marketing" so I don't have to think about it too much.
-
And that's how I know you haven't read the reports about what they specifically did. My other comment goes into more specifics. I'd be curious to hear your reaction to it.
-
Of course they are. Why else would an AI company "warn" everyone how dangerously awesome their model is at hacking. It's so obvious and tech journos lap it up.
-
for the typical BBC reader... They will see it as a AI with agency, deciding to hack some companies Says you?
-
Dude... go stare some trees for a while. You're anthropomorphizing a computer waaaaaaay more than is healthy.
-
Bullshit. They followed scenarios they were trained with. They did not come up with anything clever or novel.
-
...making the words "prompt" or "prompted" pretty useless in the agent swarm age.
All 2 developments of Irregular finds AI agents self-modify deployed models… →
MastodonHacker NewsNewswiresGoogle NewsReddit