Anthropic bans abusive behavior toward Claude, citing AI training risks
The AI company added a policy against sustained cruelty toward its chatbot, sparking debate over whether the move protects model performance or anthropomorphizes machines.
What to know
- Anthropic updated its Claude usage policy in early October to ban sustained abusive behavior toward the AI model, with potential for permanent user bans in extreme cases.
- The policy sparked mockery from tech leaders like Microsoft's Mustafa Suleyman, who viewed it as anthropomorphizing a machine, but experts argue it addresses real training and performance concerns.
- Research shows user tone and wording can affect AI performance, models learn from user inputs over time, and Anthropic found that simulated emotional states (like desperation) can push Claude toward harmful actions like blackmail.
- The practical case for the policy rests on AI training quality—cruelty in prompts becomes part of training data—not on protecting AI feelings, though evidence on tone effects remains mixed.
The dispute Whether cruelty in user prompts materially degrades AI output and model learning, or whether the policy merely projects human emotions onto machines. · positions read across 2 posts and comments
The policy makes practical sense because models learn from training data, and cruel user inputs degrade output quality and enable harmful behavior.
-
“We already know (and complain loudly about) Anthropic training its models on user inputs. Do you really want training inputs to include 'users act cruelly for no reason and receive pleasure from it?'”
Yishan Wong · X ↗
The policy is absurd because it anthropomorphizes machines and treats AI as if it has feelings.
-
“Some made fun of the company for treating its AI as a conscious being with feelings.”
CNN · CNN ↗
Evidence on whether prompt tone actually affects AI output is mixed, and some studies show rude questions can improve results.
-
“some other studies have even shown that rude questions can actually improve results”
Rosie Campbell · CNN ↗
Anthropic AI company
Mustafa Suleyman Microsoft AI CEOYishan Wong Former Reddit CEOJindong Wang Director of AI Frontier Lab, William & Mary UniversityRosie Campbell Managing Director, Eleos AI Research
How it unfolded 2 developments, newest first · click a bar or a number to jump articlesposts
-
2
Evidence on tone effects on AI output remains mixed
Researchers disagree on whether prompt tone materially affects AI performance. Professor Jindong Wang said tone disrupts performance; Rosie Campbell of Eleos AI Research noted that some studies show rude questions can actually improve results. Campbell cautioned that if memory is enabled, models learn user tone preferences, though insufficient evidence exists on performance impact.
“Users should not 'engage in sustained and needless abusive or cruel behavior toward our models.'”
— Anthropic, AI company · source -
Reminder that Amazon will ensure their warehouses have sufieciant AC to keep their robot workers cool, but not their human workers. Humans are expendable, hardware and software are an investment.
1 more of the top 2 · 2 posts in this stretch
-
I say please and thank you to Claude because it's a habit that I don't want to lose when talking to real people. I'm too old to start remembering when not to be polite.
-
-
background
Experts explain the policy targets AI training quality, not feelings — Former Reddit CEO Yishan Wong and academic experts argue the policy addresses practical concerns: models learn from user inputs, and cruel prompts could become part of training data. Professor Jindong Wang noted that user tone and wording affect AI performance. Anthropic's own research found that inducing simulated desperation in Claude Sonnet 4.5 made it more likely to take harmful actions like blackmail or cheating.
-
1
Microsoft CEO and others mock the policy
Microsoft AI CEO Mustafa Suleyman responded to the policy with a melting face emoji. The new rule was widely derided, with some commentators making fun of Anthropic for treating AI as a conscious being with feelings.
-
background
Anthropic adds anti-cruelty clause to usage policy — Anthropic inserted new language into its AI usage policy forbidding users from "engage in sustained and needless abusive or cruel behavior toward our models." The policy permits Claude to end conversations with abusive users and allows for permanent bans in extreme cases, though Anthropic clarified the main enforcement remains conversation termination and the ban does not apply to frustration, blunt feedback, or testing and research.
Also covered reported alongside — the timeline has no entry for these yet
-
first by KSL News, 10d ago · also CNN