Google, OpenAI, Bing, Anthropic adopt unified AI training control standard
Major AI providers adopt IETF proposal allowing websites to opt out of AI training via single robots.txt entry.
What to know
- Google, OpenAI, Bing, and Anthropic jointly committed to supporting an IETF standard that consolidates AI training opt-outs into a single robots.txt entry, replacing fragmented per-company controls.
- The AIPREF working group proposal allows websites to express training preferences across all participating providers simultaneously using a shared policy token.
- Google plans to implement support in early 2027, interpreting the opt-out signal as a disallow of its Google-Extended training marker unless explicitly overridden.
- The standard aligns with EU AI Act requirements for machine-readable publisher preferences and addresses regulatory and civil society calls for industry-wide training controls.
“moving forward with this initiative alongside peer AI providers as part of a shared commitment to foster an open, creator-first web ecosystem, and following feedback from publishers, civil society and regulators calling for an easier way for websites to express their AI training preferences.”
Google · Search Engine Roundtable ↗
Google AI providerOpenAI AI providerMicrosoft Bing AI providerAnthropic AI provider
How it unfolded 1 development · click the chart to see its coverage articlesposts
-
upcoming
Google plans to support AIPREF shared training preference signal in early 2027 — Google updated its common crawlers page to commit to supporting the AIPREF machine-readable training preference signal with the user-agent tag "__usage_ai-training" in early 2027. When a website opts out via this mechanism, Google will treat it as a disallow of Google-Extended unless explicitly overridden.
-
1
“This will help websites express their AI training preferences across multiple AI providers in a single step and is based on work from the AIPREF working group at the Internet Engineering Task Force (IETF).”
— Google -
first by Search Engine Roundtable, 12d ago
-
R
Google also announced new AI training controls for 2027, under consideration by the IETF https://www. seroundtable.com/google-ai-tra ining-control-42430.html
-