🔍 Read the full analysis: AI Training Opt-Out: A Closer Look At What It Leaves Out on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have said they will honor a shared robots.txt signal asking that a site’s content not be used to train generative AI models. The signal is voluntary, adoption dates vary, and it does not block crawling, control AI answers in every service or remove material from models already trained.
OpenAI, Google, Microsoft and Anthropic have said they will honor a shared instruction in websites’ robots.txt files asking that content not be used to train generative AI models. The announcement, made October 30, offers publishers one common signal instead of separate training opt-outs for each company, but the rule is voluntary and the companies have not set one shared start date.
The proposed instruction uses the user-agent name __usage_ai-training, followed by Disallow: /. A publisher adding that group is expressing a preference against use of its site content for generative AI training. The text appears in an individual Internet-Draft, “Machine-Readable Training Preference Signal,” written by Anthropic standards lead Kevin Kelley and published October 30.
Implementation plans differ. Anthropic says it has already started adding support, though it says support across its services will take time. OpenAI says it aims to implement the signal “in the coming months.” Microsoft says it will honor the training preference from February 1, 2027, as it crawls sites. Google says it aims to support it in 2027; its crawler documentation specifies “early 2027.”
Meta expressed support for publishers’ ability to state preferences, but its spokesperson did not say whether the company will honor this particular signal. The source article reports no statement from Apple, Amazon, xAI or Perplexity in the material reviewed. Existing company-specific crawler rules remain in place while adoption proceeds.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A Simpler Request, Not a Block
The shared rule could reduce the work publishers face when setting training preferences. A site currently has to account for different crawler names and controls; one common instruction could make its stated choice easier for participating companies to recognize. That may matter particularly to publishers managing large sites or updating policies across many properties.
But a robots.txt instruction is a request to participating crawlers, not technical access control. The source describes the signal as voluntary; it does not establish that every AI company will follow it or that a site can prevent access through other means. Publishers should not read the announcement as a guarantee that content will be excluded from training.
The distinction between training and other uses also matters. Microsoft separately announced a page-level preference for “AI Grounding,” referring to web content used to inform answers in Bing, Copilot and Microsoft 365. That separate control illustrates why a training opt-out alone may not settle how a company uses a page in AI products. Microsoft says neither preference guarantees inclusion, traffic or payment.
Products Worth Considering
FLEXISPOT Standing Desk with Monitor Riser & Cable Management
Samsung Essential S3 27-Inch Curved FHD Monitor
Brother QL-800 High-Speed Professional Thermal Label Printer
As an affiliate, we earn on qualifying purchases.
The signal is based on work associated with the IETF’s AI Preferences working group, which discusses machine-readable ways for websites to express preferences. The new text is an individual Internet-Draft, not an adopted internet standard. The source article notes that at a working-group meeting on October 21, a chair said: “We haven’t declared consensus on anything yet.”
The announcement does not replace existing crawler instructions. OpenAI says its current crawler controls will continue to apply. Google says that if a site uses the new signal but has no Google-Extended rule, it plans to treat the shared opt-out as if the publisher had disallowed Google-Extended. Microsoft says its new training and AI Grounding preferences take effect February 1, 2027, and that NoArchive and NoCache will stop counting as training or grounding preferences after March 1, 2027. Microsoft says those changes do not alter eligibility for Bing search results; Google says Google-Extended does not affect Google Search.
The source article also discloses that the draft was written by an Anthropic standards lead and that its research and drafting used AI assistance, including Anthropic’s Claude. Its publisher says it belongs to a network of publisher websites affected by AI companies’ use of online content. Those disclosures provide relevant context for readers evaluating its account; the company positions above are attributed to the companies or their published materials.
““has already started implementing support for this draft””
— Anthropic
Adoption and Enforcement Remain Open
It is not yet clear how consistently the companies will honor the signal across their products, crawlers and services. Anthropic says implementation has begun, but does not give a completion date; OpenAI’s timing is described as the coming months; Google’s target is 2027. Meta has not made an implementation commitment in the statement cited by the source, and the material reviewed does not establish which other AI companies will participate.
The draft also has no formal standing in the IETF standards process, according to the source. Its wording, status and eventual uptake could change. The signal expresses a training preference; it does not itself stop crawling, undo training on material already used, cover every form of AI-generated answer, or create a payment obligation. How companies handle edge cases or disputes is not specified in the source material.
Watch Company Rollouts Through 2027
The next test is whether each company implements the draft on the timelines it has described and makes the behavior clear in its crawler documentation. Anthropic’s rollout is underway, OpenAI has cited the coming months, and Google is targeting early 2027. Microsoft’s announced date is February 1, 2027, with its change to how NoArchive and NoCache preferences count scheduled for March 1.
For site operators, the source recommends treating training, AI answers and search as separate decisions. Publishers who want to state a training preference can add the shared rule, while keeping existing company-specific controls until each provider’s implementation is clear. Microsoft’s separate AI Grounding preference may be relevant to sites concerned about how their pages inform its AI answers. Publishers should check provider documentation as dates approach; the announced opt-out alone does not settle every use of website content.
Key Questions
What does the shared AI-training opt-out say?
The proposed robots.txt group uses User-Agent: __usage_ai-training and Disallow: /. It tells companies that honor the signal that the publisher prefers its content not be used to train generative AI models.
Does the rule prevent AI companies from crawling a site?
No. The signal expresses a training preference; it is not a technical block on crawling. Its effect depends on whether a company chooses to honor it.
When will the companies support it?
Anthropic says support is being implemented. OpenAI aims to implement it in the coming months, Google targets early 2027, and Microsoft says it will honor it from February 1, 2027. The companies’ plans are not all on the same schedule.
Does opting out remove content from AI models already trained?
The source says the signal does not remove content from models already trained. It is a preference for future use, not a mechanism for reversing past training.
Does the shared rule control AI answers or guarantee payment?
Not by itself. The signal concerns training, while Microsoft has announced a separate preference for AI Grounding in its products. Microsoft says neither choice guarantees inclusion, traffic or payment.
Source: ThorstenMeyerAI.com
