Training blocks don't cost you AI answers
Google, Bing, OpenAI and Anthropic moved AI training controls into one robots.txt token. Training and grounding are now separate switches, on separate clocks.
On October 30, Google, Microsoft, OpenAI and Anthropic each announced — within hours of each other — that AI training preferences move into robots.txt as one shared token. The coordinated part is the headline. The part that changes your work is underneath it: training and grounding are now two switches, with different owners, different clocks, and on Bing one deadline that quietly widens what a very old directive permits.
Short answer first, because most coverage buries it: blocking AI training does not remove you from AI answers. Google's own post is explicit that sites which specifically allow Google-Extended stay eligible to appear in responses and receive clicks through grounding in Gemini Apps. Blocking grounding is what removes you. After this week, four providers document that difference in writing. That makes "which switch am I actually pulling" an answerable question for the first time.
Three things people call "blocking AI"
The confusion is a vocabulary problem. There are three distinct levers and they fail in different directions.
- Training decides whether your content becomes part of a model's weights. It is a one-way door: a block applies from the point it is honoured, and nothing un-trains a model that already learned from you.
- Grounding decides whether your page is read at answer time to inform a generated response. It does not touch the weights. It is the lever that determines whether an answer can cite you today.
- Indexing decides whether you appear in a list of results. Bing's documentation draws the hardest line of the three: "Indexing and ranking pages in traditional search results is not grounding." They are separate systems in the same company.
The failure mode is pulling the training switch and expecting a citation outcome. A training block is a promise about the future of the model. A grounding block is a decision about the present of the answer.
What each provider actually lets you switch
Google and Microsoft did not ship the same control with different logos. Read the rows before you write a line of robots.txt.
| What you are switching | Microsoft Bing | OpenAI | |
|---|---|---|---|
| Shared training signal | Policy token in robots.txt, based on the IETF AIPREF draft. Support planned 2027 | Shared AI Training preference in robots.txt at site or path level, plus a page-level override. Honoured from Feb 1, 2027 | Will honour the shared signal; "in the coming months", no date given |
| Vendor training token | Google-Extended, offered since 2023, keeps working alongside it | Existing controls continue | Existing crawler controls continue and "will not be replaced" |
| Grounding / answer use | No dedicated token. Gemini Apps grounding follows Google-Extended; AI Overviews and AI Mode are governed by the separate Search Console control | AI Grounding, page-level only, three values: Allow, DisallowExtended, Disallow | No separate grounding switch |
| Legacy directives | — | NoArchive and NoCache stop counting for training after Mar 1, 2027 | — |
| Page-level override | Not announced for the shared token | Yes, for both training and grounding | — |
Two details in that table are worth more than the announcement itself.
First, Google's opt-out direction: if you use the policy token to opt out of training and do not specify Google-Extended, Google plans to read that as disallowing Google-Extended. If you want the broad block and to stay citable in Gemini, you have to say so explicitly. The specific allow beats the broad opt-out.
Second, Microsoft went one level deeper than the shared signal. Training can be set at site, path or page. Grounding is page-level only, and it is three-state — DisallowExtended keeps you inside Microsoft's own products (AI Mode, AI Overview, Copilot, Microsoft 365) while refusing the grounding services Microsoft sells to third parties, such as Web IQ and the Foundry grounding API. Nobody else offers that middle setting.
The four dates, and the one that applies to you
Bing published a calendar; Google and OpenAI published intentions. Only Bing's dates bind anyone:
- December 15, 2026 — temporary grounding transition preferences open in Bing Webmaster Tools. They exist only for sites using
NoArchiveorNoCacheas of that date. Every other site can ignore this step. - February 1, 2027 — Microsoft starts honouring the new preferences as it crawls. Not on a date you pick; on the date it reaches your pages.
- March 1, 2027 — the deadline. If you use
NoArchiveorNoCacheand want to keep content out of grounding, you must publish an AI Grounding preference on your pages and record a temporary preference in Bing Webmaster Tools before this date. - September 30, 2027 — temporary preferences expire, or end earlier when Microsoft recrawls the page and finds your on-page setting.
Google says 2027 with no month. OpenAI says the coming months with no date. Plan for the one that has a day attached.
The setting that expands by inaction
Here is the sentence in Microsoft's post that deserves a meeting, and did not get one:
"If you do not record a temporary grounding preference by March 1 in Bing Webmaster Tools, pages using
NoArchiveorNoCachedefault to Allow for grounding until Microsoft recrawls them and discovers your new AI Grounding preference."
NoArchive means "do not keep a stored copy of this page". Sites set it years ago for all kinds of reasons — licensing, syndication terms, an old CMS default, a legal review nobody remembers. Microsoft read that directive as an AI restriction from 2023 onward. On March 1, 2027 the training half of it dies outright, and the grounding half falls back to allow unless a human acts.
That is not a policy change you opted into. It is a permission you lose by not showing up. If your AI posture lives in directives you inherited rather than wrote, the first useful thing you can do this week is list every page still carrying NoArchive or NoCache and decide, one by one, which ones you actually want out of answers.
Training is becoming portable. Grounding is not.
Four providers agreed on training in a single day. That is not because training is easy — it is because training preference is a binary that costs a provider almost nothing to standardise.
Grounding is where answer quality lives, so nobody is standardising it yet. Google's post says the mechanism brings "the benefit of the AI training definition developed at the IETF while multistakeholder discussions continue on the rest of the AIPREF vocabulary." Read that clause literally: the training vocabulary shipped, the rest is still being argued. Bing's grounding control is per-page and per-vendor. Google's is split across Google-Extended and a Search Console toggle that only covers AI Overviews, AI Mode and AI Overviews in Discover — not Gemini.
So the working assumption for the next year is asymmetry. Write one training policy you can defend in a paragraph. Maintain a grounding list page by page, vendor by vendor, with an owner and a review date, because there is no shared token coming to do it for you.
The recrawl trap nobody schedules for
Page-level settings are only real once a crawler reads them. Microsoft states the limit plainly: it must crawl the page to discover the setting, and IndexNow "can signal that a page has changed and increase the chance of a recrawl, but it does not guarantee a recrawl date."
That kills a use case people assume they have. If a page must be out of AI answers this week — a retraction, a legal hold, a licensing dispute — a page-level grounding preference is not the tool, because its effective date belongs to the crawler. Robots.txt text is read on the next fetch of the path; a meta tag in the head is read when the page body is fetched, which may be much later. Those are different latencies and they are chosen by someone else.
What none of these controls do
- They do not un-train anything. A preference governs future crawling and future training runs. A model that already learned from your archive keeps what it learned.
- They do not bind crawlers that ignore robots.txt. A preference is an offer of terms, not enforcement. A robots.txt rule is a fetch switch, not a visibility switch, and it never was.
- They do not remove a page from search. For that you need noindex, which is a different lever with a different mechanism and a different failure mode. The distinction between OpenAI's three crawlers and who decides your ChatGPT citations is the same lesson one vendor down.
- They are not final. The shared signal is an Internet-Draft, not a standard. One practitioner credited in that draft's acknowledgments said on publication day that the work is not finished at the IETF level and that significant areas of dispute remain. Treat 2027 as a plan with a date attached to it, not a promise.
robots.txt today, robots.txt after February 2027
| Write now | Re-read after Feb 2027 |
|---|---|
| One training preference: the shared token if you want out everywhere, plus an explicit Google-Extended allow if you still want Gemini grounding | Whether each provider is actually honouring the token, and whether the opt-out is being read as you intended |
An inventory of every page carrying NoArchive or NoCache, with a keep-or-retire decision per page | The retired directives are gone; the kept ones need a page-level AI Grounding preference to still mean anything |
| A grounding position per vendor: Google via Google-Extended and Search Console, Bing per page with a three-state value, OpenAI covered by its crawler controls | Whether a shared grounding vocabulary appeared. If it did, the per-page list collapses into one token |
| An owner and a review date next to the policy | The same, because the asymmetry is the part that persists |
The rule
Decide training and grounding separately, in writing, with a name and a date attached to each. If your AI policy fits on one line, it is a training policy and you have not made the grounding decision yet. Grounding needs a page-by-page list, because no provider has given you a portable way to express it — and Microsoft's calendar shows what happens to a page whose owner never revisited the directive it inherited.
If you want the audit that produces that list — what your headers and robots.txt say today, which of the four providers each directive binds, and what changes on February 1 — that is the work described on our SEO and GEO service page. Bring the inventory. The decisions are yours to make, but they should be made deliberately rather than by default.
Frequently Asked Questions
Does blocking AI training remove me from AI answers?
No. Google's announcement says explicitly that websites which allow Google-Extended stay eligible to appear in responses and receive clicks through grounding in Gemini Apps. Training and grounding are separate switches: a training block stops your content going into model weights, while grounding controls whether your page is read at answer time. Blocking grounding is what removes you from the answer surface.
What is the shared AI training token in robots.txt?
It is a single policy token, being drafted in the IETF AIPREF working group, that expresses an AI training preference for many providers at once instead of one rule per crawler. Google, Microsoft, OpenAI and Anthropic all said on October 30, 2026 that they intend to honour it. Google plans support in 2027 and says an opt-out through the token will be read as disallowing Google-Extended unless Google-Extended is specified separately.
What happens to NoArchive and NoCache after March 1, 2027?
Microsoft stops honouring them for AI training on that date, and for grounding they only keep restricting if you publish a page-level AI Grounding preference and record a temporary grounding preference in Bing Webmaster Tools. If you do neither, pages carrying those directives default to Allow for grounding until Microsoft recrawls them.
