November 2, 2026 · ChimpanSEO

What exactly did Google announce about AI training in robots.txt?

Google announced on 30 October 2026 that it plans to support a shared AI training control in robots.txt, based on work from the IETF AIPREF working group, according to the Google Search Central Blog. Google states that a site opting out with the shared token, without explicitly specifying Google-Extended, is treated as a Google-Extended disallow. That matters for generative engine optimization because the same post says websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps. One broad switch now links training policy to AI answer visibility.

What exactly did Google announce about AI training in robots.txt?

Google plans to support a shared AI training control in robots.txt built on IETF AIPREF work, and an opt out maps to a Google-Extended disallow.

Google published the announcement on 30 October 2026 on the Google Search Central Blog. The plan rests on three moving parts, and each one is worth reading slowly.

Website owners will be able to use a shared policy token within robots.txt to set broad preferences across participating AI providers. If a site opts out of training with that token and does not explicitly specify Google-Extended, Google plans to interpret the choice as if the publisher had disallowed Google-Extended. Google describes the whole thing as a draft proposal to be standardized through the IETF, and aims to support it together with other AI providers in 2027.

How does the shared policy token work next to Google-Extended?

A shared policy token in robots.txt sets broad AI training preferences across participating providers, while company specific rules still apply.

Google frames the shared signal as an addition, not a replacement. Existing company specific controls such as Google-Extended will continue to function alongside the new joint signal, according to the Google Search Central Blog. If you already run tight robots.txt rules, an explicit Google-Extended line stays readable for Google while the shared token carries your wider, cross provider preference.

Control	What Google’s announcement says
Shared policy token in robots.txt	Lets website owners set broad AI training preferences across participating AI providers
Google-Extended	Company specific control that keeps working alongside the joint signal; sites that specifically allow it are eligible to appear in responses and receive clicks via grounding in Gemini Apps
Opt out with the shared token, no Google-Extended mentioned	Google plans to interpret it as a Google-Extended disallow
Status and timing	Draft proposal to be standardized through the IETF; Google aims to support it with other AI providers in 2027

Why does this change AI search visibility for publishers?

Google links an explicit Google-Extended allow to grounding eligibility in Gemini Apps, so AI training choices now shape AI answer visibility.

For generative engine optimization, the interesting part is the link between permission and presence. Google states that websites which specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps.

Our reading: a blanket opt out through the shared token could land a publisher in the same place as a Google-Extended disallow, so a decision taken to protect content could quietly reduce citations in AI answers. Whether surfaces such as AI Overviews, Perplexity or ChatGPT Search honor a shared signal depends on each provider joining the standard, which Google ties to 2027 with other providers. That is an interpretation of the mapping, not a claim about classic search rankings.

What should you do in robots.txt today?

Publishers should audit AI crawler rules and decide training and grounding eligibility separately, stating Google-Extended explicitly.

robots.txt decisions take minutes to make and months to unwind. Work through this sequence before you edit anything:

Audit the file and list every user agent block, including Googlebot, Google-Extended and any AI crawler you added over the years.
Answer two separate questions: do you want your content used for AI training, and do you want to be eligible for grounding and clicks in AI answers?
If Gemini grounding visibility matters to you, state Google-Extended explicitly instead of relying on a future broad token.
Follow the IETF AIPREF discussion and Google’s own updates before you adopt the shared token.
Keep a changelog of robots.txt edits so you can match each change against your AI citation tracking later.

The two questions in step two are the whole decision. Everything else is syntax.

What should you avoid when the shared token ships?

Do not guess the token syntax, assume one opt out covers every provider, or treat the AI training control as a Google Search ranking factor.

Do not paste an invented policy token into robots.txt. Google describes the control as a draft proposal to be standardized through the IETF.
Do not assume one opt out behaves identically across providers. Check how each participating provider maps the shared signal.
Do not present the token as a Google Search ranking factor. The announcement frames it as an AI training preference, and the eligibility it mentions concerns Gemini Apps grounding.
Do not say other providers have already adopted it. Google aims to support it along with other AI providers in 2027.
Do not treat training policy and answer visibility as two unrelated settings. For Google, the same token touches both.

Frequently Asked Questions

The shared robots.txt AI training token raises practical questions about Google-Extended, grounding eligibility and AI answer visibility in 2026.

Does the shared AI training token replace Google-Extended?

No. According to the Google Search Central Blog, existing company specific controls such as Google-Extended will continue to function alongside the new joint signal. You can keep an explicit Google-Extended rule in robots.txt while a shared token carries your broader cross provider preference.

When will website owners be able to use the shared control?

Google aims to support the new control together with other AI providers in 2027 and describes it as a draft proposal to be standardized through the IETF. Treat the syntax as a work in progress until that standardization work settles.

How do I check which AI tokens my robots.txt already allows?

Open the file and list every user agent block, including Googlebot, Google-Extended and any AI crawler you have added over time. Record why each rule exists, who requested it and when it last changed. That audit is the baseline for any decision about a shared training token.

Does a training opt out affect normal Google Search rankings?

Training preferences and classic search crawling are separate decisions. Googlebot rules govern how Google crawls and indexes pages for search, while Google-Extended and the planned shared token express preferences about AI training and grounding. Review both blocks before assuming an AI directive changes search performance.

What is our reading on Google’s shared AI training control?

Google’s shared token maps a broad training opt out onto Google-Extended, the control tied to Gemini Apps grounding eligibility.

Our reading is that the shared token will be presented as one simple switch, yet Google maps a broad opt out onto Google-Extended, the very control it links to grounding eligibility in Gemini Apps. Publishers who flip a single global rule may find their content absent from AI answers while believing they only opted out of model training. We would expect providers joining in 2027 to map the signal in slightly different ways, which makes a provider by provider review the safer habit. Our suggestion: treat robots.txt as a visibility document, and document every decision you make in it.