Model trainingYour call — not scored
Google-Extended
A robots.txt control token, not a crawler. Disallowing it keeps your content out of Gemini training and grounding without touching Search.
- robots.txt token
Google-Extended- Operator
- Google ↗
- Powers
- Gemini model training · Gemini grounding
- Honours robots.txt
- Yes, documentedDocumentation ↗
A control token, not a crawler: it is honoured in robots.txt but never sent as a user-agent, so it cannot be tested with a live request.
Allow it
User-agent: Google-Extended
Allow: /Block it
User-agent: Google-Extended
Disallow: /Need the whole file, with every crawler and your private paths — and the llms.txt to go with it? The generator writes them. AI config generator →
Check your own site
robots.txt is only half the answer: your CDN can refuse Google-Extended before it ever reads the file. A scan checks both.
Free, no account, about 5 seconds.