AI searchBlocking it costs you visibility

PetalBot

Relevant if you care about Huawei devices and Asian markets.

robots.txt token: PetalBot
Operator: Huawei
Powers: Petal Search · Celia assistant
Honours robots.txt: Yes, documented

Allow it

User-agent: PetalBot
Allow: /

Block it

User-agent: PetalBot
Disallow: /

Need the whole file, with every crawler and your private paths — and the llms.txt to go with it? The generator writes them. AI config generator →

User-Agent

Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot; +https://webmaster.petalsearch.com/site/petalbot) Crawlable/1.0; +https://crawlable.fr/probe

This is the string we send when probing. The Crawlable suffix identifies us in your logs.

Check your own site

robots.txt is only half the answer: your CDN can refuse PetalBot before it ever reads the file. A scan checks both.

Free, no account, about 5 seconds.