OAI-SearchBot: OpenAI's AI search crawler
OpenAI's crawler for search in ChatGPT.
- Free, no account.
- A few seconds.
- Nothing is stored.
OAI-SearchBot at a glance
Everything OpenAI documents about it, with links to the source.
- Operator
- OpenAI
- Type
- AI search and answers
- Used for
- Pages shown and cited in ChatGPT search
- Obeys robots.txt
- Yes. OpenAI documents that OAI-SearchBot follows robots.txt, so a rule for its token is honoured.
- robots.txt token
OAI-SearchBot- User-agent string
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
- IP addresses
- openai.com/searchbot.json
- Documentation
- developers.openai.com/api/docs/bots
How to block OAI-SearchBot
Fetches the pages AI answers cite and link to. Blocking it keeps your pages out of those answers: no citation, no link.
Block the whole site
Add this group to robots.txt at the root of your site.
User-agent: OAI-SearchBot Disallow: /
Block only some paths
List the paths; everything else stays open to OAI-SearchBot.
User-agent: OAI-SearchBot Disallow: /private/ Disallow: /drafts/
How to verify OAI-SearchBot
Anyone can send a crawler's user-agent string. Where the request comes from is what proves it.
- 1
Find requests whose user agent contains "OAI-SearchBot" in your server or CDN logs.
- 2
Check each source IP against OpenAI's published list (openai.com/searchbot.json). A request from outside it is not OAI-SearchBot, whatever it says.
- 3
Block what fails the check at the firewall. A robots.txt rule only reaches crawlers that choose to read it.
Other crawlers from OpenAI
Each has its own token, and blocking one leaves the others untouched.
Frequently asked questions
OpenAI's crawler for search in ChatGPT. It finds the pages ChatGPT shows and links to in its answers. OpenAI says sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links.
Yes. OpenAI documents that OAI-SearchBot follows robots.txt, so a rule for its token is honoured.
Add a group for its token to robots.txt at the root of your site: "User-agent: OAI-SearchBot" followed by "Disallow: /". To close only part of the site, list those paths instead of "/". To enforce the block rather than request it, deny the IP addresses OpenAI publishes at your firewall.
OpenAI's AI answers stop finding your pages through it, so they cannot cite or link to you. If you want to keep your content out of training but stay in answers, block OpenAI's training token instead and leave OAI-SearchBot allowed.
Checked against OpenAI's documentation on 29 September 2026.
Next: see whether AI recommends you
Letting the right crawlers in is the first step. AskWatch shows what ChatGPT, Perplexity, Gemini and Google AI answer when buyers ask about your category, and who they name.
- Free, no credit card.
- Report in minutes, link sent to your email.