DuckDuckBot: DuckDuckGo's search crawler
DuckDuckGo's crawler for its search results.
- Free, no account.
- A few seconds.
- Nothing is stored.
DuckDuckBot at a glance
Everything DuckDuckGo documents about it, with links to the source.
- Operator
- DuckDuckGo
- Type
- Search engines
- Used for
- DuckDuckGo search
- Obeys robots.txt
- Yes. DuckDuckGo documents that DuckDuckBot follows robots.txt, so a rule for its token is honoured.
- robots.txt token
DuckDuckBot- User-agent string
DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)
- IP addresses
- duckduckgo.com/duckduckbot.json
How to block DuckDuckBot
Crawls for a search engine's index. Blocking it removes your pages from that search engine, and from the AI answers built on it (AI Overviews on Googlebot, Copilot on Bingbot).
Block the whole site
Add this group to robots.txt at the root of your site.
User-agent: DuckDuckBot Disallow: /
Block only some paths
List the paths; everything else stays open to DuckDuckBot.
User-agent: DuckDuckBot Disallow: /private/ Disallow: /drafts/
How to verify DuckDuckBot
Anyone can send a crawler's user-agent string. Where the request comes from is what proves it.
- 1
Find requests whose user agent contains "DuckDuckBot" in your server or CDN logs.
- 2
Check each source IP against DuckDuckGo's published list (duckduckgo.com/duckduckbot.json). A request from outside it is not DuckDuckBot, whatever it says.
- 3
Block what fails the check at the firewall. A robots.txt rule only reaches crawlers that choose to read it.
Other crawlers from DuckDuckGo
Each has its own token, and blocking one leaves the others untouched.
Frequently asked questions
DuckDuckGo's crawler for its search results. DuckDuckGo says it respects robots.txt and publishes the addresses it crawls from.
Yes. DuckDuckGo documents that DuckDuckBot follows robots.txt, so a rule for its token is honoured.
Add a group for its token to robots.txt at the root of your site: "User-agent: DuckDuckBot" followed by "Disallow: /". To close only part of the site, list those paths instead of "/". To enforce the block rather than request it, deny the IP addresses DuckDuckGo publishes at your firewall.
Only indirectly. DuckDuckBot crawls for DuckDuckGo's search, and AI features built on that search use the same index.
Checked against DuckDuckGo's documentation on 29 September 2026.
Next: see whether AI recommends you
Letting the right crawlers in is the first step. AskWatch shows what ChatGPT, Perplexity, Gemini and Google AI answer when buyers ask about your category, and who they name.
- Free, no credit card.
- Report in minutes, link sent to your email.