Skip to content

Claude-SearchBot

Anthropic's search crawler. Anthropic says blocking it may reduce how visible you are in Claude's search results.

Check if your site lets Claude-SearchBot in Run the full AI Visibility Check

Last verified: October 2026, against Anthropic's own documentation.

Claude-SearchBot at a glance.

Owner
Anthropic
Purpose
Search. Anthropic says Claude-SearchBot navigates the web to improve search result quality for users. If it's disabled, your content isn't indexed for search optimization, which may reduce its visibility and accuracy in Claude's search results.
robots.txt token
Claude-SearchBot
Full user agent
Not published.

Anthropic's page names the robots.txt token Claude-SearchBot but doesn't publish a full user-agent string.

IP ranges
claude.com/crawling/bots.json

One shared IP list for all of Anthropic's crawlers.

Respects robots.txt?
Respects robots.txt. Yes. Anthropic says its bots honor industry-standard robots.txt directives.

The robots.txt rules to copy.

Your robots.txt lives at the root of your domain, like yoursite.com/robots.txt. On Shopify you edit it by adding a robots.txt.liquid template to your theme. Shopify warns that a wrong edit can cost you all your traffic, so change one rule at a time.

Allow Claude-SearchBot

User-agent: Claude-SearchBot
Allow: /

A bot with no rule is already allowed. Add this only when a broader rule, such as User-agent: * with Disallow: /, would otherwise block it.

Block Claude-SearchBot

User-agent: Claude-SearchBot
Disallow: /

Robots.txt rules apply per host. Repeat them on each subdomain you want covered.

Should you block Claude-SearchBot on a local business site or a Shopify store?

Anthropic's version of OAI-SearchBot. Per Anthropic, blocking it can reduce how visible and how accurate your business is in Claude's search answers.

Allow it.

A rule you didn't write can block it too: an old robots.txt template, a security plugin or a CDN bot setting. The AI crawler checker reads your robots.txt for Claude-SearchBot and the other bots in this directory.

The 12 bots, sorted by job

  1. Claude-SearchBot is in this group

    Search and answers

    3 bots

    They build the index an AI assistant searches when it answers.

    If you block themPer their owners, blocking one can drop you from that assistant's search answers.

  2. Fetching a page when a user asks

    3 bots

    They fetch your page because a person asked the assistant about it.

    If you block themBlocked at your firewall, the assistant can't read your page for the person who asked. Two of the three may not follow a robots.txt block.

  3. Model training

    6 bots

    A block here tells the owner to keep your pages out of AI model training. CCBot is here because its open archive can be used that way.

    If you block themFor most, the owner says blocking doesn't remove you from its search features. Meta says its crawler also indexes content for its products.

Where Claude-SearchBot sits among the 12 AI bots in this directory, grouped by what each one does, with what blocking each group costs you.

Related crawlers.

Sources

Last verified: October 2026. Where the owner doesn't document something, this page says so instead of guessing.

  1. Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler?

Crawler access is the first thing I check in an AI answer audit. Most of what decides who gets named is what other sites say about you.

Check your robots.txt Or run the full AI Visibility Check