Skip to content

Claude-User

Claude fetching a page because a user asked a question. Anthropic says it honors robots.txt.

Check if your site lets Claude-User in Run the full AI Visibility Check

Last verified: October 2026, against Anthropic's own documentation.

Claude-User at a glance.

Owner
Anthropic
Purpose
User-triggered. Anthropic says Claude-User supports Claude users: when someone asks Claude a question, it may visit websites with this agent. Site owners can use it to control which sites those user-initiated requests can reach.
robots.txt token
Claude-User
Full user agent
Not published.

Anthropic's page names the robots.txt token Claude-User but doesn't publish a full user-agent string.

IP ranges
claude.com/crawling/bots.json

One shared IP list for all of Anthropic's crawlers.

Respects robots.txt?
Respects robots.txt. Yes. Anthropic's statement that its bots honor robots.txt covers all three of its bots, and its page describes Claude-User as the token for controlling user-initiated fetches.

The robots.txt rules to copy.

Your robots.txt lives at the root of your domain, like yoursite.com/robots.txt. On Shopify you edit it by adding a robots.txt.liquid template to your theme. Shopify warns that a wrong edit can cost you all your traffic, so change one rule at a time.

Allow Claude-User

User-agent: Claude-User
Allow: /

A bot with no rule is already allowed. Add this only when a broader rule, such as User-agent: * with Disallow: /, would otherwise block it.

Block Claude-User

User-agent: Claude-User
Disallow: /

Robots.txt rules apply per host. Repeat them on each subdomain you want covered.

Should you block Claude-User on a local business site or a Shopify store?

Anthropic says disabling Claude-User stops Claude from retrieving your content in response to a user's question, which may reduce your visibility in user-directed web search.

Unlike the user agents from OpenAI and Perplexity, this one is documented as respecting robots.txt, so a stray rule can keep Claude from reading you when someone asks. Keep it open.

A rule you didn't write can block it too: an old robots.txt template, a security plugin or a CDN bot setting. The AI crawler checker reads your robots.txt for Claude-User and the other bots in this directory.

The 12 bots, sorted by job

  1. Search and answers

    3 bots

    They build the index an AI assistant searches when it answers.

    If you block themPer their owners, blocking one can drop you from that assistant's search answers.

  2. Claude-User is in this group

    Fetching a page when a user asks

    3 bots

    They fetch your page because a person asked the assistant about it.

    If you block themBlocked at your firewall, the assistant can't read your page for the person who asked. Two of the three may not follow a robots.txt block.

  3. Model training

    6 bots

    A block here tells the owner to keep your pages out of AI model training. CCBot is here because its open archive can be used that way.

    If you block themFor most, the owner says blocking doesn't remove you from its search features. Meta says its crawler also indexes content for its products.

Where Claude-User sits among the 12 AI bots in this directory, grouped by what each one does, with what blocking each group costs you.

Related crawlers.

Sources

Last verified: October 2026. Where the owner doesn't document something, this page says so instead of guessing.

  1. Anthropic, Does Anthropic crawl data from the web, and how can site owners block the crawler?

Crawler access is the first thing I check in an AI answer audit. Most of what decides who gets named is what other sites say about you.

Check your robots.txt Or run the full AI Visibility Check