Skip to content

Meta-ExternalAgent

Meta's crawler for training foundation models and indexing content for its products.

Check if your site lets Meta-ExternalAgent in Run the full AI Visibility Check

Last verified: October 2026, against Meta's own documentation.

Meta-ExternalAgent at a glance.

Owner
Meta
Purpose
Training. Meta says Meta-ExternalAgent crawls the web to train foundation AI models or to improve products by indexing content directly. Meta lists a separate crawler, Meta-WebIndexer, for Meta AI search results and citations.
robots.txt token
meta-externalagent
Full user agent
meta-externalagent/1.1

Meta lists this string, sometimes followed by a link to its crawler documentation.

IP ranges
Not published.

Undocumented. Meta's page doesn't publish an IP list for this crawler. It suggests allowlisting by user agent or IP address, and lists webmasters@meta.com for questions.

Respects robots.txt?
Respects robots.txt. Yes, by Meta's account. Meta says it uses robots.txt to let site owners state their preferences and tells you to block a crawler with a Disallow rule. It names only Meta-ExternalFetcher and FacebookExternalHit as crawlers that may bypass robots.txt. Meta says changes can take up to 24 hours, since its crawlers may cache robots.txt.

The robots.txt rules to copy.

Your robots.txt lives at the root of your domain, like yoursite.com/robots.txt. On Shopify you edit it by adding a robots.txt.liquid template to your theme. Shopify warns that a wrong edit can cost you all your traffic, so change one rule at a time.

Allow Meta-ExternalAgent

User-agent: meta-externalagent
Allow: /

A bot with no rule is already allowed. Add this only when a broader rule, such as User-agent: * with Disallow: /, would otherwise block it.

Block Meta-ExternalAgent

User-agent: meta-externalagent
Disallow: /

Robots.txt rules apply per host. Repeat them on each subdomain you want covered.

Should you block Meta-ExternalAgent on a local business site or a Shopify store?

A meta-externalagent block opts you out of Meta's model training. Meta also describes the crawler as indexing content for its products, so a block may reach further than training. Meta doesn't spell out which products.

For being cited in Meta AI, the crawler Meta names is Meta-WebIndexer, which it lists separately. Leave FacebookExternalHit alone too: Meta uses it to build the link previews when someone shares your page.

A rule you didn't write can block it too: an old robots.txt template, a security plugin or a CDN bot setting. The AI crawler checker reads your robots.txt for Meta-ExternalAgent and the other bots in this directory.

The 12 bots, sorted by job

  1. Search and answers

    3 bots

    They build the index an AI assistant searches when it answers.

    If you block themPer their owners, blocking one can drop you from that assistant's search answers.

  2. Fetching a page when a user asks

    3 bots

    They fetch your page because a person asked the assistant about it.

    If you block themBlocked at your firewall, the assistant can't read your page for the person who asked. Two of the three may not follow a robots.txt block.

  3. Meta-ExternalAgent is in this group

    Model training

    6 bots

    A block here tells the owner to keep your pages out of AI model training. CCBot is here because its open archive can be used that way.

    If you block themFor most, the owner says blocking doesn't remove you from its search features. Meta says its crawler also indexes content for its products.

Where Meta-ExternalAgent sits among the 12 AI bots in this directory, grouped by what each one does, with what blocking each group costs you.

Related crawlers.

Sources

Last verified: October 2026. Where the owner doesn't document something, this page says so instead of guessing.

  1. Meta for Developers, Meta Web Crawlers

Crawler access is the first thing I check in an AI answer audit. Most of what decides who gets named is what other sites say about you.

Check your robots.txt Or run the full AI Visibility Check