Skip to content

GPTBot

OpenAI's training crawler. It collects pages that may be used to train OpenAI's foundation models.

Check if your site lets GPTBot in Run the full AI Visibility Check

Last verified: October 2026, against OpenAI's own documentation.

GPTBot at a glance.

Owner
OpenAI
Purpose
Training. OpenAI says GPTBot crawls content that may be used to train its generative AI foundation models. Disallowing it tells OpenAI not to use your content for that training.
robots.txt token
GPTBot
Full user agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot

OpenAI says the version number may change. When it fetches robots.txt, it may add a robots.txt marker to the string.

IP ranges
openai.com/gptbot.json
Respects robots.txt?
Respects robots.txt. Yes. OpenAI manages GPTBot through the GPTBot robots.txt token, and says each of its tokens is independent: you can block GPTBot and still allow OAI-SearchBot.

The robots.txt rules to copy.

Your robots.txt lives at the root of your domain, like yoursite.com/robots.txt. On Shopify you edit it by adding a robots.txt.liquid template to your theme. Shopify warns that a wrong edit can cost you all your traffic, so change one rule at a time.

Allow GPTBot

User-agent: GPTBot
Allow: /

A bot with no rule is already allowed. Add this only when a broader rule, such as User-agent: * with Disallow: /, would otherwise block it.

Block GPTBot

User-agent: GPTBot
Disallow: /

Robots.txt rules apply per host. Repeat them on each subdomain you want covered.

Should you block GPTBot on a local business site or a Shopify store?

Blocking GPTBot tells OpenAI not to use your future pages for model training. OpenAI says the setting is independent of OAI-SearchBot, so a GPTBot rule on its own doesn't remove you from ChatGPT search.

For most small business sites, allowing it costs little: your service pages, policies and product facts are what you want a model to know. The case for blocking is content you sell, like a paid course or original research. I block it on this site because my writing is the product, and I keep OAI-SearchBot allowed so ChatGPT search can still cite me.

A rule you didn't write can block it too: an old robots.txt template, a security plugin or a CDN bot setting. The AI crawler checker reads your robots.txt for GPTBot and the other bots in this directory.

The 12 bots, sorted by job

  1. Search and answers

    3 bots

    They build the index an AI assistant searches when it answers.

    If you block themPer their owners, blocking one can drop you from that assistant's search answers.

  2. Fetching a page when a user asks

    3 bots

    They fetch your page because a person asked the assistant about it.

    If you block themBlocked at your firewall, the assistant can't read your page for the person who asked. Two of the three may not follow a robots.txt block.

  3. GPTBot is in this group

    Model training

    6 bots

    A block here tells the owner to keep your pages out of AI model training. CCBot is here because its open archive can be used that way.

    If you block themFor most, the owner says blocking doesn't remove you from its search features. Meta says its crawler also indexes content for its products.

Where GPTBot sits among the 12 AI bots in this directory, grouped by what each one does, with what blocking each group costs you.

Related crawlers.

Sources

Last verified: October 2026. Where the owner doesn't document something, this page says so instead of guessing.

  1. OpenAI, Overview of OpenAI Crawlers

Crawler access is the first thing I check in an AI answer audit. Most of what decides who gets named is what other sites say about you.

Check your robots.txt Or run the full AI Visibility Check