How to check if ChatGPT can see your website
Whether ChatGPT can read your site comes down to a few things you control: your robots.txt, your firewall or CDN, and how your pages are marked up. This is how to check each one, free, and how to keep an eye on them afterwards.
- ChatGPT's search uses OAI-SearchBot and ChatGPT-User. GPTBot is the training crawler. You can block training and still stay open to search.
- Start with yoursite.com/robots.txt: look for Disallow rules under the search agents, or a blanket Disallow for every bot.
- A firewall or CDN can turn AI bots away even when robots.txt allows them, and robots.txt will not show it.
- The free AI Visibility Checker runs all 9 AI Visibility checks on any URL, no signup.
- Access can break after a plugin update, a CDN setting change or a redesign, so check it again after each one.
ChatGPT can only mention or link to a page it can reach and read. If its search agents are blocked, your pages cannot appear in an answer, however good they are.
Whether they can get in is decided by three things on your own site: one text file, your firewall or CDN settings, and how your pages are marked up. You can check all three in a few minutes, without a developer.
What does "visible to ChatGPT" actually mean?
OpenAI sends more than one bot to websites, and they do different jobs. Two of them fetch pages to answer questions people are asking right now:
- OAI-SearchBot fetches pages for ChatGPT search results.
- ChatGPT-User fetches pages when ChatGPT browses live during a conversation.
A third, GPTBot, collects pages to train OpenAI's models. It has nothing to do with whether you show up in an answer today. OpenAI lists all three in its crawler documentation.
The split matters because many sites block GPTBot on purpose, as a licensing choice. That is fine. Blocking training does not block search. As long as OAI-SearchBot and ChatGPT-User are allowed, you stay eligible to be cited. The trouble starts when a copy-pasted "block all bots" rule sweeps up the search agents too.
The same pattern holds beyond ChatGPT. Perplexity, Claude and others have their own search agents and their own training crawlers, and our AI crawler access guide lists each one with the robots.txt rules to allow or block it.
One caveat: being readable does not guarantee being cited. Nothing you change on your site forces an AI answer to include you. These checks remove the reasons it would skip you.
How do I check it myself in five minutes?
1. Read your robots.txt
Type your domain into a browser followed by /robots.txt, for example yoursite.com/robots.txt. It is a plain text file of rules for bots.
Look for two things:
- A group that starts
User-agent: OAI-SearchBotorUser-agent: ChatGPT-Userand hasDisallow: /under it. That blocks the agent from your whole site. - A group that starts
User-agent: *withDisallow: /under it. That blocks every bot, the search agents included.
A Disallow: / under User-agent: GPTBot on its own is the training opt-out described above. It does not stop ChatGPT search.
If you see lines wrapped in # BEGIN Cloudflare Managed content comments, or a line starting Content-Signal:, Cloudflare is adding rules to your file at its edge. Editing your own copy will not change those. The setting lives in the Cloudflare dashboard, under your zone, in AI Crawl Control.
2. Check your firewall or CDN
Cloudflare and other providers can block AI crawlers at the network layer. When that is on, a request that identifies as an AI bot gets an error such as 403, while a normal browser loads the same page fine. Your robots.txt can say "come in" and the bot still bounces off.
You cannot see this from robots.txt, because the block never touches it. Look in your CDN or host's bot settings for anything that blocks or challenges AI bots, and check whether the search agents are exempt. If you are not sure what is switched on, the next step tests it.
3. Run the free checker
The AI Visibility Checker runs all 9 AI Visibility checks on any URL, free, with no signup. It reads your robots.txt and sorts the agents into search and training, and it requests your page twice, once as a normal browser and once as an AI bot, then warns only when the browser gets in and the bot does not. The second request is the firewall test, the one you cannot do by reading a file.
To go through each check by hand, follow the AI search visibility audit guide. The AI Visibility page explains how the nine checks become one score.
What do the nine checks look at?
| Check | What it looks at |
|---|---|
| llms.txt file | A short plain-text summary of your site written for AI assistants. |
| AI crawler access | Whether the bots behind ChatGPT, Perplexity and Claude may read your pages. |
| AI crawler firewall blocking | A firewall or CDN turning AI bots away even when robots.txt allows them. |
| AI citation schema | Markup that names your organisation so AI answers can credit you. |
| Answer engine schema | FAQ and how-to markup that AI answers lift from. |
| Semantic content structure | The article kept apart from menus and footers, so AI reads the right part. |
| Content freshness signals | Machine-readable publish and update dates. |
| Author attribution schema | Markup naming who wrote the page. |
| Question-format headings | Headings phrased as the questions people ask. |
The first three decide whether the bots can read your pages at all, so fix those first. The other six make a page easier to quote and credit once it has been read. Our guide to getting cited by AI covers how to fix each of them.
A word on llms.txt: the check flags a missing file, but it carries little weight. The evidence so far says AI crawlers rarely request the file, and we wrote up why in Is llms.txt actually working?. Publish one if it takes ten minutes, then spend the real effort on access and structure.
The checker covers one URL. A full AuditZap audit runs the same 9 checks across the pages it crawls, as part of its 40 checks.
Why can it break without me noticing?
A site that passes today can fail next month. Access usually breaks as a side effect of something else:
- A plugin or theme update rewrites robots.txt. On WordPress and similar systems the file is often generated by the CMS or an SEO plugin, so an update or a changed setting can add rules nobody wrote by hand.
- A CDN or host changes its bot settings. Cloudflare can add its own rules to the robots.txt it serves when AI Crawl Control is on, and providers can block AI crawlers at the firewall on their default settings. A plan change or a new security setting can switch either on.
- A redesign drops the markup. A new theme or page builder can lose the schema, the author markup or the publish dates the old pages had. The page looks the same to you and reads differently to a bot.
None of these send you a warning.
How do I keep an eye on it?
Checking once is not enough, so check again after every change. Every AuditZap account includes weekly monitoring, which does it for you:
- Free re-checks 1 site every Tuesday, on top of your 3 audits a month. It is the site you audited most recently.
- Pro re-checks up to 5 sites every Monday.
- Agency re-checks up to 25.
Once a site has been audited more than once, AuditZap shows how its AI Visibility score has changed since the last audit. On the report the change sits beside the score, as an arrow and the number of points it moved, with a small trend line. Your Dashboard shows the same for each site. The email you get after a monitoring re-check includes it too, so a drop after a plugin update or a CDN change reaches your inbox that week. Our post on weekly monitoring for Free accounts explains how the re-checks work.
What should I do today?
- Open your robots.txt and make sure OAI-SearchBot and ChatGPT-User are not blocked, by name or by a blanket rule.
- Run the AI Visibility Checker on your homepage and your most important page, and fix anything it flags under access or firewall blocking first.
- Run a full audit so weekly monitoring has a site to re-check, then look at the AI Visibility trend after each change you make.
Get one AI fix instruction tailored to your platform, free. 3 audits a month, no card. Pro unlocks fixes for every issue.