AI Search Visibility: How to Check Whether AI Engines Can Read Your Site

A website can rank on the first page of Google and return almost nothing to ChatGPT. The two outcomes are unrelated, and the second one is invisible from inside any analytics tool you already use.

There is no report for it. Nothing drops, because nothing was ever there. The only way to find out is to ask.

The check

curl -A "GPTBot" yoursite.com | wc -w

This requests your homepage while identifying as OpenAI's crawler, then counts the words that come back. On a Mac, open Terminal and paste it. On Windows, use PowerShell or ask any AI assistant to run it for you.

Run it against your main pages, not just the homepage. Pricing and service pages are the ones that get cited in a recommendation.

Why a full website returns five words

Sites built with React, Vue, Angular, Svelte and similar frameworks send an almost-empty HTML file. The browser then downloads a JavaScript bundle, executes it, and assembles the page. It takes a fraction of a second, so no human notices.

Crawlers can stop before that step.

Google executes JavaScript, but on a delayed second pass, so its index eventually fills in. Most AI crawlers do not execute JavaScript at all. They read the server's first response and move on. If that response is a shell, the shell is what they store.

This is why the two systems disagree, and why the disagreement is so easy to miss. Nothing is broken. Google is fine. The site looks perfect. And a set of increasingly important systems see an empty page.

Traditional WordPress sites, static sites, and anything server-rendered generally pass this test without effort. The risk concentrates in modern, well-built, often expensive sites — which is the counterintuitive part.

The second gate: permission

Rendering determines what a crawler can read. robots.txt determines whether it's allowed to look.

Open yoursite.com/robots.txt and search for these names:

Google-Extended deserves particular attention. It governs whether Google's AI products may use your content and has no bearing on ranking whatsoever. Block it and every ranking stays exactly where it was. You simply stop appearing in AI Overviews, with no notification and no diagnostic.

Most robots.txt files never mention any of these names. That is not a decision to opt out — it is the absence of a decision, usually made by whatever template or plugin generated the file.

Fixing it, in order

1. Rendering, before anything else. No amount of content work matters while crawlers receive an empty page. The fix is prerendering or server-side rendering, depending on the framework — generating real HTML at build time instead of assembling it in each visitor's browser. This is an engineering task, typically one to two days for a small marketing site.

The quickest way to explain it to a developer is to show them the curl output. The number does the arguing.

2. Crawler permissions, deliberately. Add explicit rules for the crawlers you want, and leave out the ones you don't. Blocking AI crawlers is a legitimate business decision — publishers who sell access to content have real reasons — but it should be a decision.

3. Structure that survives extraction. AI systems pull passages rather than whole pages. Clear headings, a plain definition near the top of important pages, and FAQ content marked up with FAQPage structured data all make a page easier to quote accurately.

4. Freshness signals. datePublished and dateModified in your structured data. Without them, recency is unknown, and unknown recency reduces confidence.

5. Register with Bing. ChatGPT search draws on Bing's index, which makes Bing Webmaster Tools disproportionately useful and almost universally skipped. It takes about twenty minutes.

6. Re-run the check. Same command. The number should be in the hundreds.

What this does not fix

Being readable is necessary, not sufficient. Once crawlers can reach your content, whether they cite it depends on the content itself.

Google's own guidance is direct about what earns that: content that is "unique, compelling, and useful" and specifically not commodity material that could have been produced by anyone — or by a language model. A page of generic advice that is perfectly readable to every crawler will still lose to a page with something specific in it.

Access gets you considered. Substance gets you cited.

Measuring afterwards

For Google, generative AI performance data is now available in Search Console, so AI Overview visibility can be observed rather than guessed.

For ChatGPT, Claude and Perplexity there is no equivalent reporting. The practical substitute is a monthly manual check: ask each engine the questions your customers would actually ask, record whether you appear, and note which sources it cited instead.

That last detail is the useful one. The cited source tells you where to be — usually a comparison page or a roundup you're not included in yet, which is a different job from anything on your own website.

Questions