The Honest Answer: Three Things, Not One
"AI visibility" gets talked about as if it were a single number, like a keyword ranking. It isn't. It's three different things stacked together, and they have very different levels of proof behind them.
Here is what you can actually know, and where each number comes from:
| What you want to know | Can you measure it? | Where the number comes from |
|---|---|---|
| Did an AI send me a visitor? | Yes — countable | GA4 AI Assistant channel and referrer data |
| Did my page appear inside a Google AI Overview? | Sometimes — a raw count | Search Console Generative AI report, where it has rolled out |
| Which search triggered that AI Overview? | No | Google does not report it anywhere |
| Does ChatGPT name my brand for buying questions? | Only by sampling | Prompt testing, repeated on a fixed schedule |
| Do I get named more often than my competitor? | Only by sampling | Your own prompt panel, same prompts every month |
| How many people saw my brand and never clicked? | No — this is the guess | Nothing reports it. Estimates are modeled. |
If a tool hands you one confident number that covers all six rows, it is blending measured data with modeled data and not telling you the ratio. That's the part worth being skeptical about.
What We Tested, and How
Rather than repeat what other blogs say, we went to the sources directly on July 28, 2026 and ran three checks — the first two across all eight Search Console properties we manage, the third against our own Analytics property.
- Search Console API, search types.We asked the Search Analytics API for AI data by passing
aiModeandgenerativeAias the search type. - Search Console API, search appearance.We asked every property for the
searchAppearancedimension, which is where special result types like rich results get broken out. - GA4 channel data.We pulled 90 days of sessions by channel group and by traffic source, looking specifically for AI assistant referrers.
All three checks returned something useful. Two of them returned nothing at all — which is itself the finding.
Finding 1: Search Console Has an AI Report, but the API Doesn't
Google launched a dedicated Generative AI performance report inside Search Console on June 3, 2026. It splits AI Overview and AI Mode impressions out of the main Performance report for the first time. That's real progress, and it's worth checking whether your property has it.
But read the limits carefully before you build a report around it:
- Impressions only.No clicks, no click-through rate, and no query data. You can see that you appeared. You cannot see what you appeared for.
- No history.Data starts May 18, 2026. Google did not backfill anything before that date.
- Phased rollout.It went to a portion of UK properties first. There is no published eligibility list and no way to opt in. It simply appears when your property is included.
- One impression, not two.If the same page appears inside the AI summary and in the normal results for a single search, Google counts that as one combined impression.
Then there's the part nobody mentions. When we asked the Search Console API for that same data, it refused. Passing aiMode or generativeAi as the search type returns an HTTP 400 error: Invalid value at 'type'. The only search types the API accepts are web, image, video, news, googleNews, and discover.
We also requested the searchAppearance dimension — the field where Google breaks out special result types — across all eight properties, over a 180-day window. Seven returned zero rows, with no error. The eighth returned exactly one row, for an ordinary review snippet. Not one AI-related appearance type came back on any property.
So the Generative AI report is a screen you look at, not data you can pull. You can't automate it, can't chart it next to your other numbers, and can't join it to anything. If your agency shows you an automated AI Overview dashboard sourced from Search Console, ask them which endpoint it came from.
Finding 2: GA4 Added an AI Assistant Channel — and It Leaks
This is the genuinely good news. GA4 now has a default channel group called AI Assistant, which catches visits arriving from tools like ChatGPT. That is real, countable traffic — a person clicked a link inside an AI answer and landed on your site.
On our own site over the last 90 days, the AI Assistant channel recorded more sessions than the Organic Search channel did. For a site that has been doing SEO for years, that's a genuinely strange sentence to write.
But the channel undercounts. When we pulled the same 90 days by traffic source instead of by channel, GA4 showed more sessions from chatgpt.com than it had placed in the AI Assistant channel. The rest were filed under plain Referral.
That gap matters. If you report on the AI Assistant channel alone, you will understate your AI traffic and won't know by how much. Two fixes, both quick:
- Report on source, not just channel.Build an exploration on Session source and look at the AI domains directly. It's the raw number, before GA4's classification touches it.
- Build your own channel group.Create a custom channel group in GA4 Admin that matches AI referrers by pattern, so new assistants get caught as they appear instead of falling into Referral.
- Check it quarterly.New AI assistants launch constantly. A rule you wrote six months ago is already missing traffic.
Finding 3: Search Console and GA4 Will Never Agree
Any time you compare Search Console clicks to GA4 sessions for the same period, the two numbers will differ. Often by a lot. This is normal and it is not a bug in either tool.
They count different events. Search Console counts a click on a result in Google. GA4 counts a session that fired a tag on your site. Between those two moments you lose visitors to consent banners, ad blockers, slow pages people abandon, redirects that drop the referrer, and privacy settings that strip it entirely.
The practical rule: don't try to reconcile them. Pick one as your trend line and stay with it. Use Search Console to understand what Google is showing you for. Use GA4 to understand what people did once they arrived. Comparing the two absolute numbers tells you nothing except that measurement is lossy.
The Three Tiers: Countable, Sampleable, Unknowable
Once you accept that no single tool sees everything, the useful move is to sort every AI visibility metric into one of three buckets and treat each one differently.
| Tier | What it covers | How much to trust it |
|---|---|---|
| 1. Countable | Sessions from AI referrers, AI crawler hits in your server logs, conversions from AI traffic | High. This is a census, not a sample. Report it as a hard number. |
| 2. Sampleable | Citation rate, share of voice against competitors, sentiment in AI answers | Medium. Real, but it's a survey. Only meaningful as a trend across a fixed prompt set. |
| 3. Unknowable | Total AI impressions across all platforms, brand mentions that led to no click | Low. Any figure here is modeled. Treat it as directional, never as a KPI. |
Most disappointment with AI visibility tracking comes from treating a tier 2 or tier 3 number like a tier 1 number — putting a modeled estimate in a board deck as if it were counted.
How to Build a Setup That Isn't a Guess
Here's the setup we run. None of it requires an expensive platform, and every part of it produces a number you can defend.
- Turn on the countable layer first.Get AI referrer traffic reporting correctly in GA4 by source, not just by channel. This is the only AI number with no estimation in it. Start here.
- Check your server logs for AI crawlers.GPTBot, PerplexityBot, ClaudeBot and Google-Extended show up in your logs. If they're never crawling you, no amount of content strategy will get you cited. This is a hard yes or no.
- Write a fixed prompt set — 20 to 40 questions.Use the questions your actual buyers ask, in their words. Do not change the list between runs. The whole value is in comparing the same prompts over time.
- Run the set monthly, on the same day, across every model that matters to you.Record three things per prompt: were you named, were you linked, and who else was named.
- Never trust a single spot-check.AI answers vary between users, sessions, and even reruns of the same prompt. One test tells you almost nothing. Ten runs of the same prompt tells you a rate.
- Look at the Generative AI report in Search Console if you have it— but as a sanity check on direction, not as a KPI. It's impressions with no context.
For the full metric definitions and benchmark targets behind this, see our companion guide on how to measure AI search visibility.
Mistakes That Make the Number Meaningless
We see the same handful of errors turn a reasonable measurement effort into noise:
- Changing the prompt list between runs.Now you're comparing two different surveys. The trend line is worthless.
- Asking the AI about itself.Prompting "do you recommend [my brand]?" is a leading question. Ask the buying question a real customer would ask and see whether you come up unprompted.
- Counting a mention as a citation.Being named in prose and being linked are different outcomes with different traffic value. Track them separately.
- Reporting a percentage without the sample size."40% citation rate" from four prompts is not a finding. Always publish the denominator.
- Assuming a rank tracker sees AI answers.Traditional rank trackers were built for a list of ten blue links. A generated answer has no position to track.
So — Measurable, or a Guess?
Both, and the split is knowable.
The traffic AI sends you is measurable today, precisely, in tools you already own. The rate at which AI names your brand is measurable as a sample, the same way a poll measures an election — real, useful, and honest only when you publish the method and the sample size alongside it.
Everything past that is a model. Total AI impressions, unclicked brand exposure, "AI market share" — those are estimates dressed as measurements. They aren't useless, but they belong in a slide about direction, not in a slide about results.
The agencies worth hiring will tell you which of your numbers is which. If you'd like us to look at what your own stack can currently see, that's what our AI visibility work starts with.