Google Says It Can't Report AI Search Properly Yet — and Is Asking the Industry for Ideas

Replying to a thread in r/SEO, Google's John Mueller acknowledged that Search Console's generative AI performance reporting doesn't do the job — and that Google doesn't currently have a better answer.

On position reporting specifically:

"Position for these is hard to do in a way that makes it useful, so we're currently tracking it like we do for many search features (as a block), & it's not separated out in the Gen-AI performance report."

And on the documentation:

"This is pretty much it – we tried to document it as clearly as possible in the help center page (which you can find if you search for 'impressions clicks search console'), it's a giant doc in the meantime."

"This is pretty much it" is doing a lot of work in that sentence.

The four specific problems

Position is the block, not your link. Search Console reports where the AI feature sat on the page, not where your link sat inside it. Being the first source in an AI Overview and being the eighth are the same number in your reporting, despite being entirely different outcomes.

Impressions count links nobody saw. A URL can register an impression in an AI feature without the user ever seeing it. Your impression count overstates real visibility, and there's no way to tell by how much.

Content behind "Show More" isn't counted at all. The inverse problem. Where your presence sits behind an expansion the user didn't click, it doesn't count until expanded — so the same report understates presence in a different set of cases.

The whole frame is ten blue links. Impressions, clicks, position and CTR were built for a ranked list of results. AI Mode isn't a ranked list, and metrics designed for one describe it badly.

The awkward part is the direction of the errors. One mechanism inflates your numbers, another deflates them, and you can't separate them. That isn't a number with a known margin of error — it's a number whose bias you can't determine.

Why this matters more this week than last

Google is currently piloting a scheme that pays publishers when their content significantly contributes to AI answers. That's the AI Contribution pilot, and it has no published payout formula.

Put the two together. Google is building compensation on top of a measurement layer it has just conceded doesn't describe the surface accurately. Publishers in the pilot see a monthly figure they can't audit, in a reporting environment Google says thinks in ten blue links.

None of which means the pilot is bad faith. It does mean the obvious next question — "is this payment fair?" — currently has no way of being answered by either side.

The genuinely encouraging part

Mueller is asking the industry for better ideas, and that's not nothing.

Most of the measurement complaints of the last two years have been met with reassurance that everything is fine. This is a Google engineer saying the reporting is hard, the current approach is a placeholder, and better suggestions are welcome. That's an open door, and the SEO industry has a reasonable track record of walking through those when they appear.

It's also a rare admission against interest. Google has no commercial reason to tell you its visibility metrics are unreliable.

What to do with it

Stop reporting AI impressions as visibility. They are not. If AI impression counts are in a client dashboard labelled as reach or visibility, that label is wrong in a way you can now cite Google on.

Never use AI position as a ranking. It describes the feature's position, not yours. A chart of "average position in AI features" over time is measuring where Google put the block, which is a Google decision rather than a performance signal.

Report direction, not magnitude. Changes over time in a consistently-biased metric still carry information. The absolute numbers don't.

Say so in your reporting. A single line — that Google has publicly acknowledged these metrics are inadequate for AI surfaces, with a link — protects you from a client discovering it later and is simply accurate.

Contribute an idea if you have one. The request is open. Anyone who has spent two years building workarounds for this has more practical knowledge of the problem than most.

Sources

• Search Console Help: Impressions, position, and clicks — the "giant doc" Mueller refers to
• Search Console Help: Generative AI performance report (Search)
• Google Search Central Blog: Introducing Search Generative AI performance reports in Search Console
• Search Engine Journal: Google Admits Search Console Reporting For AI Search Is Inadequate — surfaced and reported Mueller's Reddit remarks