Blog
How to Measure AI Visibility When AI Answers Keep Changing
If you want to know how to measure AI visibility, start with one fact: AI answers change between runs, models and dates. A single screenshot shows what one answer looked like once; it doesn't measure anything. A measurement is a fixed set of prompts, run several times, with the conditions written down. Below: the protocol, the metrics, and a template for the table.
Checked against Google's documentation on October 20, 2026.
Contents
Why one answer isn't a measurement
Ask an AI engine a question, see your brand in the answer, and it feels like a result. It's an observation. Here is what you'll run into as soon as you repeat it:
- The same prompt, repeated in a new chat, can return a different answer and different brands.
- Different models, and different modes of the same product, answer differently.
- Answers change over time, even when nothing on your site changed.
- Google says its AI features can differ from each other: AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary (Google's guidance on AI features).
A measurement has to survive a repeat, or it's an anecdote. That's the point of everything below: measuring without fooling yourself. If you also wonder how this differs from classic SEO reporting, see how GEO and SEO measurement differ.
Tally · same prompt, 8 runs
[engine] · [model] · [date range]Answers change from run to run, so we count repeats instead of trusting one screenshot.
The measurement protocol
This is the protocol we use at GeoAeoLab. It has ten steps, and each one can be repeated next month under the same conditions. If you want to know how to measure AI visibility in a way you can repeat, don't skip steps; each one removes a way to fool yourself.
Define the cohort
Write a fixed set of prompts in the language your buyers use. Cover the situations that matter:
- finding a vendor
- comparing options
- looking for alternatives
- asking about price
- solving a specific problem
Give the set a version. Any change to any prompt makes it a new version. Treat your set as a hypothesis, not as measured demand.
Keep prompts unbranded and verbatim
Don't put your brand name in the prompts. A buyer who doesn't know you yet won't type it. Enter each prompt word for word, with no edits between runs. If you want branded checks, keep them as a separate group and never mix them into the main share.
Fix the conditions
Decide and record these before the first run:
- engine
- [engine]
- model
- if the interface shows it (otherwise "not shown")
- date
- [date]
- market and language
- for example US / en
- prompt set version
- [prompt set version]
- account state
- signed in or not, memory or personalization on or off
We record them because they can differ between runs.
Choose engines
A common base set is ChatGPT, Perplexity and Gemini. You can add AI Overviews and AI Mode in Google (see Google's guidance on AI features). Each engine is its own series, and results from one engine don't carry over to another. If you only want to track ChatGPT brand mentions on your own, run the same protocol on one engine, and don't generalize to the others.
Repeat every prompt
Run every prompt on every engine several times within the same date window. Our protocol uses at least three runs per prompt per engine, and we start each run in a fresh chat so earlier turns don't carry over. More runs give a steadier picture; how many you need is something you decide from the spread you see.
Log every run in full
One row per run. Don't skip rows and don't tidy up answers you don't like. The columns are in the template below.
Count across runs, not per screenshot
Calculate the metrics over all runs in the period. The result for a prompt is "named in k of n runs", not "named" or "not named".
Report the spread
Next to each metric, show how much it varied: in how many of the n runs the result was the same. A change smaller than the spread is noise, not progress.
Compare like with like
Compare two periods only if engine, prompt set version, market and language, and account state match. If they don't, write "not comparable". Same conditions, next month. Put model changes, new engine features and edits to your site in Notes (see why answers change).
Start with a baseline
Your first measurement is a baseline, not a grade. If your brand or site is new, it may not appear in the answers at all, and that is a result too. The most useful output of a first run is the list of competitors named and sources cited. Or have us run the baseline.
AI search visibility metrics worth tracking
Calculate every metric across all runs of the period, separately for each engine and prompt set.
| Metric | Definition |
|---|---|
| Mention rate | the share of runs in the period where your brand is named. |
| Citation rate | the share of runs where the answer links to a page on your site. A mention is not a citation; count them separately. When ChatGPT names you but links elsewhere, see ChatGPT brand citation mismatch. |
| Spread between runs | how much the result for one prompt differs across repeats within the same period. |
| Cited sources | which domains and pages the answers cite across the prompt set: yours, competitors' and third parties'. |
Share of voice gets its own section below.
Don't collapse these into a single "visibility score". One number glues engines together and hides the spread. If you also want to know how answers describe you, mark excerpts by hand as accurate or not; no scales or points needed.
Why answers change between runs
Some changes you can't control and can't see. So write down what you can see: a new model in the interface, a new engine feature, edits to your site, changes at competitors you noticed. Put each in the Notes column of the template, with a date. Then when you compare periods you have something to explain a shift, or you can say honestly "not explained". We don't know why an engine changed its answer; we log what we can see. The causes in detail: AI visibility prompt volatility.
The measurement template
One row is one run of one prompt on one engine.
Column | What goes in it | Why it matters |
|---|---|---|
Prompt ID | Your own code for the prompt in the set | Links runs of one prompt across periods |
Prompt text | The exact text as entered | Any edit makes it a different prompt |
Prompt set version | The version of the set | Only compare identical versions |
Engine | The platform you ran it on | Each engine is its own series |
Model | The model if shown, otherwise "not shown" | A model change can explain a shift |
Date | Date of the run, plus time if a series spans days | Defines the period's date window |
Run # | Repeat number for this prompt on this engine | Gives you k of n |
Market / language | For example US / en | Don't mix markets |
Account state | Signed in or not; memory or personalization on or off | A condition to hold constant |
Target brand named? | Yes or No | Basis of mention rate |
Brands mentioned | Every brand named, in order of appearance | Basis of share of voice |
Cited links | URLs the answer links to | Basis of citation rate and cited sources |
Answer excerpt | A short quote where your brand is named or should be | Lets you check and mark how you're described |
Notes | What changed: new model, engine feature, site edits | Explains shifts between periods |
Required
on every row: engine, model (or "not shown"), date and prompt set version.
A blank spreadsheet with these columns: no data, no sample results.
Common mistakes
- Treating one screenshot as the result.
Repeat and count instead. See step 5.
- Putting your brand in the prompt.
It skews the answer toward you. See step 2.
- Editing prompts between periods without a new version.
You're no longer comparing the same thing. See step 1.
- One number for all engines.
Each engine is its own series. See step 4.
- Mentions and citations in one column.
They answer different questions. See metrics.
- Comparing periods with different conditions, or calling a change smaller than the spread progress.
If you don't want to run it yourself
We run the same protocol as a service: AI visibility tracking. If you just need a one-time measurement to start from, have us run the baseline. We can't promise any result in AI answers; we work to improve the odds, measure, and report. If you're choosing a provider, see what to ask an agency about measurement.
AI visibility trackinghave us run the baselinewhat to ask an agency about measurement
Frequently asked questions
How many times should I run each prompt?
Should my brand name be in the prompts?
Not in your main prompt set. Run branded checks as a separate group so they don't inflate your share. See step 2.
Can I compare results from ChatGPT and Perplexity?
Side by side, yes; as one combined number, no. Results from one engine don't carry over to another, so keep each engine as its own series. See step 4.
Why did our visibility change when nothing changed on our site?
Answers change between runs, models and dates, so a shift isn't always about your site. First check whether the change is bigger than the spread and whether the conditions match. Then look at your Notes column. See why answers change and step 9.
Do I need a paid tool to measure AI visibility?
No. The protocol works in a spreadsheet. A tool can speed up runs, but check that it logs engine, model, date and every run. For Google specifically, its documentation states: "No third-party tool has access to our internal ranking or AI systems" (Google's guide to generative AI search).
Can GeoAeoLab run this for us?
Yes. We run the same protocol as AI visibility tracking, or as a one-time baseline audit. We don't guarantee results in AI answers.
Download the measurement template
Start with the table and a first run of your prompt set; that's how to measure AI visibility without guessing.
Or start with an AI visibility audit.