Blog

AI Visibility Prompt Volatility: Why a GEO Report Can Change Without Site Changes

AI visibility prompt volatility is the reason a GEO report can move up or down from one month to the next while nothing on your site changed. AI answers differ between runs, models and dates, so the same prompt can return a different set of brands and sources. We call this prompt drift.

This article explains why results change and how to read the change. If you need the measurement method itself, see how to measure it. GeoAeoLab builds its reports on that method. See monitoring sample.

Contents

What prompt drift means

"Prompt drift" is not a term from any platform's documentation. It is the working vocabulary we use here, and it helps to separate three things:

Run-to-run volatility. The spread between runs of the same prompt under the same conditions.
Drift. A shift over time with the same prompts and the same conditions.
Condition change. The model, mode, account state, market or version of the prompt set changed.

The first is noise. The second is a signal, or a change on the engine's side. With the third, the two periods are not comparable at all. Telling these apart is most of the work of reading AI visibility prompt volatility.

Causes at a glance

Here is the short version: seven causes, how well each one is documented, and where to look first in a report.

Cause	Documented?	What it looks like in a report	Where to look first
Same prompt, different answer	Yes (OpenAI API docs)	Brand appears in some runs, not others	The spread
Model changed behind the interface	Yes (OpenAI release notes; Google)	Many prompts shift on one engine at once	The change log
Live search picked other sources	Yes (Google, OpenAI, Perplexity)	Cited sources change, then mentions	Cited sources
Prompt wording or set version changed	Partly (API docs; otherwise hypothesis)	Shift starts with a new prompt set version	Prompt set version
Location and language	Location: OpenAI; language: hypothesis	Results differ by market	Market / language field
Account, memory, personalization	Yes (OpenAI, Google)	Signed-in runs differ from signed-out runs	Account state
The web changed (competitors, publishers)	Hypothesis	A new competitor appears with new cited pages	Cited sources and notes

Why a GEO report changes when your site didn't

If you are asking why AI answers change, the short answer is that several things sit between your site and the answer. Each of the six causes below either has an official source or is marked as a hypothesis. Engines differ, so compare like with like.

Same prompt, different answer

A single prompt can produce a different answer on each run, so a brand may appear in some runs and be missing from others. OpenAI's API documentation says Chat Completions are non-deterministic by default: outputs may differ from request to request, and even with a fixed seed the results are only "mostly" the same (OpenAI API docs).

That is documentation for the API, not for the ChatGPT app, and we do not claim the app behaves the same way. It does show that run-to-run variation is expected rather than a malfunction, which is why one number without a spread tells you very little. For how that variation shows up in practice, see why answers change between runs.

The model behind the interface changed

The models behind an AI product are added, updated and retired on specific dates. OpenAI publishes model release notes for ChatGPT to show when models are added, updated and retired. OpenAI's help article What to expect when models change covers what changes for existing conversations. Google says AI Overviews and AI Mode may use different models and techniques, so the responses and the links shown can differ between them (Google Search: AI features).

In a report this usually looks like many prompts shifting at once on a single engine. So a model update can change your AI visibility without any change on your site: you see it only in the engine's own notes and in your change log.

Live search picked different sources

Many AI answers are built from a live web search, and the search itself can vary. Google says its AI features may use "query fan-out", running several related searches across subtopics and sources, and that showing up in them is not guaranteed (Google Search: AI features). OpenAI says ChatGPT can search the web when it decides a question needs it, and that ChatGPT search typically rewrites your question into one or more targeted queries for its search providers (Searching the web with ChatGPT). Perplexity says it searches the internet in real time and shows numbered citations (How does Perplexity work?).

The consequence is that the cited sources can change from run to run, and the brand mentions that depend on them can change with them. This is a hypothesis, not documented behavior: new pages from competitors and publishers, review sites or forums probably change what live search finds. We don't know how much. For the origin of those sources, see where cited sources come from.

The prompt or the prompt set changed

This cause sits on the reporting side, not the engine's. For the apps it is a hypothesis that small wording changes can change the answer, which is why the prompt set is versioned and periods are compared only when the version is the same. For the API the wording is one of the inputs to a request (OpenAI API docs); for the apps we do not know how sensitive the engines are.

A report that quietly swapped five prompts for five new ones will show a shift that has nothing to do with the engines.

Location, language and market

OpenAI says ChatGPT search estimates a general location from your IP address and may share it with third-party search providers (Searching the web with ChatGPT). So results for different markets are different series and should not be added together.

Language is a hypothesis: we have no official source on how the language of the prompt or the interface changes which brands appear. Treat each language as its own series as well, and mark the field in the report.

Account, memory and personalization

AI search personalization is documented in places. OpenAI says saved memories and chat history in ChatGPT may influence responses, and that Temporary Chat does not create or update memories (Memory in ChatGPT; Temporary Chat in ChatGPT). Google says AI Mode may personalize responses using your past searches and activity if you are 18 or older and have history and personalized recommendations turned on; its memory feature is available in the US, in English (Google Search Help).

So the state of the account (signed in or signed out, memory on or off) is part of the conditions of a measurement. A run from your own signed-in browser and a run from a clean session are not the same observation.

What's documented and what isn't

Part of the picture is described in official documentation. The rest is hypothesis, and from the outside you cannot see why an engine picked a particular answer or page. The practical rule is to log what you can see and label what you cannot. Here is what we don't know:

how much changes on competitor and publisher sites move the answers;
how the language of the prompt changes the set of brands;
why brands appear in a given order, and why a given page was chosen;
how long it takes engines to take new content into account.

How to read a change in a GEO report

Work through these questions in order before reacting to any AI visibility fluctuations. This is how to interpret a finished report, not how to run the measurement.

Is the change bigger than the spread? If not, treat it as noise. Read more about the spread.
Were the conditions the same in both periods? Engine, model as shown in the interface, prompt set version, market and account state. If not, the periods are not comparable. See how periods stay comparable.
What does the change log say? A new model, a new engine feature, a new prompt set version or an edit on your site can each explain a shift. Check the change log.
One engine or all? One group of prompts or all? A shift on one engine across many prompts is likely an engine change. A shift on one topic across all engines is likely about content or sources. This is a working rule, not a law.
Did the cited sources change first? Compare cited sources over time.
Nothing explains it? Write "not explained". Do not invent a cause.

Run-to-run variance chart: how to read it

A run-to-run variance chart shows how much a result moves between runs before anything has changed. Read it by looking at the shaded band, which is the spread between runs. A point outside the band deserves a closer look. A vertical marker shows an event from the change log, so you can see whether a shift lines up with a known change.

What a report needs so changes can be read

These are the fields without which a change cannot be read. Each belongs in the report:

Engine.
Model, as shown in the interface, otherwise "not shown".
Date of each run.
Prompt set (cohort) version.
Number of runs.
Spread next to every number: a range, not one number.

AI answers change between runs, models and dates, so a measurement is repeated runs of a cohort of prompts with the engine, model and date recorded, not a single screenshot. For the steps, see how to run the measurement.

What not to do

Don't react to a single screenshot.
Don't compare periods with different conditions.
Don't "fix" noise with edits to your site.
Don't try to stabilize answers with gray methods: prompt injection, hidden text for models, mass-produced pages written for prompts, hidden paid mentions or fake reviews. These break the rules of search engines and platforms (see Google's spam policies). We don't do this.

How GeoAeoLab handles prompt drift

In monitoring we run a cohort of prompts repeatedly, record the engine, model and date, keep the spread next to every number and maintain a change log. We improve the chances of being mentioned, measure and report; we do not promise a mention, a citation or a stable result. See monitoring sample.

If you need a one-time starting point first, start with an AI visibility audit.

FAQ

What is AI visibility prompt volatility?

It is the spread and shift in results for the same prompts between runs, models and dates. We separate run-to-run volatility, drift and condition change; see what prompt drift means.

Why does ChatGPT give different answers to the same question?

OpenAI's API documentation says Chat Completions are non-deterministic by default: outputs may differ from request to request. That is about the API, not the ChatGPT app. For the app, OpenAI says ChatGPT search typically rewrites your question into search queries and may use a general location, and that memory and chat history may influence responses. We don't claim more about how the app works. See same prompt, different answer.

Can a model update change our AI visibility?

Yes, it can. Models in the interface change on specific dates, and Google says its AI features may use different models. You may see it in the change log as a shift across many prompts on one engine. See the model behind the interface changed.

Does personalization affect AI visibility results?

It can. OpenAI says memory and chat history may influence responses, and Google says AI Mode may personalize responses for users who have history and personalized recommendations turned on. That is why the account state is recorded for every run. See account, memory and personalization.

How big a change is worth acting on?

One that is larger than the spread and seen under comparable conditions. If it also repeats in the next period, that is a reason to investigate. We do not give a numeric threshold, because it depends on your own spread. See how to read a change.