
What matters first.
- Does ChatGPT give the same answer to everyone? No, and not even to the same person twice: ChatGPT samples every reply word by word, so the same question can name different brands from one run to the next, and memory, plan and location add further differences between people.
- One answer or one screenshot says little about your brand. A reading you can defend is a rate across a fixed set of buyer questions and many runs, on one engine, country and period, reported with its interval.
- To make a difference count, add questions before you add repeats: answers to one question resemble each other, so 50 answers to 50 questions tell you more than 50 answers to 10.
For marketing leads, founders and SEO leads holding two ChatGPT answers that disagree: yours and a colleague’s, or today’s and last week’s screenshot. Most guides list why answers vary; this page pairs each cause with who controls it, gives you a repeat test to run, and shows how many questions and runs it takes before a difference between two readings counts.
FameScan asks the buyer questions you choose in up to five engines (ChatGPT, Perplexity, Gemini, Claude, Google AI Overviews) again and again, stores every answer with its sources, and reports how often your brand is named as a rate with a 95% interval, not as one answer.
Does ChatGPT give the same answer to everyone?
The same question gets a fresh draw every time: each ChatGPT reply is written one word at a time, and each next word is drawn from a list of likely candidates. OpenAI’s API reference describes the dial behind that draw, temperature, as making output “more random” at higher values and “more focused and deterministic” at lower ones. A small difference early in a reply, such as one brand named first instead of another, can change what follows.
For a brand, the draw matters most in recommendation questions. In a study published by SparkToro on January 27, 2026, 600 volunteers, each keeping their usual settings, ran 12 recommendation prompts through ChatGPT, Claude and Google’s AI a combined 2,961 times in November and December 2025. ChatGPT and Google’s AI returned the same list of brands in fewer than 1 in 100 repeated runs, and the same list in the same order closer to 1 in 1,000.
Between two people, more differences can add to the draw: the chat so far, memory, plan, location and that day’s search results. Two screenshots that disagree are therefore expected, and neither is a mistake. Each is a single draw.
Why does ChatGPT give me different answers to the same question?
Eight causes change a ChatGPT answer, and you control only some of them. The table pairs each cause with what it changes and who controls it, taken from OpenAI’s help pages and two named studies, all opened on November 20, 2026.
| Cause | What it changes | Who controls it |
|---|---|---|
| Sampling of each word | Which brands, their order, the length | OpenAI, not you |
| Server-side variation | Wording, even at temperature 0 (shown on an open model) | Nobody |
| The chat so far, memory, chat history | What ChatGPT assumes about you | You: Temporary Chat, Memory settings |
| Custom instructions | Focus and tone | You |
| Plan, chosen model, model updates | Which model answers, how much memory | You and OpenAI |
| Web search | Sources and recent facts | Mostly ChatGPT; you can pick Search |
| Location | Local results, the search query | Partly you: VPN, location setting |
| Wording and intent of the question | Which set of brands is drawn on | You |
Sources: OpenAI Help Center, Memory in ChatGPT, Temporary chat in ChatGPT and Searching the web with ChatGPT and Model Release Notes; the ChatGPT pricing page (Free, Go, Plus and Pro; Plus adds advanced reasoning models, Free has limited memory and context); Thinking Machines Lab, September 10, 2025; SparkToro, January 27, 2026. All opened on November 20, 2026.
Web search starts on its own: ChatGPT “may search the web automatically when your question would benefit from current information”, so one run of a question can cite fresh pages and the next can answer from the model alone. The wording row has a limit worth knowing. SparkToro found that 142 people asking for the same thing, headphones for a traveling family member, wrote very different prompts, yet the answers kept drawing on the same few brands; asking for gaming headphones instead produced a different list. A new intent changes the answer far more than a new phrasing.
OpenAI also updates the models behind ChatGPT, so last week’s screenshot may come from a different model than today’s answer.
Two rows decide how you test: the ones nobody controls. Sampling and server-side variation stay when you remove every personal setting, so one clean answer is still one draw.
Does ChatGPT tailor answers to you?
Yes, it can, and that matters most for the person most likely to check: you. With Memory on, ChatGPT can draw on past chats, saved memories and custom instructions, and depending on plan and region on saved files and connected apps; OpenAI’s help page adds that memory can also shape the web searches it runs. A founder who has discussed their company in ChatGPT for months asks about their category with that history in the room.
Location works the same way. ChatGPT may estimate your general location from your IP address, down to country, state or city, and use it to rewrite a search query, and a VPN can change that estimate.
Can you chat with ChatGPT for free? Yes: ChatGPT’s Free tier gets limited memory and context and fewer models than Plus, so a colleague on Free and you on Plus are not asking quite the same ChatGPT.
Does ChatGPT save all your questions? Regular chats stay in your history until you delete them. A Temporary Chat stays out of history unless you save it, is not used to improve OpenAI’s models while it stays temporary, and OpenAI may keep a copy for up to 30 days for safety purposes, according to its help page.
To remove the personal layer, open a Temporary Chat and choose Unpersonalized before the first message: it then uses no memory, custom instructions or plugins, and it creates no memories. A logged-out browser window has no account history at all and can still search the web. What stays is the model, an approximate location and that day’s results.
A colleague’s screenshot therefore shows one draw from their account, plan, location and day. It shows that your brand can be left out of an answer; it cannot show how often, or which cause left it out. If you need to know what one specific customer sees, only their own account shows it; FameScan measures the answers to a fixed question set in each engine, with the country as a setting and nobody’s memory in the way.
Which answers stay stable, and which change?
Closed questions with one right answer mostly change in wording, not in substance. When Thinking Machines Lab sent one biography question 1,000 times at temperature 0, the replies first split at the 103rd token, on details such as “Queens, New York” against “New York City”: two ways of naming the same place.
Recommendation questions change the most, and position changes more than presence. In SparkToro’s data, City of Hope appeared in 69 of 71 ChatGPT answers to a question about cancer care hospitals on the US West Coast, yet came first in only 25 of them. A brand can be named almost every time and still hold no fixed place in the list.
How many candidates an engine can choose from sets the spread. The same study found that tight spaces, such as cloud computing providers for SaaS startups, produced similar lists across runs, while wide ones, such as recently published science fiction novels, varied far more. If your category has dozens of credible vendors, expect your presence to move more between runs than in a category of five.
The other engines vary too. In the SparkToro study, Claude was only slightly more likely than ChatGPT to give the same list twice, and Google’s AI Overview, or AI Mode where no Overview showed, was about as unlikely as ChatGPT. Perplexity and Gemini were not part of it; treat a single reply from them the same way. How each engine finds its sources differs, as our guides to Perplexity SEO and AI Overview tracking explain.
How to test it yourself: a five-run repeat test
A repeat test shows you, in one sitting, how much your own category moves. It is a test, not a measurement: it tells you whether variation is large for your questions, not what your visibility is.
- Write three to five buyer questions without your brand name, the way a buyer asks before choosing, for example “Which project management tool suits a 20-person agency?”, because a question with your name in it shows what ChatGPT says about you, not whether it names you unprompted. Our guide to prompt tracking explains how to pick them.
- Open a Temporary Chat and choose Unpersonalized, or use a logged-out browser window. Note the date, your plan and the model.
- Paste the identical wording and ask each question five times, each time in a new temporary chat, so earlier runs cannot shape later ones.
- Log every run in a sheet: question, run, every brand named in order, whether yours is among them, and every source linked.
- Count per question: in how many of five runs you appear, which brands appear in all five, and which appear only once.
- Optional: ask the same questions five times in your normal account with Memory on, and log them the same way.
Read it like this. Brands named in all five runs are the engine’s core set for that question; brands named once sit at its edge. If you appear in three of five runs, that is not 60 percent visibility: with five answers, the 95 percent interval for three hits runs from about 23 to 88 percent. If your brand appears clearly more often in your own account than in the clean runs, your history is shaping your own check, and the clean runs are closer to what a new buyer sees.
Run the same sheet again on another day, and on a second engine if your buyers use one, before you draw any conclusion about a trend.
How many answers before a difference is real?
One answer is a snapshot. The pattern is the truth. A difference between two readings counts only when it is larger than the spread you would see between two readings of the same thing, and that spread depends on how many independent answers each reading holds.
Repeats of one question are not independent. Answers to the same question resemble each other, because the engine draws on the same core set each time, so the fifth run of a question adds less information than the first run of a new one. Statisticians correct for this with a design effect, which shrinks a sample to its effective size.
Example, with invented numbers: your brand is named in 20 of 50 answers, 40 percent. If those are 50 different questions asked once, the 95 percent interval runs from about 28 to 54 percent. If they are 10 questions asked five times, and the runs of each question agree so often that five of them count like 1.7 independent answers, the 50 answers are worth about 17, and the interval widens to about 21 to 63 percent. More questions beat more repeats.
A comparison also needs the same frame on both sides: the same questions in the same wording, the same engine, the same country and periods of equal length. A changed question set looks like a change in visibility, so when you add questions, keep reporting the old set on its own.
Then judge the difference against the intervals. When the two intervals do not overlap, the change is real; when they overlap, run a two-proportion test or collect more answers before you report a move. With about 50 independent answers per period, a rate near 40 percent carries an interval of roughly 13 points either way, so a 5-point move between two periods cannot be told apart from noise.
Rough guide, for a rate near 40 percent and two periods of equal size: how many independent answers each period needs before a move counts.
| Independent answers per period | Smallest move that counts |
|---|---|
| 50 | about 19 points |
| 100 | about 14 points |
| 200 | about 10 points |
| 400 | about 7 points |
Repeats of one question count as fewer independent answers than their number, so reach these sizes with more questions first.
In FameScan, the interval on visibility, recommendation rate and citation rate already corrects for answers to the same question resembling each other, with a verdict on whether a change is real, as our guide to AI citation tracking explains.
What do you report instead of a screenshot?
Report a rate with its frame, and use a screenshot only to illustrate it. A line your boss or client can check names the engine, the country, the question set, the number of answers, the rate with its interval, and the verdict on the change.
Example, with invented numbers: “ChatGPT, United States, 40 buyer questions, 120 answers from November 1 to 14: named in 34 percent (95 percent interval 24 to 46). Previous two weeks: 29 percent (20 to 41). Change within noise.”
Three rules go with it. Do not report a position as a rank, because position moves more than presence. Do not mix a mention, a citation and a recommendation in one number, since each is a different win. Attach one stored answer per finding, so a reader sees what “named” looked like, and say that it is one draw.
The full measurement plan, from choosing questions to a monthly review, is in our guide to measuring AI visibility, and why ChatGPT leaves a brand out in the first place is in our guide to ChatGPT visibility. If you want the rates rolled into one number, the FameScan Visibility Score combines seven categories, among them Consistency: how evenly your presence holds across engines and countries.
Your next step: run the five-run test above on three buyer questions. If your brand appears in some runs and not in others, one screenshot cannot stand for you, and those questions are the start of a set worth tracking.
Frequently asked
Does ChatGPT give the same answer twice?
Rarely word for word. Each reply is a new draw, so pressing Try again or asking in a new chat brings new wording, often a new order, and in recommendation questions sometimes different brands. Short factual answers tend to keep their substance while the sentences change. If you need the exact text again, copy it or keep the chat; asking again will not reproduce it.
Is ChatGPT deterministic at temperature 0?
Not reliably. Temperature is a setting of OpenAI's developer API, where lower values make output more focused and deterministic, in OpenAI's words. Thinking Machines Lab sent one prompt 1,000 times at temperature 0 to an open model and got 80 different completions (September 2025). The cause was the server grouping requests differently as load changed, something no user controls.
Do Perplexity, Gemini, Claude and Google AI Overviews vary too?
Yes for the two of them that were measured, Claude and Google’s AI. In SparkToro's repeat study of January 2026, Claude was only slightly more likely than ChatGPT to repeat a list of brands, and Google's AI was about as unlikely as ChatGPT. Engines that search the web add a moving part: the pages they retrieve on that day. Measure each engine on its own and compare rates, never one engine's single answer with another's.
Does a Temporary Chat show what a new user sees?
It comes closer, but it is not the same. A Temporary Chat set to Unpersonalized uses no memory, custom instructions or plugins, according to OpenAI's help page. It still runs on your plan and the model you pick, an approximate location from your IP address, and that day's search results. A new user on another plan in another city can still get a different list.
Can you make ChatGPT give consistent answers?
Only partly. Fixed wording, a fixed answer format such as "list five tools, one line each" and an Unpersonalized Temporary Chat remove some variation. The draw itself stays: OpenAI's own API reference says that even with the same seed and parameters, determinism is not guaranteed. For measuring, consistency comes from many runs, because a brand's rate across them is steadier than any single list.
How can brands measure ChatGPT visibility if answers change?
As a rate, not a rank: the share of answers to a fixed set of buyer questions that name the brand, per engine, country and period, with an interval around it. Pick questions buyers ask without your brand name, keep their wording fixed, and compare periods of equal length. A brand's position in a list is the least stable number, so never report it as a ranking.
Sources & further reading
Sources checked on 20 November 2026.