What is a prompt

A prompt is not a keyword. A keyword is matched against an index and returns documents; a prompt is processed as text and drags along everything accompanying it. That difference explains why the measurement logic of classic SEO does not transfer directly to generated answers.

What reaches the model is almost never only what the person writes. In front of it there is usually a system prompt setting role, tone and limits, which the user does not see. Behind it comes the conversation history, and in many products also fragments of documents retrieved at that moment or values returned by tools. All of that occupies the context window and enters the calculation with the same standing as the question.

The prompt should also be kept apart from the product. On an API the prompt is exactly what gets sent. In a consumer assistant, between the typed text and the model sit product layers that rewrite, summarise or expand the input. When someone compares results from the web interface with the same vendor's API, that difference usually explains much of the discrepancy.

Every token of the prompt enters the calculation of the probability distribution for the next token. There is no prior step that extracts the intent and discards the form: the form is part of the input. That is why changes a person would call irrelevant shift the result.

The magnitude has been measured. A study of meaning-preserving variations such as separators, capitalisation, spacing and the order of examples found differences of up to 76 accuracy points within the same task and the same model. Another piece of work, covering 6.5 million instances across 20 models and 39 tasks, showed that rephrasing the instruction does not only move the absolute scores: it also changes which model comes out ahead.

Order adds to this. What appears earlier conditions everything that follows, so a list of options inside the prompt can favour the first one purely by position. Then there is sampling: with temperature above zero the same prompt admits several continuations and the answer is drawn among them. In AI search there is one more layer, because the system does not execute the sentence as written but breaks it into several parallel queries by subtopic and composes from that set. The typed text is the starting point, not the query running underneath.

Anyone measuring a brand's visibility in AI answers is always measuring their own set of questions as well. The figure on the dashboard does not say “how often the brand is cited”, it says “how often the brand is cited when you ask like this”. Swapping twenty prompts for twenty equally reasonable ones moves the number without anything having changed in the market.

That is why two measurement vendors can deliver different percentages for the same company in the same week and both be right. Their sets of questions differ: one asks by product category, another by user problem, a third names the brand inside the question and thereby all but guarantees it appears in the answer. The discrepancy is not a failure of either one, it is a property of what is being measured.

The decision that follows is concrete. The prompt set is fixed, documented and versioned before the first result is looked at, exactly like a keyword set, and any comparison over time holds only within the same set. If questions are added mid-quarter, the series breaks and next month's rise means nothing. The same applies when hiring an external vendor: without sight of their set of questions, their percentage cannot be interpreted.

• Fix the prompt set in writing before the first measurement and store it with a version number and a date.
• Cover several ways of asking about the same thing, by category, by problem, by comparison and by recommendation, and count each group separately instead of averaging everything.
• Separate prompts that name the brand from those that do not: the former measure what is said about you, the latter whether you appear uninvited.
• Repeat each prompt several times per model and record the appearance frequency, because a single run does not represent the system's behaviour.
• Record the model, the date, the language and the country alongside every result, since all four alter the answer.
• Document the system prompt too when measuring through an API; without it another team cannot reproduce your numbers.

• Putting the brand inside the prompt and presenting the resulting mention as visibility won.
• Extending or tweaking the prompt set mid-period and continuing to compare the series as if it were continuous.
• Comparing figures from two tools without first checking whether their sets of questions are comparable.
• Translating prompts word for word into other languages and assuming they measure the same thing, when the usual way of asking differs per market.
• Writing the site's copy for one specific prompt, which may fall out of use as soon as the interface or the model changes.