AI Prompt Tracking: Build a Reliable Brand Query Set

Define the independent prompt panel before measuring answers

AI prompt tracking is the repeated observation of declared AI products and answer surfaces using a controlled, versioned question panel. The panel-design record is independent from the observations it later produces. It explains the population the team wants to represent, the selection method, known exclusions, and the decision each prompt supports.

This separation prevents circular design. A team should not add questions because they already produced a favorable AIsee mention, remove questions after an unfavorable answer, or combine branded fact checks with unbranded discovery without labeling the difference. The panel defines what will be observed; the answer dataset records what happened.

Keep the objective narrow. A panel may examine entity accuracy, category association, comparative fit, source selection, or purchase validation. If several objectives are included, assign each prompt to one declared family and report them separately. Formal rates and denominators belong in the measurement layer; this page defines the registry that makes those calculations auditable.

Publish inclusion and exclusion rules

Build candidates from customer interviews, sales and support questions, on-site search, traditional search data, product research, and subject-matter interviews. Preserve the originating evidence and remove private information. A generated suggestion can expand the candidate list, but it does not establish audience demand by itself.

Keep branded questions separate. “What does AIsee do?” can test whether AIsee is described accurately, but the brand in the prompt partially induces its appearance and should not be treated as independent discovery. Apply the same rule to competitor-named questions.

Copy this 12-column independent prompt-panel registry

Use one row per exact prompt. Keep the header stable so a reviewer can compare versions and join later answer records to the approved design.

The NIST AI RMF Measure playbook says test sets, metrics, and evaluation details should be documented because documentation enables repeatability and consistency. A stable prompt ID and version connect that principle to answer observations without implying that the panel represents every possible user question.

Build a sample family around one audience decision

A prompt family records meaningful variations without pretending they are interchangeable. Start with one audience decision, then create only the variants required by the design.

Assign a unique prompt ID to each member and state whether the family is balanced or intentionally weighted. Do not pool branded accuracy with unbranded discovery. Do not treat a translation as equivalent until a reviewer confirms that intent, market context, and product language are preserved.

Families make expansion governable: the team can add a validation or regional variant without rewriting the original prompt or silently changing the historical panel.

Require approval for every panel version change

Freeze the core panel for the comparison period. Any addition, retirement, wording edit, family reassignment, market change, or change in repetition rule creates a proposed version event. The change record contains the prompt ID, old value, new value, reason, originating evidence, expected comparison impact, effective period, proposer, reviewer, and approver.

Use status values such as draft, exploratory, approved_core, retired, and excluded. Never overwrite a retired row; preserve its last effective version so historical answers still resolve to the design that produced them.

A measurement owner checks sampling effects, a product or brand owner checks factual scope, and a market-language reviewer approves localized meaning where applicable. Emergency additions for a material error can run immediately in an event cohort, but they do not enter the frozen trend denominator until the next approved version. Publish a concise version note beside any comparison that crosses panel versions.

Join each approved prompt to complete run evidence

At execution, append observation fields rather than editing the design row: provider, exact product surface, date and time zone, market, language, login or personalization state when known, conversation state, run number, complete answer, displayed URLs, capture method, reviewer, and uncertainty note. Preserve failed and no-qualifying-answer runs under the declared rule.

Surface names matter. Google's official AI-features guidance explains that AI Overviews and AI Mode can use different models and techniques, while OpenAI's ChatGPT Search documentation describes web search with linked sources. Record the observed product surface instead of collapsing everything under a provider name.

Interpret the panel within its boundary. NIST AI 800-3 formally distinguishes accuracy on a fixed benchmark from generalized accuracy and highlights uncertainty in evaluation. A frozen prompt panel similarly supports comparisons inside the declared design; it does not prove performance for every possible prompt.

Send verified factual errors to answer accuracy monitoring and competitive gaps to competitor analysis. Keep the registry focused on selection and version evidence.

Use AIsee to select investigations, not to design results

AIsee analyzes a public website and produces a dated report across AI Presence, Competitor Landscape, Strategy Review, provider diagnostics, and a reviewable Action Plan. A provider difference can suggest a provider-specific family; a competitive diagnostic can suggest a comparison family; and a Strategy Review finding can suggest a factual or technical validation prompt.

The independent registry still decides whether that candidate represents a real audience decision and meets the inclusion rule. Record AIsee as the origin when it generated the investigation, then require the same evidence, review, and approval as any other candidate. Do not promote a prompt merely because its current answer favors AIsee or another brand.

Recurring execution, public-interface answer capture, labeling, and trend calculation remain outside the current AIsee report. Place the dated AIsee diagnostic beside—not inside—the prompt observation dataset. After an approved change is live and discoverable, rerun the same panel version and later compare a new AIsee report under its own date and product definitions. Neither observation alone proves causality or guarantees visibility.

Frequently asked questions