How we measure it

Grounded probing

We don't ask a model what it remembers — we ask it what it finds. Every probe runs with live web search turned on, so the answer reflects the current web rather than stale training data. This matters because an ungrounded model recites priors that can be months out of date; a grounded model retrieves and reasons over live sources, which is what a real buyer using an AI assistant sees.

We probe a config-driven roster of grounded AI models across the major families. We store the full answer text, its citation annotations, and usage for every single run, so nothing about a result is unverifiable after the fact.

Honest scope. v1 measures grounded AI models reachable via live-search APIs. Consumer SERP surfaces — Google AI Overviews, Google AI Mode, Microsoft Copilot, and the ChatGPT web app specifically — have no API and are a post-v1 roadmap item. We do not claim to measure them, and we won't pretend a model result is one of those surfaces.

Repeated sampling & confidence intervals

AI answers are non-deterministic — even at temperature zero, floating-point non-associativity, mixture-of-experts routing, and non-fixed server seeds mean the same question can produce very different answers, with best-versus-worst spreads as wide as 70 percentage points. On top of that, citations drift 40–60% month over month. A single observation is noise dressed up as data.

So we sample. A scan runs each prompt against each model R times, and we report a rate with a Wilson 95% confidence interval rather than a bare percentage. We use Wilson intervals specifically because they have far better coverage than the textbook normal approximation when a rate sits near 0% or 100% — which is exactly where most brands live.

Precision comes from pooling breadth, not from hammering one prompt. The combined margin of error scales roughly as 0.98 / √(prompts × models × runs), so ~30 prompts × 4 models × 3 runs (360 observations) lands near ±5 points at 95% confidence — whereas a single prompt would need ~97 runs to reach ±10. Breadth plus topic-level pooling beats brute-forcing one question. When we compare two scans over time, a change only counts as "significant" if it exceeds the confidence band — which kills vanity-alarm noise.

Correcting for judge error (Rogan–Gladen)

We detect brand mentions with a layered pipeline whose most reliable tier is an LLM-as-judge. But any judge has imperfect sensitivity and specificity, and that imperfection biases the raw mention rate. Almost no competitor corrects for this. We do, using the Rogan–Gladen estimator:

π̂ = (p̂ + Specificity − 1) / (Sensitivity + Specificity − 1)

Here p̂ is the observed rate and π̂ is the de-biased estimate (the correction requires sensitivity + specificity > 1). We estimate the judge's sensitivity and specificity by validating it against a hand-labeled gold set of 100–200 examples, reporting Cohen's κ so agreement is chance-corrected rather than inflated. When the judge model changes, we re-validate — and every stored score carries a methodology version so historical trends stay comparable.

The true-mention gate

The scariest failure mode in this category is real: one audit found a startup's "30 mentions" were actually 28 references to a fictional character of the same name — the tool was tracking the wrong entity entirely. LLMs default to the entity with the strongest training signal, so ambiguous and smaller brands are prone to false positives.

To guard against this, we count a mention as confirmed only when an anchor is present:

• the brand's own domain is cited in the answer, or
• a product or category anchor term from the brand profile appears in the same sentence or paragraph, or
• a known competitor co-occurs in the answer.

Detection runs cheapest-first — normalization with enforced word boundaries (so "Craft" never matches inside "aircraft"), an exact alias gazetteer, fuzzy matching with conservative thresholds, then the LLM judge as the disambiguator. Anything the judge flags without a supporting anchor is marked low-confidence and excluded from the hard metrics. That dual method is what lets us publish precision honestly.

Share of Voice — with an open denominator

Share of Voice is your share of mentions across the answer set. The important choice is the denominator. We use an open denominator: every brand the models actually name, discovered from the answers themselves, with competitor labels applied after discovery.

SoV(brand) = mentions(brand) / Σ mentions(every brand discovered)

A closed, hand-picked competitor list is gameable and blind to emerging rivals; an open denominator surfaces challengers you didn't think to name. Vendors routinely hide how this number is computed — we disclose the denominator plainly. A sharper position-weighted variant (which discounts lower list positions) is available, but only once you've sampled enough repetitions for ordering to be trustworthy, since rank is close to random from one run to the next.

Mention ≠ citation

Being named in an answer and being cited as a source are different things, and we never conflate them. Roughly 62% of AI citations don't co-occur with a brand mention — so-called ghost citations. We report presence and Share of Voice (mentions) and citation share (being a linked source) as distinct metrics.

For citations, we normalize every cited URL to its registrable domain using the Public Suffix List — never a naive dot-split, which breaks on domains like example.co.uk — then bucket each source into owned, competitor, or third-party, and surface the "power sources" that your category's answers cite disproportionately.

The composite AI Visibility Score

The headline 0–100 score is a weighted blend of six components. The weights live in a single published config constant. Presence carries the most weight because it's the necessary condition and the least noisy to measure; sentiment carries little because its classifier is the noisiest; and stability is included so a brand whose presence swings run to run cannot score like one that appears reliably — that bakes our rigor wedge straight into the headline number.

We compute the score per engine and then blend, and we always publish the expandable component breakdown — a single opaque number would hide the fact that different engines cite almost entirely different sources. Every score is stamped with its methodology version, so when we tune the weights against a labeled set, your trend lines stay comparable over time. These weights are a defensible, documented default — not received truth — and they're versioned precisely so you can see when they change.