When Your AI Visibility Tool Is Tracking the Wrong Brand

An audit of one AI-visibility deployment found a startup's reported "30 mentions" were 28 references to a fictional character who happened to share the brand's name. The dashboard had been tracking the wrong entity entirely, and nothing in the interface hinted at it. Every metric built on top (presence, Share of Voice, sentiment, trend) was fiction with a chart around it.

This is the least-discussed and most consequential failure mode in the category, and it has a structural cause. Large language models resolve ambiguous names toward whichever entity has the strongest signal in their training data. If your brand is a common word, a person's name, a place, a product feature, or simply less famous than something sharing its name, the model's default reading is somebody else.

Why detection is harder than it looks

Substring matching is a catastrophe. A brand called "Craft" matches inside "aircraft", "handcrafted" and "craftsmanship". A brand called "Notion" matches every abstract use of the word. Without enforced word boundaries you're measuring letter sequences and calling them mentions. It's the most common implementation error in home-rolled tracking scripts.

Models paraphrase and periphrase. "The company behind that scheduling app" is a mention with no name in it. Pure string matching misses it, which is the false-negative side of the same problem.

Aliases multiply. Legal name, trading name, domain, product sub-brands, common misspellings, spacing variants. In our own demo data the judge listed both Loo.koo.mas and Lookoomas as separate brands present in the same answer: two names, one shop, and a naive counter would double-count.

And the failure is silent. A tool tracking the wrong entity produces plausible, populated, professional-looking reports. There's no error state. The only way to catch it is to read the answers.

The layered pipeline, cheapest first

Detection runs in tiers, each one cheaper and more precise than the next:

1. Normalise. Lowercase, Unicode NFKC, strip diacritics, collapse whitespace, handle spacing and punctuation variants so HubSpot, Hub Spot and hubspot all converge. Crucially: enforce token boundaries.
2. Exact alias gazetteer. Canonical name, legal name, product sub-brands, domain forms, known misspellings. High precision, cheap.
3. Fuzzy matching on misses. Token-set ratio for multi-word names at a threshold around 80–85; stricter ratio matching at 90+ for short names, because short names flood you with false positives at any looser setting. Names that are also common words skip this tier entirely and go straight to a judge.
4. LLM-as-judge. The disambiguator, given the brand's canonical name, aliases, competitors and an explicit ambiguity hint. This is the tier that catches paraphrase and rejects same-name confusion.

That's more rigour than most implementations have, and on its own it still falls short, because the judge has its own error rate, and a judge that says "yes, this is the brand" about a fictional character is exactly the failure we started with.

The gate: a name is not a mention without an anchor

So confirmation requires corroboration. A mention counts as confirmed only when an anchor is present:

• the brand's own domain is cited in the answer, or
• a known competitor co-occurs in the answer, or
• a product or category term from the brand profile appears in the same passage.

The first two are hard anchors: either one confirms with a single detection signal. The third is a soft anchor, and it only confirms when the alias match and the judge agree, because a topic word plus one weak signal is precisely the shape a same-named entity produces in an on-topic answer.

Anything that fails the gate gets stored, labelled low confidence, shown in the drilldown, and kept out of the hard metrics. Nothing disappears quietly. Every outcome carries a confidence value:

Evidence	Confidence	Counted?
Own domain cited in the answer	1.00	Yes
Competitor co-occurs, or category anchor with both signals agreeing	0.85	Yes
Name found and judge agrees, but no anchor	0.50	No, low confidence
Name found but judge disagrees, or judge alone	0.40	No, low confidence
No signal	0.00	No

What the gate cost us on our own demo brand

This is the part a vendor's blog post normally leaves out.

Fifteen grounded answers about Loo.koo.mas, a loukoumades shop in Reykjavík. Ten of the fifteen answers contained a brand signal. Four were confirmed.

Answers	Confidence	What the evidence looked like
2	1.00	Claude named the shop and cited lookoomas.com
2	0.85	GPT-5.5 and Gemini named the shop alongside Valdís, a known competitor
6	0.50	Named the shop, judge agreed, but no owned citation, no competitor, no anchor hit
5	0.00	Brand not named at all

Presence rate reported: 4/15 = 26.7%. Presence rate if every named answer counted: 10/15 = 66.7%. That 26.7% is the figure that flows into every other metric: the composite score of 31, the 19% Share of Voice, the trend line.

The gate cut the brand's headline presence by two thirds. On its own demo tenant. In a screenshot we ship.

Read that drilldown and the discomfort is immediate. Gemini's answer opens "The absolute best place to try loukoumades in Reykjavík is Loo.koo.mas": an unambiguous, first-position, enthusiastic mention. It's tagged Not mentioned · low confidence and kept out of the presence rate.

Why that specific answer fails the gate

Look at what the gate can see. Gemini's four citations were all google.com grounding redirects, so no owned domain. Its answer named no competitor. The judge's brands_present list contained exactly one entry: Loo.koo.mas, alone.

An answer that names one brand, cites nothing belonging to it, and mentions no rival is structurally identical to the fictional-character case. One string, agreed on by two detectors, with nothing in the surrounding text tying it to the business you're tracking. The gate can't tell "the only dessert shop worth naming" apart from "a character in a novel", because the string is the only evidence it has.

The pattern is sharpest on the branded prompt: "Is Loo.koo.mas in Reykjavík actually worth visiting?" All three models answered at length about the shop. All three answers scored 0.50. All three were excluded, which is why the branded intent stage reports 0% presence on a question that literally contains the brand name.

That looks absurd on a dashboard and it's defensible in a lawsuit. It has a useful side effect too: it structurally resists the "phantom inclusion" inflation you get from feeding your own name into a monitoring prompt and counting the guaranteed mention as visibility.

The trade-off, stated rather than hidden

Every detection system picks a point on this line, and the choice is editorial rather than technical:

	Loose gate	Strict gate
Reports	Higher presence	Lower presence
Fails by	Counting things that aren't you	Discarding things that are
Worst case	A quarter of reporting built on a fictional character	Understating a real brand's reach
Recoverable?	No, you can't tell which numbers were wrong	Yes, the excluded answers are stored and visible

Strict is the right default for anything you act on. A false positive tells you a problem is solved when it isn't, and it contaminates every metric downstream: presence, Share of Voice, sentiment, trend, and any recommendation built from them. A false negative understates you, which is annoying and correctable.

But "strict" without disclosure is just a different kind of dishonesty. So the low-confidence answers stay in the drilldown, labelled, with the reason inspectable. Six answers on the demo scan are marked in a way that says read this and judge for yourself.

There is a second correction underneath: because the judge has imperfect sensitivity and specificity, the raw rate is de-biased with the Rogan–Gladen estimator, π̂ = (p̂ + Sp − 1) / (Se + Sp − 1). On this scan that moves 26.7% to 25.7%. Both figures are stored. The methodology page sets out the estimator and the calibration procedure, including the labelled gold set the sensitivity and specificity figures have to be re-estimated against whenever the judge model changes.

How to audit any tool for this in fifteen minutes

• Ask for ten raw answers behind a mention count. Read them. If the tool can't produce the answer text, stop evaluating it. There's nothing to check.
• Check whether it distinguishes confirmed from unconfirmed. A single mention count with no confidence tiers means the vendor made this trade-off for you and kept quiet about which way.
• Test the ambiguity case deliberately. Add a competitor with a common-word name and see whether its "mentions" are real. Fastest way there is to find a substring matcher.
• Compare a branded prompt against a non-branded one. If both report similar presence, the tool is probably counting your own prompt back to you. Writing a prompt set that avoids this is mostly a matter of keeping the branded questions in their own bucket.
• Ask what the denominator and the exclusions are. Any vendor claiming high precision without publishing how they validated it is quoting a number they haven't measured. Precision claims only mean something against a hand-labelled set, with chance-corrected agreement reported instead of raw agreement, which overstates it by tens of points.

The tell is always the same. Tools that will show you the receipts have thought about this. Tools that will only show you a chart haven't.

FAQ

How do I know if my AI visibility tool is counting the wrong entity? Open ten raw answers it counted as mentions and read them. If the tool can't show you the answer text behind a number, you can't check it at all, and a brand name that's also a common word, a person, a place or a product feature is the highest-risk case.

What is an anchor in mention detection? An anchor is corroborating evidence that a named string really is your brand: your own domain cited in the answer, a known competitor named alongside you, or a product or category term from your brand profile in the same passage. Without one, a name match is just a string match.

Does substring matching work for brand mention detection? No, and it's the single largest source of false positives. Without word-boundary enforcement, a brand called Craft matches inside aircraft and handcrafted. Normalisation, word boundaries, alias lists and conservative fuzzy thresholds all have to come before any judging happens.

Is it better to over-count or under-count mentions? Under-count, for anything you act on. A false positive tells you a problem is solved when it isn't and pollutes every downstream metric; a false negative understates you, which is recoverable. The compromise is to keep both figures: confirmed mentions for hard metrics, low-confidence ones visible and labelled.