Why Smaller Brands Go Missing in AI Answers: A 37,000-Run Audit

Why this matters

A brand can only be recommended by an AI assistant if the assistant first finds it, then mentions it, then chooses it. A large 2026 audit measured where brands of different sizes drop out of that sequence, and the answer depends heavily on how well known the brand already is. It also put a number on how much AI answers change from one run to the next, which matters for anyone trying to measure their own visibility.

The study

The paper is "Prominence-Stratified Failure Modes in Retrieval-Augmented Commercial Recommendation: A 37,000-Run Audit" by Will Jack, Noah Lehman, Keller Maloney and Sarah Xu, posted to arXiv on 22 May 2026. The team ran about 37,000 queries through production versions of GPT-5.4-mini and Claude Sonnet 4.6, each at two settings, with further checks on GPT-5.4 and Claude Opus 4.6 and 4.7. They used 215 commercial prompts across 19 business sectors and tracked 533 brands sorted into five tiers of prominence.

The tiers ran from category leaders (examples include Salesforce, HubSpot and Datadog) through challengers (Pipedrive, Trello, Gusto), mid-market brands (Copper, Sentry, Grafana Labs), long-tail specialists (Logz.io, Wrike) and regional players (Pennylane, BrightPay). Results were gathered through three different web retrieval systems: the models' own built-in web search, the Exa neural search engine and Brave keyword search. Two separate AI judges had to agree before a brand counted as recommended, and the authors report 95% confidence intervals.

Readers should know that the authors work for a commercial company that sells AI search optimization services. Their method is unusually cautious. They report thin samples as thin and separate what they measured from what they think it means, but the commercial interest is worth keeping in mind.

Four places a brand can drop out

The authors sort every brand in every run into one of four outcomes. The brand may never appear anywhere, in the search results or the answer. It may appear in the search results and still be left out of the answer. It may be mentioned in the answer without being recommended, or it may be recommended.

The authors label these as failures of discoverability, compellingness and positioning, then success. They are open that these labels are interpretations. They rate the first label as well supported, the second as weaker and the third as weaker still, and they list other possible explanations for each. We follow the same caution here.

Many smaller brands never appear at all

Leaders and challengers appeared at least once in 100% of cases across the full set of runs, and mid-market brands in 88%. Long-tail specialists appeared in only 52% and regional brands in 48%. Put the other way, 48 to 52% of specialist and regional brands never surfaced once in 37,000 runs.

Per individual query the gap is wider. On one Claude configuration with built-in search, leaders surfaced on 77% of relevant queries, challengers on 60%, mid-market brands on 23%, specialists on 8% and regional brands on 3%. The authors note that this runs against a common industry hope that live web retrieval would help smaller brands catch up, and that it may even concentrate attention on a few authoritative sources.

Appearing is only the first hurdle

Being found did not guarantee being chosen, even for the biggest names. Once a category leader surfaced, it was recommended only 25.3 to 41.2% of the time, depending on the model setting. Challengers had the highest conversion of any tier in every setting, reaching 51.6% on one Claude configuration, so a challenger that gets found tends to win more often than the leader does.

At the other end, regional brands that surfaced on one Claude setting were recommended only 12.8% of the time, with a confidence interval of 9.5 to 17.1%. That is roughly once in eight appearances. Smaller brands therefore face two problems, getting found and then getting picked.

The same question gives different answers

This may be the most useful number in the paper for anyone measuring AI visibility. When the same prompt was rerun 30 times on the same model and the same day, the overlap between recommendation sets was 0.50 to 0.61 on a standard set-similarity scale, where 1 means identical. Between different providers the overlap was about 0.35.

In multi-turn conversations, stability dropped to 0.263, plus or minus 0.219, and fell with each turn: 0.387 at the first turn, 0.187 at the second, 0.084 at the third and 0.093 at the fourth. Roughly half of a recommendation list can change between identical runs. A single check of whether a brand appears is therefore close to meaningless, and any claimed change in visibility needs to be larger than this natural churn.

Other findings

The search system matters most for weaker brands. The overlap between brands found by built-in search and by Exa fell steadily with prominence, from 0.83 for leaders to 0.50 for regional brands. Among surfaced specialist brands, 60% appeared only through Exa or Brave and never through built-in search, compared with 14% of leaders. Measuring only through one search system under-counts the brands that most need help.

Buyer persona matters most in the middle. When the researchers changed the described buyer, 67 to 75% of recommendations changed for mid-market brands. The figure was 23 to 29% for leaders and 27 to 33% for specialist and regional brands combined. For a mid-market brand, the question "are we recommended" depends heavily on who is asking.

Follow-up questions do not widen the field. In 750 four-turn buyer conversations, the rate at which each tier surfaced stayed within 0.003 of single-question results. Later turns explored brands already named in the first answer. This test used one model setting and one simulated buyer, so the authors do not generalize from it.

Newer or bigger models change little for invisible brands. Moving from GPT-5.4-mini to the full model raised the number of distinct brands per answer from 7.7 to 10.6, but Claude showed no comparable effect. Across every setting and model generation measured, specialist and regional brands that never surfaced stayed invisible 96 to 99% of the time.

A famous figure hides wide variation. The 2024 Generative Engine Optimization paper by Aggarwal and colleagues reported visibility gains of up to 40%. The audit's authors read that figure as an average over very different situations, with citing external sources helping most where brands are hard to find and adding statistics or quotes helping most where brands are already found. The 40% figure should not be treated as a typical expected lift.

A note on a claim that is not in this paper

Some industry articles have said this audit shows that prominent brands are protected from the effect of negative coverage. The paper does not report that. It does not measure how negative sentiment affects recommendation, and it mentions sentiment only once, describing its sentiment data as "informative but not definitive." Whether negative coverage hurts AI recommendations remains an open question that this study did not test.

What this means for brands

The right priority depends on where a brand stands today. For specialist and regional brands, the first job is getting found at all, because work on positioning or reputation cannot pay off in answers where the brand never appears. Being included in independent lists, comparisons and reviews, and publishing clear reference content that a range of search systems can reach, are the levers the authors associate with this stage.

Category leaders are already found, so their opportunity lies in conversion, through direct comparisons and clear differentiation from named peers. Mid-market brands face all of these problems at once, along with the strongest persona effects. Whatever the tier, measurement needs repeated runs, several search systems and several buyer types before any result is trusted.

Sources

• Jack, Lehman, Maloney and Xu, "Prominence-Stratified Failure Modes in Retrieval-Augmented Commercial Recommendation: A 37,000-Run Audit," arXiv, 22 May 2026
• Aggarwal et al., "GEO: Generative Engine Optimization," KDD 2024, arXiv