What AI Recommendations Do, How to Measure Them and How They Get Fooled

Why this matters

Marketers now need to know three things about AI answers: whether being recommended changes what buyers do, whether reported visibility numbers can be trusted and whether the answers can be manipulated. A cluster of studies published between April and June 2026 speaks to all three. This piece explains what each one did, what it found and how much weight it can bear.

Being recommended appears to change behavior

A preprint by researchers at an AI search analytics company, submitted in June 2026, joined an opt-in panel of web browsing data to the same users' conversations with ChatGPT, Claude and Gemini. It asked what happened in the seven days after an assistant named a brand to someone with no recent observed engagement with that brand. When the assistant recommended the brand, same-name Google searches rose by 4.3 percentage points, visits to the brand's own site by 2.4 points and visits to a retailer's product page for the brand by 1.0 point. An incidental, neutral mention moved the same measures by only 1.8, 1.1 and 0.3 points, so a recommendation had roughly two to three times the effect of a passing mention.

The design has real strengths. The authors compared each user with that user's own earlier behavior, checked for trends before the mention and compared named brands against unnamed brands from the same category in the same answer, which stayed flat. The effect showed no clear link to how famous the brand was or to whether it was named first, second or later.

The limits matter just as much. The study is observational, so it cannot fully rule out that users already leaning toward a brand were the ones who asked about it. The authors withheld panel size and every per-cell sample size as commercially sensitive, they sell measurement services in this field, and no purchases were observed. The fair reading is that the direction is now evidenced, while the specific magnitudes should not be treated as an expected result for any one brand.

Most confidence intervals for AI answers are too narrow

A paper by Solomon Messing, posted in April 2026, addresses a problem that affects almost everyone who reports AI visibility figures. When an analyst runs the same prompt many times at the same settings and computes a standard 95% confidence interval, the interval ignores other sources of variation: how the prompt was worded, which model judged the output and what sampling temperature was used. Those sources do not shrink as more runs are added, so they act like fixed offsets.

In the paper's analysis, a naive interval meant to capture the truth 95% of the time did so 42% of the time for a single item, and only 7% of the time as the sample grew. More data made the interval narrower around the wrong center, which is why coverage fell as more data arrived. Adding three prompt variants raised coverage to about 55%, adding three judges raised it to about 85%, and also varying temperature restored full coverage. The paper covers English and frontier models only, and the author notes that the true error is probably larger than reported. A visibility score presented with a tight interval from repeated runs of one prompt is likely more precise on paper than in reality.

One polluted page can push a fake brand into an answer

Researchers at the Chinese University of Hong Kong built a benchmark called FORGE and posted it in June 2026. They rewrote retrieved web pages for 225 real products in 15 categories so that a fake brand replaced a real one, then measured how often 12 production models, six closed and six open-weight, recommended the fake. Each product was tested 15 times per model on evidence frozen in April 2026, and the authors state they received no compensation from any model provider.

Position in the results mattered enormously. A single polluted page at rank one fooled the most vulnerable models in 27% of cases, while the same page at ranks two to ten fooled them only 1% to 4% of the time. Replacing the top three results raised the fooled rate to between 13.3% and 73.8%, depending on the model, and three polluted pages in the top ten pushed the most vulnerable models past 50%. Dining was the most exposed category at 81.7% and smartphones the least at 22.8%.

Two findings run against intuition. Turning on reasoning made two open-weight models more vulnerable, by 18 and 9 percentage points. Telling the model to be skeptical of unfamiliar brands backfired, raising the fooled rate by 24 points on average for closed models and by 44 points for Gemini 3.1 Pro. Filters that required agreement with prior knowledge or across documents caught 95% and 90% of fake brands, but they also suppressed between 52% and 79% of legitimate recommendations.

A second team, led by researchers at KAUST, built a framework called SearchGEO and tested 13 model backends on 308 cases each across health, finance, consumer technology and law. Attack success ranged from 0.0% for Claude Sonnet 4.6 to 31.4% for Gemini 3 Flash, with an average of 13.4%. The most effective attacks manufactured the look of independent agreement, reaching 72.7% success on Gemini 3 Flash. The test used cached copies of real search results, so it shows how the mechanism works under controlled conditions and leaves the scale on the live web unmeasured.

AI citations are sometimes invented

Delip Rao, Eric Wong and Chris Callison-Burch at the University of Pennsylvania measured how often models cite web addresses that do not work. Across two benchmarks with more than 220,000 cited URLs, 3% to 13% of citations were hallucinated, meaning no archived record showed they had ever existed, and 5% to 18% failed to resolve. Deep research agents hallucinated 10.7% of URLs, against 4.8% for search-augmented models. When capable models used the authors' open-source checking tool to correct themselves, broken links fell sharply, for example from 4.9% to 0.8% for Claude Sonnet 4.5.

A citation count overstates influence

Zhang Kai, He Xinyue and Yao Jingang studied 602 controlled prompts on ChatGPT, Google AI Overview and Gemini, and Perplexity, yielding 21,143 citations. They separated citation, being listed as a source, from absorption, the degree to which a cited page actually shaped the answer. Perplexity cited 16.35 sources per answer on average with a mean influence score of 0.0646, while ChatGPT cited 6.88 with a mean influence of 0.2713. Perplexity listed more than twice as many sources and drew far less from each.

Some kinds of content were absorbed much more than others. Pages built around code scored 76.88% higher on influence, statistics 61.55%, definitions 57.33%, comparisons 55.28% and how-to steps 41.20%, while question-and-answer formatting was the only genre that scored lower, at minus 5.74%. Encyclopedia sites had the highest mean influence at 0.2144 and news media the lowest at 0.0726. The authors make no causal claims, and their influence score is a proxy built from observable text overlap.

Engines disagree on picks and agree on why a brand is missing

A preprint by researchers at an AI search company, posted in May 2026, compared ChatGPT and Claude models across 215 commercially framed prompts. The two providers recommended different brands about two-thirds of the time. When a brand was left out, both providers usually failed it at the same stage, and agreement on that stage was 95.1% across 7,763 joint failures. The authors caution that this figure is driven by the many brands that never surface at all, and agreement fell to 14.9% where the failure was about being mentioned or recommended. The study also found that Anthropic models answered from prior knowledge far more often than OpenAI models.

What this means for brands

The strongest message is that being recommended carries more weight than being mentioned. There is now observational, vendor-run evidence that a recommendation moves later search and site visits two to three times as much as a passing mention, so stance deserves tracking alongside volume. Brands should also question any visibility report by asking how many prompt wordings were tested and how its intervals were computed.

The content findings point in a consistent direction. Pages built around clear definitions, numbers, comparisons and steps are absorbed more than generic question-and-answer pages, and citation counts alone overstate influence. The manipulation research shows that answers can be gamed, especially at rank one, and that simple defenses can backfire. Brands should monitor for fake competitors and impersonation in their categories, and they should never use these tactics themselves.

Sources

• Messing, Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking, arXiv (April 2026)
• Luo and Chen, One Polluted Page Is Enough: Evaluating Web Content Pollution in LLM Recommenders, Chinese University of Hong Kong, arXiv (June 2026)
• Chen et al., How Much Can We Trust LLM Search Agents? Measuring Endorsement Vulnerability to Web Content Manipulation, KAUST, arXiv (June 2026)
• Rao, Wong and Callison-Burch, Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents, University of Pennsylvania, arXiv (April 2026)
• Zhang, He and Yao, From Citation Selection to Citation Absorption, arXiv (April 2026)
• A preprint by researchers at an AI search analytics company on how AI brand recommendations move consumers, June 2026 (the authors sell related services; not linked)
• A preprint by researchers at an AI search company on cross-provider recommendation failures, May 2026 (the authors sell related services; not linked)