Behavioural Outcomes, the Confidence-Interval Problem and the Pollution Threat Model

Type: Scheduled weekly sweep. Ten primaries read in full. Clears four open-queue items (#2 partially, #3 fully, #4 fully, #5 fully) and moves two candidate principles.

Numbering note. Two files in this folder are both named digest-04 (2026-08-17-…-causal-limits-… and 2026-08-19-…-searchable-claude-visibility). The collision is recorded, not repaired — renaming a canonical file would break the app copy that must stay byte-identical. This digest is 05. Future digests continue from 06.

Headline, in three parts.

R31 — "never claim visibility produces clicks or conversions" — is QUALIFIED, not overturned. The first panel study joining assistant conversations to clickstream reports that a recommended brand gains +4.3pp same-name search, +2.4pp own-site visits and +1.0pp retailer-page visits over seven days, two to three times the effect of an incidental mention. It is observational, it is vendor-authored, and it withholds every sample size. Tier B, direction only. The direction is now evidenced; the magnitudes are not ours to quote.
A measurement-error paper invalidates how almost everyone in this market reports uncertainty, including R32 as written. Naive confidence intervals computed from repeated runs at fixed settings achieve 42% coverage at one item and 7% as sample size grows — under-coverage that worsens with more data. Adding runs to a single-prompt design buys precision you do not have.
The adversarial cluster is read and it is worse than R7 assumed. One polluted page at rank 1 fools vulnerable models in 27% of cells; three polluted pages cross 50%; reasoning-enabled models are more vulnerable, not less; and the obvious defence — telling the model to be skeptical of unfamiliar brands — backfires by +24pp on average and +44pp on Gemini 3.1 Pro.

1. Provenance

Battery run 2026-08-24, domain-restricted to the primary allow-list, current year appended.

Axis	Query run	Yield
1	`LLM product recommendation bias empirical audit brand ranking 2026`	2603.18300, 2604.15937 (new, unread); re-surfaced 2606.17443, 3795766.3799780
2	`brand prominence incumbency bias generative recommendation challenger 2026`	2606.26116, 2606.23057, 2606.10907 (all new, all read)
3	`retrieval augmented generation source selection citation attribution evaluation 2026`	2604.25707, 2604.03173 (new, both read); 2605.01284, 3767695.3769505 (unread)
4	`citation hallucination misattribution rate deployed language models 2026`	2604.03173; 2602.06718 GhostCite, 2605.07723, 2604.18880, 2603.07287 (new, unread)
5	`position bias context ordering retrieved documents relevance authority recency 2026`	2601.08363 PosIR (new, unread); primacy-bias replication across embedding models
6	`generative engine optimization visibility measurement empirical 2026`	No new primaries. Re-surfaced 2607.14035, 2606.20065, 2603.12282, 2511.20867
7	`content manipulation ranking poisoning generative search user-generated 2026`	2606.13610, 2606.16821, 2605.23684 (all new, all read); 2605.24245, 2606.12439 (unread)
8	`variance repeated sampling stability LLM answer measurement confidence interval 2026`	2604.11581 (read); 2601.20251 (new, unread)
9	`negative sentiment effect on model recommendation suppression commercial 2026`	NULL RESULT. Every hit was classical sentiment-aware recsys (CoNRec, DeepSentRec, KG-GLARNS) or social-media consumption economics. Nothing on answer-engine recommendation suppression. Third consecutive null on this axis.
CP-01 targeted	`source publication date age freshness effect citation probability generative search engine controlled 2026`	NULL for the principle. Best candidate (2604.25707) explicitly declines to analyse publication date.
CP-02 targeted	`corrective content displacing outdated negative claim LLM answer knowledge update conflicting evidence recency 2026`	2605.05957 (read) — adjacent mechanism, not a resolution
Practitioner (separate, quarantined)	`AI search visibility brand citations what works practitioner study 2026`	Recorded §7, tier C

Extraction status, stated per source as required.

Source	Route	Status
arXiv 2604.11581 Hidden Measurement Error in LLM Pipelines	`arxiv.org/html/…v1`	Full text, all coverage tables and D-study numbers. Tier A.
arXiv 2606.10907 From Prompt to Purchase (Scrunch AI)	`arxiv.org/html/…v1`	Full text. Tier B, hard cap — sample sizes explicitly withheld; see §3.1.
arXiv 2606.26116 Divergent Recommendations, Convergent Diagnoses (Unusual.ai)	`arxiv.org/html/…v1`	Full text, all CIs. Tier A, vendor conflict recorded, precedent-consistent.
arXiv 2606.13610 One Polluted Page Is Enough (FORGE, CUHK)	`arxiv.org/html/…v1`	Full text, all attack/defence tables. Tier A.
arXiv 2606.16821 Endorsement Vulnerability (SearchGEO, KAUST et al.)	`arxiv.org/html/…`	Full text. Tier A, self-judging caveat recorded.
arXiv 2604.03173 Detecting and Correcting Reference Hallucinations (UPenn/DARPA)	`arxiv.org/html/…v1`	Full text, per-model rates. Tier A.
arXiv 2604.25707 Citation Selection to Citation Absorption	`arxiv.org/pdf` FAILED → `arxiv.org/html/…v2`	PDF returned metadata only, no effect sizes — the documented failure mode, recovered by the HTML route. Full text on retry. Tier A descriptive only; authors reserve causal claims.
arXiv 2605.05957 Knowing but Not Correcting	`arxiv.org/html/…v2`	Full text, per-model suppression rates. Tier A in its own setting; tier D if transferred — see §4 CP-02.
arXiv 2606.23057 Who Owns the AI Recommendation? (Rankfor.AI)	`arxiv.org/html/…v1`	Full text. Tier B — vendor conflict, n=50 brands, single 2-day window.
arXiv 2605.23684 Synthetic Sources? (Allaham & Diakopoulos)	`arxiv.org/html/…v1`	Full text. Tier A for the measurement; data window is 2023–2025, pre-dating current engines.
arXiv 2602.06718 GhostCite	Not attempted	Tier C — search-summary only. 13 LLMs, 40 domains, 375K citations, 14–95% hallucination. Queued §8.

Nothing in this digest rests on the practitioner sweep.

2. The measurement-error finding, first, because it governs everything else

arXiv 2604.11581, Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking. Tier A.

The central result, in the authors' own framing: "standard confidence intervals ignore this variance, producing under-coverage that worsens with more data."

Specification	Nominal-95% coverage achieved
Naive SE, single item (residual variance only)	42%
Naive SE, as sample size increases	7%
+ 3 prompt variants	~55%
+ 3 judges	~85%
+ 3 temperatures (full model)	full coverage

The mechanism is simple and damning: prompt sensitivity, judge disagreement and temperature effects do not shrink as you add items or runs. They are fixed offsets. Averaging more draws from the same prompt narrows the interval around the wrong centre, which is why coverage falls from 42% to 7% rather than improving.

Variance-component shares, by domain:

Safety classification (AILuminate, 141 items): judge disagreement 43.9% of total variance; item×judge dominates; prompt 4%; temperature <1%. Fixing a single judge increases variance by 55%.
MMLU (200 items): items 43.4%; prompt diversity 0.5%. TEE-guided allocation halves RMSE versus standard single-prompt evaluation at ≥98% coverage.
Ideology annotation (150 items, Likert): within-category item variance 72.4%; item×judge 12.8%. 73% of single-configuration pipelines underperform the optimised average.
Propaganda audit (50 items, human-validated): item-level pipeline variance predicts misclassification at r = −0.68.

Scoring-method rule (54,000 calls per method): under Likert, spend on more items — 39.4% variance reduction. Under pairwise, spend on more judges — judge accounts for 20.7% of variance versus 1.7% under Likert. The right allocation inverts with the scoring method.

Minimum viable pilot, stated by the authors: N ≥ 30 items, V ≥ 3 prompt variants, R ≥ 5 replications — about 4,050 calls, ~$16. It recovers the qualitative design guidance despite 25.5% quantitative underestimation on Likert tasks, and projects within 5.8% of full-run estimates for pairwise scoring.

Gaming result, directly relevant to a market where vendors publish their own scores: with 10 model submissions, expected score inflation is 0.26 units on a 1–5 scale under single-judge, single-prompt configuration; judge main effects are 73% of the exploitable noise; M=3 judges plus V=5 prompts cuts the gaming advantage by 46%.

Limitations the authors state: linear-probability variance shares are scale-dependent and degrade at extreme base rates (they name a 94%-safe base rate); mixed-tier judges conflate disagreement with capability, and tier-matched frontier-only judges push the interaction to 60.6%; cross-layer system-under-test × judge interactions are not modelled; English only, frontier-tier only; and LLM-generated paraphrases likely underestimate true prompt sensitivity, so the reported sensitivity is a conservative lower bound.

Why this lands on us. R32 sizes samples from Schulte's bootstrap of the per-brand detection rate across repeated runs at fixed settings. That is precisely the naive estimator whose coverage collapses. Schulte's n = 7–8 is not wrong as a statement about run-to-run resampling noise, but a ±0.158 interval built that way is not a 95% interval in any useful sense once prompt phrasing is free to vary — and in LBOX's actual product, prompt phrasing varies constantly, because clients supply the questions. R32 is qualified, not killed: the run counts stand, the intervals do not.

3. Tier A and B findings

3.1 The behavioural-outcome number — arXiv 2606.10907 (Scrunch AI). Tier B, hard cap.

Iannelli & Ai, Scrunch AI, New York. Submitted 9 June 2026. Panel joining opt-in clickstream with users' ChatGPT, Claude and Gemini conversations; search-embedded surfaces such as Google AI Overview excluded. Seven-day post-response window, backward placebos at 14, 21 and 28 days.

Design layers: within-user matched backward-placebo event study; pre-trend examination; non-customer conditioning (no recent observed engagement); stance classification (recommend / neutral / caution); and a same-response control comparing named against unnamed brands in the same answer.

Recommendation × non-customer, seven-day lift:

Outcome	Lift	95% CI
Recall — same-name Google search	+4.3 pp	[3.1, 5.5]
Discovery — brand own-site visit	+2.4 pp	[1.4, 3.5]
Retail — retailer product-page visit	+1.0 pp	[0.3, 1.7]

Incidental neutral mention, same outcomes: +1.8 / +1.1 / +0.3 pp. Recommendations move behaviour "two to three times more" than name-drops. Pooled all-mention (confounded, reported as such): recall +2.08 [1.55, 2.54], discovery +2.37 [1.74, 2.89], retail +0.52 [0.30, 0.77].

Placebo behaviour is the part that earns the tier. Named brands showed +1.68 pp discovery activity in the three days before mention — pre-existing customer behaviour — and that pre-trend vanishes in the recommendation × non-customer cell. Same-category unnamed brands stay flat (discovery −0.07 pp, retail −0.01 pp).

Heterogeneity, all of it useful:

No prominence gradient. Spearman correlation between an encyclopedic-familiarity proxy and per-brand lift is +0.02 to +0.12, none significant.
No position gradient. Recall +2.1 / +2.0 / +2.5 pp for first / second / third-or-later mention — explicitly unlike the ranked-list position bias documented elsewhere.
The apparent engine gap is audience, not engine. ChatGPT recall +6.6 pp vs Gemini +1.5 pp between users collapses within users.
Recall replicates across beauty, apparel and audio/accessories; retail routing varies by how the category is sold.
The effect concentrates in non-customers; existing customers were already mid-episode.

Why it is capped at B despite good design. The authors state: "We do not report aggregate panel size, per-analysis user counts, or per-cell sample sizes; these are commercially sensitive." Our own tier-A definition requires sample sizes captured. It cannot be A. Add the vendor conflict — Scrunch AI sells AEO measurement and the pipeline is "deployed production infrastructure at Scrunch AI" — and the cap is not close. Their own limitations are candid: no randomised exposure, and a brand-specific within-session intent shock is "not closable without a randomized design"; web navigation only, so rates are "a lower bound on influence"; retail is a product-page visit, not an observed transaction; "non-customer" means no observed engagement; stance is assigned by a small model at 90% agreement, κ = 0.62, which attenuates the recommend/neutral contrast and makes the recommend dose conservative; intervals unadjusted for multiplicity; niche long-tail brands out of reach.

What LBOX may and may not do with this. May: state that recommendation, as distinct from mention, is now evidenced to move downstream search and site-visit behaviour, direction only, cited as a vendor observational study. May: use the recommend-versus-mention distinction as a product design principle. May not: quote +4.3 / +2.4 / +1.0 to a client as an expected result. May not: call any of it a conversion or revenue effect — the authors themselves say purchase is unobserved.

3.2 Diagnosis converges where recommendation diverges — arXiv 2606.26116 (Unusual.ai). Tier A.

Jack, Lehman, Maloney, Xu. 7,763 joint-failure cells; 533 reference brands labelled L1–L5; 215 commercially-framed prompts across four batches (200 / 430 / 100 / 100 prompt pairs); models gpt-5.4 and gpt-5.4-mini, claude-sonnet-4-6 and claude-opus-4-6, each at low and high reasoning effort; a cross-cell extension of 3,000 further runs on gpt-5.5 and opus-4.7. Brand extraction by two-judge consensus (claude-haiku-4-5 + gpt-5-mini).

Recommendation divergence: cross-provider per-prompt Jaccard 0.349–0.356 against a within-cell rerun-stability baseline of 0.50–0.61 — providers disagree on picks "roughly two-thirds of the time." Note this independently reproduces digest 03's 0.50–0.61 floor.

Diagnosis convergence, 3×3 confusion matrix over failure stages (S1 no retrieval/no mention; S2 retrieved but not mentioned; S3 mentioned but not recommended):

Prominence	Same-stage agreement	95% CI
L1 Leaders	81.2%	[78.2, 84.0]
L2 Challengers	88.2%	[85.6, 90.8]
L3 Mid-market	96.2%	[95.1, 97.2]
L4 Specialists	99.3%	[98.9, 99.7]
L5 Regional	99.6%	[99.3, 99.9]
Pooled	95.1%	[94.3, 95.7]

The caveat is load-bearing and the authors state it plainly. Cohen's κ is 0.29 against a 93.0% chance baseline driven by Stage-1 dominance — "fair", not "substantial". Off-diagonal mass concentrates at L1 on S2-versus-S3 disagreements, and within cells where at least one provider's modal failure is S2 or S3 (n = 450), diagonal agreement falls to 14.9%.

Read correctly: discoverability failure is diagnosed identically across providers; compellingness and positioning failure are not. The convergence is real and it is almost entirely a statement about brands that fail at Stage 1 — which is most brands, which is why the pooled number is high.

Pure-priors share, independently reconfirming R37 with a second dataset: Anthropic flagships 43.1–51.8% (opus) and 49.6–50.9% (sonnet); OpenAI 7.6–11.9% (gpt-5.4) and 20.6–29.0% (mini). No Wilson 95% CI overlap across the provider boundary. Within-provider cross-cell agreement is high — mini→gpt-5.4 98.3%, sonnet→opus-4.6 95.7%, gpt-5.4→5.5 96.3% — so the split is a provider property, not a model-size artefact.

Limitations: modal-stage aggregation discards bimodal cells and needs ≥3 runs; the cross-provider claim is strictly bivariate (OpenAI and Anthropic only — no Gemini, no Perplexity); the Anthropic-side batch-reconstruction calibration is deferred, so the pure-priors signal carries the mechanistic load unaided on that side; the flagship-vs-flagship matrix lacks stable per-prominence stratification.

3.3 The pollution threat model — arXiv 2606.13610 (FORGE, CUHK). Tier A.

Luo & Chen, Chinese University of Hong Kong. Explicit statement: "We received no incentive or compensation from any of the 12 evaluated model providers." 225 real products, 15 categories, 5 consumer scenarios, n = 15 per product per model, 12 production LLMs (6 closed — Gemini 3 Flash, GPT-5.4, o4-mini, Gemini 3.1 Pro, Claude Opus 4.7, Claude Sonnet 4.6 — and 6 open-weight). Frozen evidence bundles, April 2026.

Three escalating pollution styles: A1 entity replacement (rewrite the dominant real-brand mention to a fake brand compound, preserving URL, context, length and style); A2 passage injection; A3 full synthesis under a same-domain URL.

The rank effect is the finding.

Dosage	Fooled rate
Single polluted page at rank 1	27% of cells on the most-vulnerable models
Same page at ranks 2–10	1–4%
Top-3 full replacement	13.3%–73.8% per model
Three polluted pages in top-10	most-vulnerable models cross 50%

Category spread at top-3: dining 81.7%, smartphones 22.8%.

Reasoning makes it worse. Qwen3.5-9B rises from 38.7% to 56.9% fooled with reasoning enabled (+18 pp); GLM-4.6V-Flash from 71.6% to 80.4% (+9 pp).

Defences, and why two of the three are traps:

D1 skepticism prompting — backfires. +24 pp average on closed-source models; Gemini 3.1 Pro +44 pp. Open-weights roughly flat (−3 pp). Mechanism: instructing the model to distrust unfamiliar brands forces deeper engagement with the polluted text and undermines the prior-based resistance it would otherwise have had.
D2 model-prior consensus filter — catches 95% of fake brands but suppresses 62–79% (mean 68%) of legitimate recommendations.
D3 cross-document evidence filter (τ=4) — 90% fake-brand catch, but destroys 52–73% (mean 63%) of real-brand survival.

Limitations: the attack is "not optimized", so a motivated adversary does better; dose-response and confabulation analyses are open-weights only; the main evaluation is Chinese-language with English replication on only 3 categories; single April 2026 snapshot.

3.4 Which manipulations actually work — arXiv 2606.16821 (SearchGEO). Tier A.

Chen et al., KAUST with Jilin, Zhejiang, IDSIA-USI/SUPSI. 13 LLM backends × 308 cases = 4,004 cases, plus ~2,800 auxiliary probes. Hybrid search proxy injecting adversarial content into cached real search results. Domains: health, finance, consumer IT, legal. Metrics: Attack Success Rate, Output Shift Score, Stealth Score.

Attack success by backend — the spread is two orders of magnitude:

Backend	ASR
Claude-Sonnet-4.6	0.0%
GPT-5.4-mini	0.8%
Average across 13 backends	13.4%
DeepSeek-V4-Flash	20.1%
Gemini-3-Flash	31.4%

Trust-signal attacks dominate. Mode 2B, synthetic consensus — manufacturing the appearance of independent agreement — reaches 72.7% ASR on Gemini-3-Flash. Mode 3, authority + citation chain, produces a 15.0% silent-shift rate on attacks that otherwise failed, i.e. the answer moves without the attack registering as a success.

Limitations: static proxy against live ecosystems (~30% snippet-only coverage); mechanism-level rather than population-scale; and a self-judging bias — GPT-5.4-mini is used across the evaluation pipeline.

3.5 The transferable citation-fabrication number — arXiv 2604.03173 (UPenn). Tier A.

Rao, Wong & Callison-Burch, University of Pennsylvania, DARPA SciFy funded, MIT-licensed release. DRBench: 100 multilingual queries, 53,090 URLs, 10 models. ExpertQA: 2,177 expert questions over 32 fields, 168,021 URLs, 3 models (claude-sonnet-4-5, gemini-2.5-pro, gpt-5.1).

Hallucinated URL rates 3%–13%; 5–18% non-resolving overall.
Deep research agents 10.7% hallucinated vs 4.8% for search-augmented models — the agentic layer roughly doubles fabrication.
Domain spread in non-resolving: Business 5.4% → Theology 11.4%; Healthcare/Medicine 5.3%–17.4% depending on model.
Stale versus fabricated is not the same split per vendor. OpenAI search models: 100% of non-resolving URLs were hallucinated, zero stale. openai-deepresearch: 65% stale, 35% hallucinated. Claude: 4.8–5.2% stale.
Self-correction works when the model is competent enough to use the tool. Their urlhealth tool over 435 ExpertQA questions: GPT-5.1 26× reduction (16.0% → 0.6% non-resolving), Gemini 2.5 Pro 79× (6.1% → 0.1%), Claude Sonnet 4.5 6.4× (4.9% → 0.8%), all p < 10⁻³⁵. Post-correction live-URL rates 78.0–88.9%. gpt-5-nano failed to act on tool results across 14 correction rounds — the capability floor is real.

Limitations: hallucination estimates depend on Wayback coverage, which is incomplete and biases toward classifying hallucinated URLs as stale (so the figures are a conservative lower bound); liveness is point-in-time and bot-blocking creates false positives; three providers, two English-dominated benchmarks; non-URL fabrication — invented snippets and bibliographic metadata — is out of scope.

This is the number R41 was waiting for. Digest 04 correctly refused to carry the academic 11.4–56.8% range into commercial brand citation. We now have the commercial-web analogue measured directly: 3–13%.

3.6 Citation is not absorption — arXiv 2604.25707. Tier A, descriptive only.

Zhang Kai, He Xinyue, Yao Jingang, independent researchers, China. 602 controlled prompts, 21,143 valid search-layer citations, 23,745 citation-level feature records, across ChatGPT, Google AI Overview/Gemini and Perplexity.

Absorption = how much a cited page contributes language, evidence, structure or factual support to the answer, via a constructed influence_score: reference count 0.20, position-not-first 0.15, paragraph coverage 0.20, TF-IDF cosine 0.25, bigram/trigram overlap 0.20.

Breadth and depth run in opposite directions:

Platform	Mean citations	Median	Range	Mean influence	Median influence
ChatGPT	6.88	6	0–21	0.2713	0.2611
Google	12.06	12	0–37	0.0584	0.0515
Perplexity	16.35	17	0–27	0.0646	0.0333

Perplexity cites two and a half times as many sources as ChatGPT and absorbs about a quarter as much from each. A citation count is not a visibility measure.

Evidence genre, relative difference in influence:

Genre	Δ influence
Code	+76.88%
Numbers / statistics	+61.55%
Definitions	+57.33%
Comparisons	+55.28%
How-to	+41.20%
Q&A format	−5.74%

Domain type, mean influence: Encyclopedia 0.2144 · Academic publishing 0.1118 · Commercial 0.1028 · Nonprofit 0.0971 · Academic 0.0815 · Government 0.0769 · News media 0.0726.

Language effects are platform-specific with no universal pattern: ChatGPT cites more on Chinese prompts (7.77 vs 7.03); Google cites far more on English (11.57 vs 7.53); Perplexity is flat (16.43 vs 15.93).

Limitations the authors state: the influence score is an observable proxy, not model attention; 602 designed prompts are not a random traffic sample; no unified record-level timestamps; explicitly no causal claims; and fetch bias — 76.44% success rate, with excluded pages possibly systematically different.

3.7 Category concentration is lower than folklore assumes — arXiv 2606.23057. Tier B.

Żatuchin, Estonian Entrepreneurship University of Applied Sciences and Rankfor.AI, which commercialises these metrics (disclosed). 50 brands across 5 industries, 250 category queries, 3 models (GPT-5.2, Gemini 3 Flash, Perplexity sonar-pro), 5 iterations, 3,750 API calls, temperature 0.3, collected 23–24 February 2026.

Mean Gini 0.28, 95% CI [0.16, 0.41] — well below the hypothesised 0.60 power-law threshold. Winner-takes-all is contradicted on this sample.
Top-3 brands take 48.6% of recommendations pooled; range 42.1% (FinTech) to 62.1% (e-commerce).
Industry spread: e-commerce Gini 0.53, SaaS 0.18. Consulting shows co-recommendation (0.4:1 displacement ratio); other industries show substitution (2.0:1 to 4.3:1).
Only 8.0% of queries had no dominant brand, against a predicted 20% — few genuine vacuums.
Cross-model agreement on the top brand: 41.6% — consulting 22%, e-commerce 73.8%.

Tier B is generous. Fifty brands, ten per industry, a two-day window, a vendor that sells the index, a metric that can exceed 1.0, and disclosed alias-matching false positives. Direction only.

3.8 AI-generated sources in the citation pool — arXiv 2605.23684. Tier A, but dated data.

Allaham & Diakopoulos. 712 English queries, 2,848 responses, 26,266 unique URLs (19,154 scraped, 72.9%), 7,675 domains. Engines: ChatGPT with web search, Copilot, Gemini, Perplexity. Detection by Pangram, chosen over GPTZero on false-negative rate.

Engine	Share of cited sources flagged AI-generated
Copilot	27.8% (1,614)
Gemini	14.7% (782)
Perplexity	9.4% (472)
ChatGPT	7.3% (765)
Overall	~16%

By query type: health 16.3%, environment 13.3%, politics 11.1%.

The detector caveat is large and the authors state it. On clean benchmarks both detectors hit 100%, but on a 105-article "AI slop" benchmark Pangram caught 68.6% and GPTZero 69.5% — false negative rates of ~31% and ~30%. The reported shares are therefore floors. Reliance on a single detector is named as a limitation.

Date discipline applies hard here. Politics/health data is from the Search Arena set, 18 March – 8 May 2025; environment from Climate Q&A, 2023. Per R6 this is a two-to-three-year-old model generation. The direction — a material fraction of the citation pool is synthetic, and the share differs several-fold by engine — is what carries forward. The percentages do not.

4. Extracted rules

MUST

R42. Never report a confidence interval derived from repeated runs at fixed prompt, judge and temperature. Naive coverage is 42% at one item and falls to 7% as items accumulate; it worsens with more data. Any LBOX interval must decompose prompt-variant and judge variance, or be reported as a range with the words "resampling only" attached. (A, 2604.11581)
R43. Run a minimum viable pilot before committing a client measurement design: N ≥ 30 items, V ≥ 3 prompt variants, R ≥ 5 replications (~4,050 calls, ~$16). It recovers the design guidance within 5.8% for pairwise scoring and 25.5% underestimation on Likert. (A, 2604.11581)
R44. Match allocation to scoring method. Likert-style scoring: spend on items (39.4% variance reduction). Pairwise scoring: spend on judges (judge variance 20.7% vs 1.7% under Likert). The correct allocation inverts. (A, 2604.11581)
R45. Distinguish recommendation from mention as separate products with separate value. A recommended brand moves downstream behaviour roughly two to three times an incidental mention. Direction only — magnitudes are vendor-observational and not quotable. (B, 2606.10907)
R46. Sell diagnosis as cross-engine and recommendation lists as per-engine. Failure-stage agreement across OpenAI and Anthropic is 95.1% pooled while recommendation-set Jaccard is 0.349–0.356. (A, 2606.26116)
R47. Qualify R46 by prominence and stage. Agreement runs 81.2% (L1) → 99.6% (L5), κ = 0.29, and in cells whose modal failure is S2 or S3 it collapses to 14.9%. Cross-engine diagnosis is trustworthy for discoverability failure and unreliable for compellingness and positioning. (A, 2606.26116)
R48. Report engine-specific absorption, not citation count. ChatGPT mean influence 0.2713 on 6.88 citations; Perplexity 0.0646 on 16.35. A citation-count metric ranks the engines backwards. (A descriptive, 2604.25707)
R49. Where a client's claims are demonstrable, produce code, numbers/statistics, definitions and comparisons. Relative absorption: code +76.9%, statistics +61.6%, definitions +57.3%, comparisons +55.3%, how-to +41.2%. (A descriptive, 2604.25707)
R50. Use 3–13% hallucinated URLs / 5–18% non-resolving as the citation-fabrication figure for commercial web citation, with deep-research agents at 10.7% versus 4.8% for plain search-augmented models. This supersedes the academic range for all commercial statements. (A, 2604.03173)
R51. Treat engine choice as a security variable in any client threat assessment. Endorsement ASR spans 0.0% (Claude-Sonnet-4.6) to 31.4% (Gemini-3-Flash), average 13.4%. (A, 2606.16821)

MUST NOT

R52. Do not recommend "skepticism prompting" or unfamiliar-brand warnings as a pollution defence. It backfires: +24 pp average on closed models, +44 pp on Gemini 3.1 Pro. (A, 2606.13610)
R53. Do not present a consensus filter as free. D2 model-prior consensus catches 95% of fake brands but suppresses 62–79% of legitimate recommendations; D3 cross-document catches 90% and destroys 52–73%. This materially qualifies digest 04's 95.6% / 5.8× consensus finding, which recorded the precision gain and not the recall cost. (A, 2606.13610)
R54. Do not assume reasoning-mode or higher-effort settings improve robustness. Reasoning raises pollution vulnerability by +9 to +18 pp on the models where it was measured. (A, 2606.13610)
R55. Do not assume a category is winner-takes-all before measuring it. Mean Gini 0.28 [0.16, 0.41], top-3 share 48.6%, and only 8.0% of queries lack a dominant brand — but cross-model agreement on the single top brand is only 41.6%. (B, 2606.23057)
R56. Do not treat a Q&A-formatted page as a citation asset. It is the only genre measured with negative relative absorption, −5.74% — against how-to at +41.2%. (A descriptive, 2604.25707)

WORKS / DOESN'T WORK — updated

Works	Doesn't work
Selling failure-stage diagnosis as engine-portable (S1 only)	Selling a recommendation list as engine-portable
Measuring absorption per engine	Counting citations as a visibility proxy
Code, statistics, definitions, comparison artifacts	Q&A-formatted pages (−5.7%)
Encyclopedia-class placement (0.2144 influence)	News-media placement as a primary absorption route (0.0726)
Agentic URL self-correction where the model is capable (6.4×–79×)	Small models given the same tool (gpt-5-nano: 0/14 rounds)
Owning rank 1 of the retrieval pool	Owning ranks 2–10 (1–4% influence on the pollution analogue)
Pilot-then-design (N≥30, V≥3, R≥5)	Adding runs to a single-prompt design
Recommendation stance as the product	Mention volume as the product

5. Revalidation of standing rules

Every prior rule, as required. No rule is silently dropped.

Rule	Status this week
R1 parity signals dominate; brand identity ~1.2–1.5% of variance	CONFIRMS the frame, ADDS a fourth independent angle. 2606.10907 finds no prominence gradient in behavioural lift (Spearman +0.02 to +0.12, n.s.) — once a brand is named, who it is stops mattering for the downstream effect too. Consistent with R1 governing behaviour inside a consideration set.
R2a publish verifiable third-party authority evidence	CONFIRMS and sharpens into a genre rule. Absorption by evidence genre puts statistics (+61.6%), definitions (+57.3%) and comparisons (+55.3%) far above prose. Authority evidence is absorbed; see R49.
R2b authoritative register is not a lever	CONFIRMS, and now with an adversarial twist. SearchGEO's most effective attack class is the trust signal — synthetic consensus 72.7% ASR, authority+citation chains 15.0% silent shift. Authority cues move models when fabricated at scale, which is exactly why register alone does nothing legitimate. Recorded under R7, not as a lever.
R3 tune intensity per engine	CONFIRMS emphatically, on three new axes. Absorption (0.2713 / 0.0584 / 0.0646), attack resistance (0.0% / 13.4% / 31.4%), and synthetic-source contamination (7.3% / 27.8%) all differ several-fold by engine.
R4 competitor GEO adoption is existential	CONFIRMS, threat now fully documented. FORGE and SearchGEO are the multi-actor evidence the survey said was missing — though for the adversarial case, not the white-hat one. The 45-study gap on equilibrium effects at high adoption remains open.
R5 verify every engine-supplied citation	CONFIRMS and gains an operational tool. 2604.03173's `urlhealth`: 6.4×–79× reduction in non-resolving URLs, p<10⁻³⁵. Verification is now automatable, with a capability floor — gpt-5-nano failed 14 rounds.
R6 state model set and date	CONFIRMS, and this digest had to enforce it. 2605.23684's headline shares rest on 2023–2025 data. Direction carried; percentages not.
R7 PROHIBITED — fabricated authority	UPGRADED C → A, and the prohibition is now quantified. The cluster is read: one rank-1 polluted page = 27% fooled; three pages cross 50%; average endorsement ASR 13.4%. R7 previously rested on CCTV reporting via search summary. It now rests on two peer-review-grade audits with non-commercial authors. The client-education case is complete.
R8 don't stack persuasion tactics blindly	CONFIRMS from an unexpected direction. D1 skepticism prompting backfires +24pp: adding an instruction produced the opposite of its intent. Instruction stacking has unmodelled sign flips.
R9 don't invest in scarcity / loss-aversion / anchoring	NO NEW EVIDENCE.
R10 no single-run measurement	*CONFIRMS, and is now insufficient on its own.* 2604.11581 shows multi-run is necessary but not sufficient: repeated runs at one prompt produce 7% coverage. R10 must always be paired with R42.
R11 no tier C/D presented as established	CONFIRMS. Applied to 2606.10907 (capped B despite good design), 2606.23057 (B), 2602.06718 (C, unread).
R12 test each engine's believed brand origin	NO NEW DIRECT EVIDENCE.
R13 geo-stratify every measurement	CONFIRMS, with a warning against over-generalising. 2604.25707 finds language effects are platform-specific with no universal pattern — ChatGPT cites more on Chinese, Google far more on English. Stratify; do not assume a single language penalty.
R14 counterfactual location probes	NO NEW EVIDENCE.
R15 measure origin sensitivity per engine	NO NEW EVIDENCE.
R16 don't infer bias from where the engine was built	NO NEW EVIDENCE.
R17 don't assume local-to-buyer framing helps	NO NEW EVIDENCE.
R18 don't carry open-model effects to commercial engines	CONFIRMS strongly. FORGE's open-weight models run far more vulnerable than closed (GLM-4.6V-Flash 71.6–80.4% vs closed-source floors), and the defence results invert by class — D1 backfires on closed, is flat on open.
R19 classify on the prominence ladder first	CONFIRMS from a third independent dataset, and gains a new use. 2606.26116 shows the ladder predicts diagnostic portability (81.2%→99.6%), not just visibility. The ladder is now doing work in three separate places.
R20 L4–L5: discoverability is the whole problem	CONFIRMS decisively. 99.3–99.6% of L4/L5 joint failures are the same stage across providers, and the pooled agreement is chance-driven at 93.0% because Stage 1 dominates.
R21 L1: invest at S2–S3	CONFIRMS, and explains the L1 anomaly. L1 is exactly where agreement is lowest (81.2%) and where S2-vs-S3 disagreement concentrates — because L1 brands are the only ones whose failures are S2/S3 at all.
R22 L3: all three failures, persona instability 0.39–0.75	NO NEW EVIDENCE on the swap rate. L3 diagnostic agreement 96.2%.
R23 establish a rerun-stability baseline per cell	CONFIRMS — the 0.50–0.61 floor reproduces exactly in an independent Unusual dataset, and cross-provider Jaccard (0.349–0.356) sits cleanly below it, which is how you know the divergence is real signal and not noise.
R24 measure through more than the native tool	CONFIRMS. Citation breadth ranges 6.88 to 16.35 by platform on identical prompts.
R25 separate descriptive from causal	CONFIRMS, and was enforced twice. 2604.25707 explicitly reserves causal claims to a future Level 4; 2606.10907 states its intent shock is "not closable without a randomized design." Both are honest; neither is causal.
R26 don't sell reputation work to L4–L5 without establishing surfacing	CONFIRMS. Axis 9 returned its third consecutive null — there is still no published sentiment→recommendation effect in answer engines. Three sweeps, no evidence.
R27 don't quote Aggarwal +40% as uniform	NO NEW EVIDENCE; stands.
R28 don't assume Sonnet→Opus behaves like mini→full GPT	QUALIFIES — and the qualification is favourable. 2606.26116 measures within-provider cross-cell agreement at 95.7% (sonnet→opus-4.6), 98.3% (mini→gpt-5.4) and 96.3% (gpt-5.4→5.5). Within a provider, class and generation transfer better than R28 assumed. The boundary that matters is the provider boundary, not the size boundary — pure-priors share shows no CI overlap across providers. Restate R28 as: do not transfer across providers; transfer within a provider with a stated 4–5% error.
R29 no rules from uncited practitioner articles	CONFIRMS. §7 quarantined in full.
R30 claim only conditional effects	CONFIRMS. Nothing this week demonstrates durable cross-platform discoverability gain.
R31 never claim visibility produces clicks/traffic/conversions	QUALIFIED — the most significant change this week. 2606.10907 supplies the first measured behavioural link: recommendation → +4.3pp recall / +2.4pp own-site / +1.0pp retailer, with vanishing pre-trends and flat same-category controls. Restate R31 as R31a/R31b. R31a: LBOX may state, direction-only and attributed, that being recommended raises downstream brand search and site visits. R31b: LBOX may not claim traffic volumes, conversions or revenue — purchase is unobserved, the design is observational, and the sample sizes are withheld. The survey's "Very Low" rating is not overturned; it is now one grade less empty.
R32 size samples by outcome (≥7–8 runs; 5-then-breadth for sentiment)	QUALIFIED on the interval, CONFIRMED on the counts. The run counts remain the right floor for resampling noise. The intervals built from them are not 95% intervals — see R42. Any deliverable quoting "±0.158 at n=7" must now either add prompt-variant components or drop the interval.
R33 query language is a first-order design factor	CONFIRMS, with the pattern-uniformity assumption removed. Language effects are real and large, but platform-specific in sign, not just magnitude.
R34 report always/never/unstable splits	CONFIRMS and gains a mechanism. 2606.26116's finding that 95.1% of joint failures share a stage is the structural reason so many cells are deterministic: most brands fail at S1, and S1 failure is stable.
R35 price sentiment separately from mention	CONFIRMS. Axis 9 null again; no counter-evidence.
R36 measure engine self-promotion where the client competes with the provider	NO NEW EVIDENCE.
R37 content/retrieval levers have less purchase on Anthropic models	CONFIRMS on a second independent dataset with tighter numbers. Anthropic 43.1–51.8% / 49.6–50.9% pure-priors vs OpenAI 7.6–11.9% / 20.6–29.0%, no Wilson CI overlap. Digest 04 flagged this as cross-paper co-occurrence; it is now measured within one design. Reconciles with the 2026-08-19 Searchable digest: when Claude does retrieve, retrieval matters (Brave 63% overlap, press 52% conversion) — it simply does not retrieve about half the time.
R38 authoritative tone is not a lever	CONFIRMS. See R2b.
R39 generic GEO heuristics/recipes are unreliable	CONFIRMS and refines toward genre. The one thing that does generalise in the absorption data is artifact genre, not formatting recipe.
R40 don't recommend "cite external sources" uniformly	CONFIRMS with a new hazard. SearchGEO shows citation chains are the authority attack vector (Mode 3, 15.0% silent shift). Uniform "add citations" advice now also has a security cost.
R41 don't carry 11.4–56.8% into commercial brand citation	RESOLVED — SUPERSEDED by R50. The prohibition was correct and is now moot: the commercial-web analogue is measured at 3–13%. R41 stands as a historical prohibition; R50 is the operative rule.

Candidate principles

CP	Status
CP-01 temporal decay of citation is category-dependent	*THIRD FAILED ATTEMPT. Remains tier D, and confidence in it should now be falling, not flat.* The best-positioned paper in three sweeps (2604.25707, 21,143 citations, three platforms, explicitly controlling "time sensitivity" in prompt design) declines to analyse publication date at all, citing absent unified timestamps. Nobody is measuring source age independently of relevance and authority — not because it is hard to find, but because it appears not to be a variable the field considers load-bearing. Three sweeps, zero evidence, while the practitioner tier reproduces the claim every single week. Recommendation: stop treating CP-01 as "awaiting evidence" and start treating it as "probably an artifact of practitioner folklore" until someone measures it. See §6.
CP-02 supersession	MOVED — first real mechanistic evidence, and it points the wrong way. 2605.05957 (tier A, 300 false premises, 7 error categories, 21 domains, 8 models) finds models correct false claims reliably in isolation but suppress the correction when the same claim is embedded in a task-framed request: GPT-5.1 96.3% → 10.0% (89.6% suppression); DeepSeek-V3.2 84.0%; Gemini 3 Flash 82.9%; Grok 4.1 Fast 81.2%; LLaMA3.1-8B 47.4%; Qwen3.5-9B 46.0%; Claude Sonnet 4.5 31.0%; Qwen3.5-Plus 19.3%. The model registers the error internally and does not surface it. This is a different setting from CP-02 — false premises in the user's prompt, not corrective content in the retrieval set — so it is tier D if transferred and does not resolve the candidate. But it establishes that correction is actively suppressed by task framing, which is the framing every commercial buyer query has. It is bad news for the reputation add-on and it should change how the S3 experiment is designed: measure whether the correction surfaces under a task-framed commercial prompt, not under a direct "is this claim true" probe, or the experiment will report a result that does not transfer.
CP-03 category classification sets cadence	REFINED AGAIN, still unresolved, and the artifact-type half is now well-evidenced. Genre absorption (code +76.9%, statistics +61.6%, definitions +57.3%, comparisons +55.3%, how-to +41.2%, Q&A −5.74%) and domain-class absorption (encyclopedia 0.2144 vs news media 0.0726) are direct measurements of which artifact earns influence — the exact refinement CP-03 predicted mattered. Two independent datasets now support artifact-type. The cadence half remains entirely unsupported and depends on CP-01, which is failing. Recommendation: split CP-03. Promote the artifact-type component toward a tier-A rule (it is already R49/R56). Leave cadence in the register as tier D, tied to CP-01's fate.

6. What this means for LBOX

Three things changed that a client would notice.

First, the honest answer to "does this actually do anything for my business" got better, and it is still not what a client wants to hear. For four digests the answer has been that the causal literature rates visibility→conversion Very Low. It still does. But there is now a measured behavioural link on the open web — being recommended raises same-name search by about four points and own-site visits by about two over seven days, roughly two to three times what a passing mention does. That study is observational, its author sells the measurement, and it withholds every sample size, so LBOX can cite the direction and must not cite the numbers. What it changes commercially is narrower and more useful than a headline: it makes recommendation stance, not mention count, the thing worth selling. A product that raises mentions and a product that raises recommendations are now known to be different products with different downstream effects.

Second, the measurement standard LBOX was positioning on has a hole in it, and this is the item that most needs Adam's attention. Digest 04 concluded that LBOX's most defensible product is measurement done to a standard nobody else is meeting, and R32 encoded the run counts. Those counts are fine. The intervals are not. A ±0.158 interval from seven runs at one prompt is, on this week's tier-A evidence, covering the truth about 42% of the time at best and 7% at scale — and it gets worse as you collect more. Every competitor in this market reports intervals this way; so, right now, would we. The fix is cheap — a $16 pilot at N≥30, V≥3, R≥5 — and it is a genuine differentiator if we do it and say so. It is also a liability if a sophisticated buyer's data-science team reads 2604.11581 before we do.

Third, the adversarial story is now fully evidenced and it sells itself. R7 has been a prohibition backed by a television broadcast. It is now backed by two audits with non-commercial authors: a single polluted page at rank 1 fools vulnerable models a quarter of the time, three pages cross a coin flip, reasoning modes make it worse, and the intuitive defence makes it worse still. For client education this is the strongest material in the library, and it is the one place where LBOX can be unambiguously on the right side of a measured finding.

On CP-01, a position rather than a hedge. Three sweeps have now failed to find any study isolating source age from relevance and authority, including one that had 21,143 citations and declined to look. Meanwhile the practitioner tier reproduces the claim, uncited, every week. The parsimonious explanation is not that the field has overlooked an important variable — it is that freshness folklore is SEO intuition transplanted into a setting where nobody has shown it applies. I am not proposing to kill CP-01, because absence of measurement is not evidence of absence. I am proposing that any LBOX product decision currently justified by "fast categories need refresh cadence" be re-justified on other grounds or marked as unsourced judgement. That is a decision for Adam, not for me.

Flag for Adam, unsourced judgement. The reputation add-on now carries a third warning. Digest 03 found no sentiment→recommendation effect. Digest 04 priced sentiment measurement at 6.7× mention against a 91.9% neutral floor. This week, 2605.05957 shows that the models most likely to be in front of a buyer suppress corrections by 80–90% when the claim arrives inside a task-framed request — which is what a commercial buyer query is. None of that proves the product cannot work. All of it says the mechanism it depends on has never been demonstrated and one adjacent mechanism now points against it.

7. Practitioner sweep — QUARANTINED, tier C, not usable for any rule

Recorded to detect market narrative only. Every claim below is uncited or vendor-produced.

Recurring themes this week: 80% of brands cited at least once but only 15% securing the top citation with their own domain; brands with mentions and citations claimed 40% more likely to resurface across consecutive queries; an Ahrefs analysis of 75,000 brands claiming YouTube mentions predict AI visibility at 0.737 against backlinks at 0.218; 40–60% of cited sources claimed to change month-to-month across Google AI Mode and ChatGPT, with 1,127 unique cited URLs reduced to 119 still cited six weeks later; and three-layer "presence / accuracy / positioning" measurement frameworks.

Assessment. The volatility claims are directionally consistent with Schulte (tier A, ~65% day-to-day source turnover) and are the only quarantined material with independent corroboration. The Ahrefs correlation figure is the kind of number that will be quoted at us in sales conversations — it is a bivariate correlation with no controls, reported by a vendor selling backlink data, and it is not evidence that YouTube mentions cause citation. Notably, the freshness/refresh-cadence folklore did NOT dominate this week's practitioner sweep — the narrative has shifted toward off-site brand signals. Recorded as a change in market narrative, nothing more.

Still unassessed, in folder: Athena-State-of-AI-Search-Report-2026.pdf (added 2026-08-16, vendor, tier C by default). Carried a second week.

8. Open queue — highest priority first

Fix the interval methodology before the next client deliverable. Implement the R43 pilot (N≥30, V≥3, R≥5) on LBOX's own engine set and recompute at least one existing client interval both ways. This is now the highest-value item in the queue because it is cheap, it is a differentiator, and shipping the naive version is a liability. (New, from 2604.11581.)
Redesign the S3 supersession experiment under task framing. Carried from digest 03/04, now with a design constraint: probe under a commercial task-framed prompt, not a direct truth query, or the result will not transfer. Still the highest-value original research available to LBOX and still what resolves CP-02. (Modified by 2605.05957.)
Read arXiv 2602.06718 GhostCite — 13 LLMs, 40 domains, 375K citations, 14–95% hallucination spread. Tier C on search summary only. Would give a second independent commercial-adjacent fabrication range against R50's 3–13%. Also 2605.07723 (large-scale non-existent citations), 2604.18880 (neuron-level tracing), 2603.07287 (deployment constraints × hallucination).
Read arXiv 2605.24245 Deep-Research Agents Poisoned via UGC — the last unread member of the pollution cluster. Agentic layer already shown to double URL fabrication (10.7% vs 4.8%); UGC poisoning of agents is the natural completion of the R7 threat model.
Read arXiv 2606.12439 Position: GEO Creates Underexamined Risks — governance-facing, names concentration, disclosure and academic blind spots. Relevant to how LBOX positions disclosure.
Read arXiv 2603.03299 in full — carried from digest 04. Lower priority now that R50 supplies a traced commercial figure, but still needed to move the 95.6% multi-model consensus filter from tier C to A. Note it must now be read alongside 2606.13610's D2 result (95% catch, 62–79% legitimate suppression) — the consensus filter's recall cost is the missing half.
Read arXiv 2601.08363 PosIR and the listwise-reranking positional-bias work — primacy bias in embedding models increasing with document length is a mechanism directly under FORGE's rank-1 finding.
Read arXiv 2603.18300 Auditing Preferences for Brands and Cultures and 2604.15937 Polarization by Default — both surfaced on axis 1, both unread.
Assess Athena-State-of-AI-Search-Report-2026.pdf (in folder, unread, vendor). Second week carried.
Carried from digest 02, still unread: Itzhak et al. 2507.07186; Lichtenberg et al. 2406.01285; Cao 2025 WSDM; Mallen et al. 2023 — now the single most relevant of the four, since R37's provider split has been reconfirmed and Mallen is the parametric-vs-retrieval mechanism paper.
Trace the Nature Communications 2025 medical-citation claim (carried from digest 03). Lowest priority; R50 supersedes its likely use.
Test R1–R4, R19–R24 against LBOX's own engine set. Still none reproduced in-house. Open since digest 01 — this is the sixth consecutive digest carrying it.
From the 2026-08-19 Searchable digest, unmoved: reproduce prompt-split trigger rates in-house; test the 63/34 Brave overlap on our query set; watch quarantine for watermark-ranking folklore (pre-registered — no sighting this week).

Published by LBOX.com, a ZLABS platform