Evidence, editorial refresh and citations: registered field test
We have registered a three-arm test of 45 live answer pages; data collection is pending, and no outcome has been measured.
Observed answers on stated dates, not a ranking.
The accepted OSF registration preserves the complete protocol and exact proposed additions. Its registration timestamp is November 03, 2026 at 00:10:26 UTC. Acceptance and archival completion were independently confirmed at November 03, 2026 at 00:18:29 UTC; that is a confirmation time, not an inferred approval timestamp. Registration is a prospective commitment, not peer review or evidence of an effect. We will publish the result whatever its direction, including a null estimate or an implementation failure.
This test asks a narrow question within generative engine optimization(GEO), AEO and AI search visibility: does adding a relevant statistic, a checked quotation and a primary-source link change citation of the page beyond a similarly sized editorial addition? It concerns our selected documentation pages, rather than client work or a universal recipe. You can inspect the page list before any baseline answer is collected.
What is fixed in the registered plan?
The sample has 45 pages:15 assigned to evidence,15 to an editorial placebo and 15 left unchanged. Fifteen blocks contain one page from each arm. The three clusters are measurement, GEO basics and content fixes, with five blocks per cluster. The selection frame contains 77 eligible general pages within the 300 served answer pages checked on November 2,2026.
Eligibility excludes client pages, private cases, named-provider procurement, local services and platform-traffic questions outside the added evidence. This restriction makes a relevant addition possible; it also limits where a result can be generalized. Assignment is fixed before any answer measurement. A page does not leave the sample because its initial citation rate is low, its later result is unfavorable or its source becomes difficult to retrieve.
Historical page impressions supplied a limited additional matching variable. Only three eligible pages had observed positive Web impressions in the retained August 1 throughOctober 31,2026 export, with counts 5,2 and 4. The export is capped, and a missing row is UNKNOWN rather than zero. All three observed-positive pages are included, one in each arm. Each arm also contains 14 UNKNOWN pages. We do not call this complete matching on retrieval potential.
Within each cluster, the plan randomly orders the eligible unknown pages, selects 14 alongside its known page and forms five blocks. Arm labels are permuted within every block, with full assignments redrawn until the three known pages occupy different arms. The deposited assignment fixes the actual 45 URLs, blocks and arms, so the allocation does not depend on a later choice of favorable pages. Source fingerprints use a stated text normalization and remain downloadable.
What changes in each arm?
The direct answer already appears as the first paragraph on every selected page. It stays there in every arm. This design therefore cannot estimate an effect of moving an answer to the opening, and we will not describe an unchanged layout component as an intervention.
The evidence arm adds two paragraphs before the existing explanatory section. Each addition contains a relevant statistic with its count, base and observation period, a short exact quotation from our published research and a link to the primary study. The original text and qualifications remain. The added source-choice counts do not become accuracy estimates, and page-signature counts do not become causal citation effects.
The placebo arm also adds two paragraphs in the same location. They expand and reflow the existing explanation without a new factual proposition, source, quotation or figure. Both modified arms therefore add text. The comparison does not give evidence pages a length increase while leaving placebo pages with a zero-length reflow. Additions range from 118 to 127 words, and the treated and placebo additions within a block differ by no more than four words.
Control pages receive no content, modification-date, metadata, navigation or link changes. Both modified arms use the same modification-date convention in a single release. The exact proposed additions are frozen with the registration. A necessary truth or security correction takes precedence over the freeze, is dated and remains an implementation deviation in the analysis.
Which answers will be observed?
The five products and modes are ChatGPT with web search, Claude with web search, Perplexity with web search, Google AI Overviews in its normal search surface and Google AI Mode. The declared location is the United States where supported, and the actual network country is recorded. English, account/search state, unsupported settings and configuration changes remain visible. We do not invent a location control that a surface does not expose.
One question is used for each page, from its existing public question heading, with the same wording in both waves. Three independently initiated repeats per question and product give 675 scheduled observations per wave and 1,350 across baseline and post. The order mixes arms within time bands. A refusal, absent overview, empty answer or service error stays in the scheduled record rather than being replaced with a more favorable return.
The registration distinguishes a scheduled outcome from a linked diagnostic retry. A retry cannot silently replace the original result or expand the primary denominator. There is no optional second post wave in this confirmatory plan. More repeats or another wave require a prospective amendment before the relevant data, with the original observations retained and the later work labeled appropriately.
What is the calendar?
The planned baseline window opens November 6,2026 at 14:00 UTC and must finish and validate before November 9 at 14:00 UTC. The fixed edit date is November 9, with a release window from 14:00 to 15:00 UTC, or 09:00 to 10:00 Eastern standard time. No experimental page edit occurs before the baseline is validated.
Actual completed public deployment defines time zero. Post collection opens 21 full days later, on November 30 at the matching completion time, and lasts no more than 72 hours. The lag allows recrawl and indexing; it does not guarantee that either occurs. If the edit window cannot be met, the plan requires a prospective calendar amendment rather than a silent late edit. Late, incomplete or interrupted observations remain disclosed.
The report, claim audit, academic access page, protocol and discovery links are planned publications before baseline begins. Their completed release times are recorded as common context. Baseline cannot start until that publication work finishes. Thereafter the context and sample remain fixed through post collection apart from necessary dated corrections. Individual sample pages are not differentially promoted or submitted for recrawling by arm.
Which outcomes remain separate?
Retrieval means the exact target page appears among the sources a product exposes as consulted. When that information is not exposed, the outcome is null, not zero. Its eligible exposure base is printed beside the full scheduled base so that missing visibility into retrieval cannot become an absence finding.
Citation means the answer explicitly cites the exact page URL or a checked canonical equivalent. The primary citation base retains all scheduled attempts, including failures and no-overview returns as unsuccessful exact-page citations. A valid-return sensitivity is reported separately. An answer's mention of Broadcastwell is another outcome: a source URL alone is not an answer-text mention.
Position is the first target-page appearance in an exposed ordered source list. Reciprocal rank is zero for an uncited attempt where citation order is exposed; unexposed order is null. The analysis also reports conditional rank among cited returns. Different paths, languages or subdomains do not become equivalent merely because they name the same organization. Each canonical decision remains inspectable.
How will the comparison be analyzed?
For each page and product, the analysis computes the post-minus-baseline change over its three scheduled repeats. Citation and mention give each of the five products equal weight within page, then each page equal weight within arm. The main contrast is the evidence-minus-placebo difference in these page changes. Placebo-minus-control is the freshness check; evidence-minus-control and the within-arm changes remain secondary descriptions.
Retrieval and reciprocal rank have a separate exposure-conditional analysis. A page-product change needs a non-null exposed outcome in both waves. The available paired products receive equal weight within page, with all usable page, product and attempt bases printed. If no paired exposure exists, the contrast is not estimable. It is not recorded as zero, and it cannot replace the full-scheduled-base citation result.
The uncertainty plan resamples five whole blocks within each of the three clusters,10,000 times, retaining pages, products, repeats and paired waves together. It reports percentile 95%intervals. A randomization sensitivity respects both within-block assignment and the original balance of known-impression pages. Repeats are not treated as independent pages. Citation and its evidence-minus-placebo contrast are primary; other outcomes remain secondary, with Holm adjustment across the four outcome tests in any joint claim.
What can this sample resolve?
Fifteen pages per arm produce 225 scheduled observations per arm per wave, not 225 independent pages. At a planning citation probability of 0.05, the approximate detectable absolute contrast is 8.1 percentage points with no within-page correlation and 12.6 points with correlation 0.10. At probability 0.50, those approximations are 18.7 and 28.9 points. The protocol prints the equation and assumptions, including zero assumed beneficial baseline/post covariance.
These are planning approximations at two-sided 5%and 80%power, not a power guarantee for the registered bootstrap. A small true effect may remain unresolved. A wide interval crossing zero does not establish equivalence or prove no effect. The final report will show the estimate, interval, denominators and implementation record together, rather than select a favorable product or outcome after inspection.
What our own data shows
No answer outcome exists for this field test yet. Our completed descriptive studies concern the September 2026 software-answer archive and later page observations. They establish the source and meaning of the proposed additions; they do not predict whether these additions will help a target page receive a citation. The distinction between a completed description and a prospective experiment remains visible here.
Supports: a fixed prospective design, allocation, calendar, outcomes and analysis commitment. Suggests: a controlled comparison of this complete evidence package with editorial freshness on selected pages. Does not support: an effect estimate before data, an isolated component effect, a universal tactic, a product-quality ranking or a revenue promise. A negative result, null interval or compromised implementation will be published with the same prominence as a positive estimate.
Related studies
Read the research hub, the page-signature study, the citation-source study, the measurement library and the Index method.
Sources
- Broadcastwell research team. (2026). Evidence, editorial refresh and citation of answer pages [Registered protocol]. OSF. https://doi.org/10.17605/OSF.IO/DMFPJ. Registered November 03, 2026 at 00:10:26 UTC; acceptance and archival completion confirmed November 03, 2026 at 00:18:29 UTC.
- Full protocol PDF, assignment CSV and intervention JSON. Version 2, November 2,2026; frozen before baseline.
- Broadcastwell research team, Priya Nair, Ananya Rao and Vikram Patel. (2026). AI search visibility: source evidence and two registered analyses. https://doi.org/10.5281/zenodo.23804758. CC BY 4.0.
- Broadcastwell. (2026). Absence Index, September 2026. https://doi.org/10.5281/zenodo.22907695.
- Claim Registry and academic access retain the source and historical-data boundaries.
Cite this page
Broadcastwell research team. (2026, November 3). Evidence, editorial refresh and citations: registered field test. Broadcastwell. https://docs.broadcastwell.com/research/studies/evidence-refresh-field-test/
@online{broadcastwell2026fieldtest,
author = {{Broadcastwell research team}},
title = {Evidence, editorial refresh and citations: registered field test},
year = {2026},
date = {2026-11-03},
url = {https://docs.broadcastwell.com/research/studies/evidence-refresh-field-test/},
doi = {10.17605/OSF.IO/DMFPJ}
}
Prepared by the Broadcastwell research team. Lead: Vikram Patel.
Sources checked November 3, 2026. Not peer reviewed.
Last updated:2026-11-03. Next review:2027-02-02.