How many repeated runs make an AI visibility rate useful?
There is no universally sufficient number of repeated runs in these studies. Procore was named in 3 of 3 Google AI Mode answers to one construction question on 22 September 2026, yet the descriptive 95% Wilson interval was 43.9% to 100.0%. Three recorded appearances establish what happened in those runs, with considerable uncertainty about a broader rate.
What we measured
Measured. The September Index scheduled three runs of each fixed buyer question. The Procore example concerns the question about construction-management software for a small residential builder. It is a single engine-by-question cell, rather than three different buyers or a test of purchase intent.
An earlier Volume III repeat collection observed twenty-five selected questions three scheduled times on 4 August 2026. Across 74 usable Claude repeat pairs, mean vendor-set Jaccard agreement was 0.442, with bootstrap 95% interval 0.389 to 0.494. This measured shortlist variation, a different outcome from the September binary Procore naming rate.
Reasoned. A perfect small cell and a variable set comparison both caution against treating a chosen run count as certainty. Neither source tests whether five, ten or twenty repeats are enough for every use. Repeated outputs under shared conditions also should not be described as independent buyer decisions.
Our answer-study method, v1.1.
What this means for you
State what uncertainty would affect your decision before selecting a run count. A broad category summary and a question-specific claim can require different evidence even if both are called visibility rates.
Ask a supplier to disclose the scheduled runs, usable answers, exclusions and interval calculation. More runs are informative only when you can see what stayed fixed and what was counted.
Treat a study's run count as a reproducible design choice. If someone claims their count is sufficient for your decision, ask for the justification and its limits rather than borrowing the authority of a different study.
Also asked as
How many repeated runs make an AI visibility rate useful?
Related
Next step
With Free 10-question check (one engine), retain the recorded run conditions instead of turning one result into a stable rate. No email is needed; runs are limited per network and per day, and by the site's daily allowance.
Use Fix one answer, $490 for a specific evidenced correction when that is the task. Agree separately how any later answer comparison will handle variation.
Claiming or correcting a vendor record is free and never depends on a purchase.
Prepared by the Broadcastwell team.
Page owner: Vikram Patel, AI Architect. Corrections: hello@broadcastwell.com