>_ ANALYSIS
Agreement is not independent evidence
Before treating consensus as confirmation, examine how the judgments were produced. Shared inputs and correlated estimates require different checks.
Several matching judgments can be reassuring. Before increasing confidence, I would ask a different question: how much distinct information does the agreement contain?
For an intelligence team, the practical issue is not whether consensus is inherently good or bad. It is whether the process producing that consensus has been examined.
What the collected research reports
The abstract of “Bayesian combination of correlated subjective probability estimates,” collected in OV INT, describes a model that accounts for dependence between forecasters’ estimates. In an evaluation involving 85 forecasters and 180 queries, the authors report improved Brier-score performance relative to related Bayesian fusion models, including models that assumed independence and became overconfident.
The abstract describes question difficulty as a source of correlation. It does not establish that the forecasters copied one another, shared a source, or behaved like a real intelligence team. That distinction matters: correlated outputs and duplicated evidence are related analytical concerns, but they are not interchangeable explanations.
My proposed workflow: inspect agreement before using it
I would begin by separating the estimate from its provenance. Ask each contributor for a probability, the evidence driving it, the strongest alternative explanation, and the observation that would change their view. Record this before a group discussion where practical.
Then map overlap. Are several assessments built on the same original report? Did contributors see each other’s estimates? Are they applying the same model? These questions can reveal possible dependence; they do not measure its size by themselves.
A useful check is to remove one shared assumption and ask contributors to revise their judgments independently. If the apparent consensus disappears, the assumption deserves attention. That is a proposed sensitivity exercise, not a calibrated mathematical correction.
Do not replace one shortcut with another
The wrong response would be to dismiss every agreement as groupthink. People can agree because evidence is strong or a question is straightforward. Disagreement is also not proof of intellectual independence: contributors can disagree while relying on the same incomplete input.
I would avoid claiming an exact “effective number of independent sources” from a visual source map. Without a defensible model and appropriate data, a precise number would add an appearance of rigor the process has not earned.
For a live assessment, describe the limitation directly: several judgments converge, but some share an evidential basis, so the agreement should not be counted as wholly separate confirmation.
Evaluate the process over resolved questions
Retain forecasts before outcomes are known and define resolution criteria in advance. Compare how different aggregation rules perform across a suitable set of resolved questions, including their calibration and their sensitivity to shared inputs.
The research abstract motivates this investigation; it does not select the right aggregation rule for every operational setting. I have not independently reproduced its dataset or results.
My analytical position is therefore procedural: make the production of consensus visible before assigning weight to consensus itself. The useful follow-up to “everyone agrees” is “what did each judgment add?”
Source and scope
This article uses the abstract stored in OV INT. The team workflow and sensitivity exercise are my proposed applications, not interventions tested by the cited study. Conclusions are limited accordingly.