Within Weakest Link
What Else Could Explain the Same Evidence?
A conclusion is weaker when another explanation fits the same facts better than the story you started with.
On this page
- Why missing alternatives create fragile conclusions
- How to compare rival explanations fairly
- Examples from diagnoses, hiring and customer behaviour
Page outline Jump by section
Introduction
A conclusion becomes fragile when it treats one explanation as if it were the only explanation. In practice, the same facts can often support several different stories. The question is not simply, “Does this explanation fit the evidence?” but “Does it fit the evidence better than the strongest alternative?” Weakest-link thinking therefore places special emphasis on rival explanations. If an overlooked alternative explains the evidence just as well—or better—the original conclusion loses much of its force, even if none of its individual facts are wrong. Structured analytical methods, scientific reasoning and clinical practice all rely on this principle because comparing competing explanations is often more informative than collecting additional supporting evidence alone. [Department of Statistics+2ialeia.org]stat.berkeley.eduDepartment of StatisticsStructured Analytic Techniques for Improving Intelligence…April 28, 2009 — by AT Primer · 2009 · Cited by 62 —…
Why missing alternatives create fragile conclusions
Most reasoning errors are not caused by a complete lack of evidence. They arise because evidence is evaluated against only one hypothesis.
Suppose a company sees falling sales after raising prices. One explanation is that customers are price-sensitive. Another is that a new competitor entered the market. A third is that product quality declined. A fourth is that the broader economy weakened. The observation—lower sales—is real, but it does not identify the cause by itself.
When people fail to generate plausible alternatives, they often mistake compatibility for proof. Evidence that is compatible with an explanation is not necessarily evidence that distinguishes it from competing explanations.
This distinction is central to critical thinking:
- Supporting evidence shows that an explanation could be true.
- Diagnostic evidence helps distinguish between competing explanations because it fits one much better than the others.
A fever, for example, supports many possible diagnoses. A laboratory test identifying a particular pathogen is far more diagnostic because it separates one explanation from its rivals.
Research on structured analytic techniques similarly argues that systematically considering alternative explanations reduces the risk of overlooking relevant possibilities, particularly when information is incomplete or ambiguous. [Department of Statistics+2ialeia.org]stat.berkeley.eduDepartment of StatisticsStructured Analytic Techniques for Improving Intelligence…April 28, 2009 — by AT Primer · 2009 · Cited by 62 —…
How to compare rival explanations fairly
The goal is not to invent endless possibilities. It is to identify the strongest realistic competitors and compare them using the same standards.
A practical sequence is:
- State the current explanation clearly.
- List at least two plausible alternatives.
- Ask what each explanation predicts if it is true.
- Look for evidence that separates them rather than merely supports them all.
- Reduce confidence when multiple explanations remain equally consistent with the available facts.
The discipline comes from evaluating every hypothesis against every important piece of evidence instead of collecting evidence only for the preferred explanation.
One structured method that embodies this idea is the Analysis of Competing Hypotheses (ACH), originally developed for intelligence analysis. Rather than asking which explanation has the most confirming evidence, it encourages analysts to examine which explanations survive attempts at disconfirmation. Although researchers debate how much ACH improves performance in practice, its central insight—that competing explanations should be evaluated explicitly rather than implicitly—remains influential. [ialeia.org+2Wiley Online Library]ialeia.organalysis of five alternative paths for making counterintelligence judgments in the…Read more…
Focus on evidence that changes your mind
Not every observation is equally valuable.
Imagine three explanations predict exactly the same outcome. Gathering more evidence of that predicted outcome adds little information.
Instead, ask:
- What observation would strongly favour one explanation over the others?
- Which prediction differs across the competing accounts?
- What evidence would force me to abandon my preferred explanation?
Evidence that would genuinely alter the ranking of explanations deserves much greater weight than evidence that merely accumulates within an already-favoured narrative.
Why confirmation alone is a weak test
People naturally search for evidence that supports what they already suspect. Psychologists describe this tendency as confirmation bias: favouring information that agrees with an existing belief while paying less attention to conflicting evidence. [Wikipedia]WikipediaConfirmation biasConfirmation bias
Testing rival explanations counteracts this tendency because it changes the question from:
“Can I find evidence that supports my idea?”
to:
“Does my idea explain the evidence better than its strongest competitor?”
This shift produces several benefits:
- hidden assumptions become visible;
- missing data become easier to identify;
- confidence becomes proportional to evidence rather than commitment;
- uncertainty is recognised instead of concealed.
Importantly, the outcome is sometimes greater uncertainty. That is not analytical failure. It is often a more accurate reflection of what the evidence actually permits. Heuer’s work on intelligence analysis argues that explicitly considering alternatives often reduces unwarranted confidence because it exposes possibilities that intuitive reasoning tends to ignore. [ialeia.org]ialeia.organalysis of five alternative paths for making counterintelligence judgments in the…Read more…
Examples from diagnoses, hiring and customer behaviour
Medical diagnosis
A patient reports chest pain.
Possible explanations include a heart attack, acid reflux, muscular strain, anxiety or lung disease.
Early symptoms may fit several diagnoses simultaneously. Good clinical reasoning therefore seeks tests that discriminate between possibilities rather than collecting more evidence consistent with all of them.
A blood marker or imaging result may sharply increase confidence in one diagnosis while reducing confidence in others. The strength of the conclusion comes not from having “lots of evidence” but from having evidence that separates competing explanations.
Hiring decisions
An interviewer concludes that a candidate performed poorly because they lack competence.
Competing explanations might include:
- unfamiliar interview format;
- temporary illness;
- language barriers;
- anxiety despite strong technical ability;
- poor questioning by the interview panel.
If only one explanation is considered, hiring decisions become vulnerable to attribution errors. Looking for additional evidence—work samples, references, structured assessments or job simulations—helps distinguish among the competing accounts instead of reinforcing an initial impression.
Customer behaviour
An online retailer notices falling repeat purchases.
Several explanations may fit:
- prices increased;
- competitors improved their offers;
- product quality declined;
- delivery performance worsened;
- customer needs changed.
Each explanation predicts different patterns.
If delivery problems are responsible, complaints and shipping delays should increase. If prices are the main driver, discount experiments should have larger effects. If competitors are attracting customers, market share data may shift even where prices remain stable.
The best investigation therefore seeks observations that discriminate among explanations rather than treating every decline in sales as proof of the first plausible story.
Common mistakes when considering alternatives
Several predictable errors weaken reasoning even when people attempt to compare explanations.
Creating weak straw-man alternatives. If competing explanations are implausible from the start, defeating them proves little. The comparison should always involve the strongest realistic rivals.
Treating absence of evidence as evidence of absence. Failure to observe something does not automatically eliminate an explanation, particularly if that explanation predicts the evidence would be difficult to detect. Intelligence analysts have long warned against confusing hidden activity with nonexistent activity. [ialeia.org]ialeia.organalysis of five alternative paths for making counterintelligence judgments in the…Read more…
Stopping after finding one plausible explanation. The first coherent story often feels convincing because it reduces uncertainty quickly. That feeling should trigger further comparison rather than end it.
Ignoring mixed causes. Rival explanations are not always mutually exclusive. Falling productivity, for example, may result from staffing shortages, outdated software and poor management simultaneously. Competing explanations should therefore be tested without assuming that exactly one must be true.
A practical habit for stronger conclusions
Before accepting an important conclusion, ask four questions:
- What is the strongest alternative explanation?
- Which facts support both explanations equally?
- Which pieces of evidence genuinely distinguish between them?
- If the alternative proved correct, which part of my reasoning would fail?
These questions transform weakest-link thinking from an exercise in finding flaws into a disciplined comparison of explanations. Conclusions become more robust not because they accumulate more supporting evidence, but because they survive serious competition from credible alternatives.
Amazon book picks
Further Reading
Books and field guides related to What Else Could Explain the Same Evidence?. Use these as the next step if you want deeper reading beyond the article.
Thinking, Fast and Slow
Explains common reasoning errors, hypothesis testing, and why people overlook alternative explanations.
The Demon-Haunted World
Rating: 4.5/5 from 43 Google Books ratings
Promotes comparing competing explanations and evaluating evidence before accepting conclusions.
Super Thinking
Introduces frameworks for weighing multiple explanations and avoiding narrow reasoning.
How to Lie with Statistics
Shows how the same evidence can support misleading interpretations without careful comparison.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: ialeia.org
Link: https://www.ialeia.org/docs/Psychology_of_Intelligence_Analysis.pdfSource snippet
analysis of five alternative paths for making counterintelligence judgments in the...Read more...
-
Source: onlinelibrary.wiley.com
Link: https://onlinelibrary.wiley.com/doi/full/10.1002/acp.3550Source snippet
Wiley Online LibraryThe “analysis of competing hypotheses” in intelligence...by MK Dhami · 2019 · Cited by 81 — We examined the use of t...
-
Source: Wikipedia
Title: Confirmation bias
Link: https://en.wikipedia.org/wiki/Confirmation_bias -
Source: Wikipedia
Title: Analysis of competing hypotheses
Link: https://en.wikipedia.org/wiki/Analysis_of_competing_hypothesesSource snippet
Analysis of competing hypothesesThe analysis of competing hypotheses (ACH) is a methodology for evaluating multiple competing hypothes...
-
Source: stat.berkeley.edu
Link: https://www.stat.berkeley.edu/~aldous/157/Papers/Tradecraft%20Primer-apr09.pdfSource snippet
Department of StatisticsStructured Analytic Techniques for Improving Intelligence...April 28, 2009 — by AT Primer · 2009 · Cited by 62 —...
Published: April 28, 2009
-
Source: sosintel.co.uk
Title: analysis of competing hypotheses
Link: https://sosintel.co.uk/tag/analysis-of-competing-hypotheses/Source snippet
Archives20 Jun 2025 — The Analysis of Competing Hypotheses (ACH) is a structured method designed to cut through ambiguity and support obj...
Additional References
-
Source: futuribles.com
Link: https://www.futuribles.com/wp-content/uploads/related-documents/analysis-of-competing-hypotheses.pdf?postId=73706Source snippet
Analysis of Competing HypothesesAnalysis of Competing Hypotheses (ACH) is an intelligence analysis method based on evaluating plausible a...
-
Source: methods.sagepub.com
Title: qualitative data analysis design
Link: https://methods.sagepub.com/book/mono/introduction-to-educational-research/chpt/qualitative-data-analysis-designSource snippet
Data, Analysis, and DesignGood qualitative research contributes to science via a logical chain of reasoning, multiple sources of convergi...
-
Source: drcharlesmrusso.substack.com
Title: causal reasoning and explanation
Link: https://drcharlesmrusso.substack.com/p/causal-reasoning-and-explanationSource snippet
Reasoning and Explanation Building as a Core...The article concludes that disciplined causal reasoning is indispensable for moving from...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=Y-J0FYOQRMYSource snippet
This CIA Manual Trains the World's Sharpest Analytical Minds...
-
Source: youtube.com
Title: Analysis of Competing Hypotheses (ACH): Finding Plausible Answers
Link: https://www.youtube.com/watch?v=xt4EnzvGA4wSource snippet
Analysis of Competing Hypotheses (ACH): A Structured Analytic Technique (SAT) for FinCrime...
-
Source: youtube.com
Title: First Conclusion Bias
Link: https://www.youtube.com/watch?v=zOaKf_1RgZESource snippet
Analysis of Competing Hypotheses (ACH): Finding Plausible Answers - YouTube Analysis of Competing Hypotheses (ACH): Finding Plausible Ans...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8763848/Source snippet
Impact of Cognitive Biases on Professionals' Decision...by V Berthet · 2022 · Cited by 306 — The author reviewed the research on the imp...
-
Source: youtube.com
Title: This CIA Manual Trains the World’s Sharpest Analytical Minds
Link: https://www.youtube.com/watch?v=NMElghTG_kISource snippet
Intelligence Analysis Skills: Analysis of Competing Hypotheses (Part 1)...
-
Source: youtube.com
Title: Intelligence Analysis Skills: Analysis of Competing Hypotheses (Part 1)
Link: https://www.youtube.com/watch?v=J_eDCBf7R2ISource snippet
First Conclusion Bias...
Topic Tree