Within Alternatives
Does This Evidence Really Pick a Side?
A fact can fit your favorite story without doing much to rule out serious rivals.
On this page
- Compatible evidence versus selective evidence
- Why vivid facts can be non diagnostic
- How to ask what would separate the explanations
Page outline Jump by section
Introduction
A piece of evidence does not become strong simply because it matches your preferred explanation. The key question is whether it matches that explanation better than the serious alternatives. If the same observation would be expected under several competing explanations, it provides little help in deciding between them. Such evidence is often described as non-diagnostic: it is compatible with multiple hypotheses and therefore does little to distinguish which one is most likely.[Internet Encyclopedia of Philosophy]iep.utm.eduInternet Encyclopedia of PhilosophyEvidenceEvidence is taken to be what confirms or refutes scientific theories, and thereby constitutes…
This distinction is central to good reasoning. Many thinking errors arise because people ask only, “Does this fit my idea?” rather than, “Would I have expected this even if my idea were wrong?” The second question shifts attention from evidence that merely supports a story to evidence that actually discriminates between competing explanations.
Compatible Evidence Versus Selective Evidence
A useful way to think about evidence is to separate compatibility from selectivity.
Compatible evidence is simply evidence that does not contradict an explanation. Many different explanations can be compatible with the same fact.
Selective evidence is evidence that one explanation predicts substantially better than its rivals. It changes the balance between explanations because it would be noticeably less expected if another explanation were true. Philosophers of science and epistemologists emphasise that evidence earns its value by helping rational observers decide between competing accounts of the world, not merely by fitting one of them.[Internet Encyclopedia of Philosophy]iep.utm.eduInternet Encyclopedia of PhilosophyEvidenceEvidence is taken to be what confirms or refutes scientific theories, and thereby constitutes…
Consider a simple example:
- Observation: “The garden is wet.”
- Explanation A: It rained overnight.
- Explanation B: The sprinkler system ran.
- Explanation C: Someone washed the driveway.
A wet garden is compatible with all three explanations. Until additional evidence appears, the observation itself does not pick a side.
Now add another observation:
- The weather station recorded no rain.
- Neighbours report hearing sprinklers at 6 a.m.
These new facts are more selective because they favour one explanation over the others.
The important lesson is that evidence should be judged comparatively, not in isolation.
Why Vivid Facts Can Be Non-Diagnostic
People naturally give extra weight to evidence that is memorable, emotional or unusually detailed. Unfortunately, vividness and diagnostic value are different qualities.
Suppose someone believes that a colleague is deliberately avoiding them. They notice:
- delayed email replies,
- brief conversations,
- cancelled meetings.
Each observation feels persuasive because it fits the story. Yet each is equally compatible with many ordinary alternatives such as workload, illness, travel or competing priorities.
The vividness of the observations can create an illusion of strong support even though none of them distinguishes among the rival explanations.
Psychological research on confirmation bias notes that people frequently evaluate only the probability of seeing evidence if their preferred hypothesis were true, while neglecting the equally important question of how likely the same evidence would be if competing hypotheses were true. This tendency has been called pseudodiagnosticity because apparently supportive evidence turns out not to discriminate between alternatives.[UC San Diego Pages]pages.ucsd.edunickerson Confirmation BiasUC San Diego PagesConfirmation Bias: A Ubiquitous Phenomenon in Many…by RS Nickerson · 1998 · Cited by 12455 — An observation that is…
The Mechanism: Why Compatible Evidence Feels More Convincing Than It Is
The mistake usually follows a predictable sequence.
- A preferred explanation comes to mind.
- Evidence consistent with that explanation is noticed.
- The mind treats consistency as confirmation.
- Alternative explanations receive little attention.
- Confidence increases despite little change in the actual balance of evidence.
The critical missing comparison is:
How surprising would this observation be if another explanation were correct?
If the answer is “not very surprising”, then the evidence contributes little.
This is closely related to the idea of likelihood in scientific and statistical reasoning. Evidence is informative when it is substantially more probable under one explanation than another. When the probabilities are similar across explanations, the evidence has little discriminating power.[UC San Diego Pages+2TheNNT]pages.ucsd.edunickerson Confirmation BiasUC San Diego PagesConfirmation Bias: A Ubiquitous Phenomenon in Many…by RS Nickerson · 1998 · Cited by 12455 — An observation that is…
How to Ask What Would Separate the Explanations
A practical way to avoid over-interpreting compatible evidence is to ask what observation would genuinely distinguish the competing explanations.
Instead of asking:
> What evidence supports my explanation?[core.ac.uk]core.ac.ukvidence supports—promises epistemological riches; it has struck many philosophers as…Read more…
Ask:
What evidence would look different if another explanation were true?
Examples include:
Competing explanationsWeak evidenceMore selective evidenceWebsite sales fell because demand dropped vs tracking software failedSales dashboard shows lower revenuePayment records, server logs and independent transaction countsFriend is angry vs simply busyNo reply to a messageEvidence they are actively responding to others while consistently ignoring you over timeMachine has a faulty motor vs low batteryDevice stopped workingVoltage tests, motor diagnostics or replacement battery results
Notice that the stronger evidence is not necessarily more dramatic. It is simply more capable of separating one explanation from another.
Scientific Thinking Relies on Discriminating Evidence
Scientific investigation deliberately seeks observations that competing hypotheses predict differently.
The classic method of multiple working hypotheses, proposed by T. C. Chamberlin, encourages researchers to develop several plausible explanations before collecting evidence. The goal is to avoid becoming attached to one “ruling theory” and instead search for observations capable of distinguishing among alternatives.[Woods Hole Oceanographic Institution+2railsback.org]whoi.eduWoods Hole Oceanographic Institution The Method of Multiple Working HypothesesWoods Hole Oceanographic InstitutionThe Method of Multiple Working HypothesesJanuary 23, 2002 — by TC Chamberlin · Cited by 856 — The wor…
This approach changes the purpose of evidence collection.
Instead of asking:
Can I find facts consistent with my hypothesis?
Researchers ask:
Which experiment would produce different outcomes depending on which explanation is true?
That shift dramatically improves the quality of inference because it prioritises diagnostic evidence over merely compatible evidence.
Everyday Warning Signs That Evidence Is Weak
Several patterns suggest that apparently persuasive evidence may not actually distinguish explanations:
- The evidence could easily occur under several reasonable explanations.
- Only one explanation has been examined in detail.
- The supporting facts are all individually unsurprising.
- No one has identified what evidence would count against the preferred explanation.
- Confidence grows faster than the evidence becomes more selective.
None of these guarantees that the preferred explanation is wrong. They simply indicate that the current evidence may not justify choosing it over serious rivals.
A Useful Habit of Mind
Whenever a fact seems to support a conclusion, pause before asking whether it is convincing.
Instead, ask three questions:
- Which alternative explanations also predict this observation?
- Would I expect this evidence almost as often if one of those alternatives were true?
- What observation would genuinely separate these explanations?
These questions transform the evaluation of evidence from simple story-matching into comparative reasoning. The goal is not to reject evidence that fits your preferred explanation, but to recognise that compatibility alone is a low bar. Strong reasoning depends on finding evidence that does more than fit a story—it must help decide which story is most likely.
Amazon book picks
Further Reading
Books and field guides related to Does This Evidence Really Pick a Side?. Use these as the next step if you want deeper reading beyond the article.
The Art of Statistics
Shows how to evaluate evidence by comparing competing explanations rather than relying on vivid examples.
The Demon-Haunted World
Rating: 4.5/5 from 43 Google Books ratings
Promotes asking what evidence would distinguish between rival hypotheses.
How to Think About Weird Things: Critical Thinking for a New Age
Covers hypothesis testing, evidence quality, and distinguishing explanatory alternatives.
Calling Bullshit
Encourages evaluating whether evidence truly discriminates between competing claims instead of merely fitting a preferred narrative.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
1.
Source: thennt.com
Link:https://thennt.com/diagnostics-and-likelihood-ratios-explained/
Source snippet
Diagnostics and Likelihood Ratios, ExplainedLikelihood ratios (LR) are used to assess two things: 1) the potential utility of a particula...
2.
Source: railsback.org
Link:https://railsback.org/railsback_chamberlin.html
Source snippet
TC Chamberlin's "Method of Multiple Working Hypotheses"The method of multiple working hypotheses involves the development, prior to our r...
3.
Source: iep.utm.edu
Link:https://iep.utm.edu/evidence/
Source snippet
Internet Encyclopedia of PhilosophyEvidenceEvidence is taken to be what confirms or refutes scientific theories, and thereby constitutes...
4.
Source: pages.ucsd.edu
Title: nickerson Confirmation Bias
Link:https://pages.ucsd.edu/~mckenzie/nickersonConfirmationBias.pdf
Source snippet
UC San Diego PagesConfirmation Bias: A Ubiquitous Phenomenon in Many...by RS Nickerson · 1998 · Cited by 12455 — An observation that is...
5.
Source: whoi.edu
Title: Woods Hole Oceanographic Institution The Method of Multiple Working Hypotheses
Link:https://www.whoi.edu/cms/files/chamberlin65sci_72744.pdf
Source snippet
Woods Hole Oceanographic InstitutionThe Method of Multiple Working HypothesesJanuary 23, 2002 — by TC Chamberlin · Cited by 856 — The wor...
Published: January 23, 2002
6.
Source: plato.stanford.edu
Link:https://plato.stanford.edu/entries/evidence/
Source snippet
Stanford Encyclopedia of Philosophy11 Aug 2006 — Evidence is by its very nature the kind of thing that can generate rational convergence...
7.
Source: Wikipedia
Title: Confirmation bias
Link:https://en.wikipedia.org/wiki/Confirmation_bias
Source snippet
Confirmation biasConfirmation bias is the tendency to search for, interpret, favor and [recall]({{ 'recall/' | relative_url }}) information in a way that confirms or su...
8.
Source: ndl.ethernet.edu.et
Link:https://ndl.ethernet.edu.et/bitstream/123456789/29276/1/303.pdf
Source snippet
ethernet.edu.etBelief, Evidence, and Uncertainty Problems of Epistemic...by PS Bandyopadhyay · Cited by 48 — That the standard error-sta...
Additional References
9.
Source: catalogofbias.org
Link:https://catalogofbias.org/biases/confirmation-bias/
Source snippet
Confirmation biasConfirmation bias occurs when an individual looks for and uses the information to support their own ideas or beliefs. in...
10.
Source: royalsocietypublishing.org
Link:https://royalsocietypublishing.org/rsos/article/4/1/160756/36818/Why-are-we-not-evaluating-multiple-competing
Source snippet
Why are we not evaluating multiple competing hypotheses in...by GS Betini · Cited by 66 — The use of multiple working hypotheses to gain...
11.
Source: researchgate.net
Link:https://www.researchgate.net/publication/3412783_Confirmation_Bias_in_Complex_Analyses
12.
Source: researchgate.net
Link:https://www.researchgate.net/publication/232664354_Revisiting_Chamberlin_Multiple_Working_Hypotheses_for_the_21st_Century
13.
Source: youtube.com
Link:https://www.youtube.com/watch?v=Y-J0FYOQRMY
Source snippet
Analysis of Competing Hypotheses (ACH): Finding Plausible Answers explains how structured methodologies isolate truly diagnostic evidence...
14.
Source: core.ac.uk
Link:https://core.ac.uk/download/pdf/187116074.pdf
Source snippet
vidence supports—promises epistemological riches; it has struck many philosophers as...Read more...
15.
Source: reddit.com
Link:https://www.reddit.com/r/askphilosophy/comments/1u8b2cz/epistemology_what_constitutes_evidence_and_how_do/
Source snippet
compatible with multiple, mutually exclusive hypotheses, what...Read more...
16.
Source: hackscience.education
Title: on multiple working hypotheses
Link:https://www.hackscience.education/on-multiple-working-hypotheses/
Source snippet
15 Dec 2025 — By recognizing multiple hypotheses, Chamberlin suggests “the outcome, therefore, is better and truer observations and more...
17.
Source: undsci.berkeley.edu
Title: science at multiple levels
Link:https://undsci.berkeley.edu/understanding-science-101/how-science-works/science-at-multiple-levels/
Source snippet
at multiple levelsMisconception: If evidence supports a hypothesis, it is upgraded to a theory. If the theory then garners even more supp...
18.
Source: psychology.town
Title: Cognitive biases are not
Link:https://psychology.town/psychodiagnostics/overcoming-confirmation-bias-psychological-assessment/
Source snippet
Overcoming Confirmation Bias in Psychological Assessment27 Sept 2025 — A diagnosis shaped by confirmation bias rather than evidence can r...
Topic Tree