Within Problem Parts
What evidence would actually change your mind?
The best evidence is not what sounds supportive, but what helps distinguish one explanation from another.
On this page
- Supportive evidence versus diagnostic evidence
- Examples from retention, sales, and team performance
- Designing questions that separate rival causes
Page outline Jump by section
Introduction
Analytical thinking improves when you stop asking, “What evidence supports my current explanation?” and instead ask, “What evidence would distinguish this explanation from its competitors?” That distinction lies at the heart of diagnostic evidence. Supportive evidence may make a favourite explanation seem more plausible, but diagnostic evidence changes your judgement because it fits one explanation substantially better than the alternatives.
This shift is one of the most effective ways to avoid confirmation bias. Rather than collecting facts that all point in roughly the same direction, analytical thinkers actively seek information that could separate rival hypotheses. Whether you are explaining falling customer retention, disappointing sales, or uneven team performance, the most valuable question is not what sounds convincing, but what finding would genuinely change your mind. [pages.ucsd.edu]pages.ucsd.eduConfirmation Bias: A Ubiquitous Phenomenon in ManyOctober 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation bias, as the term is typically used in the psychological literat…
Supportive evidence versus diagnostic evidence
Many pieces of evidence are compatible with several explanations at once. They may be true, yet contribute little to deciding between competing causes.
For example, suppose customer retention has declined.
- Customers report lower satisfaction.
- Complaint volumes have increased.
- Renewal rates have fallen.
These observations support the conclusion that “something is wrong”, but they do not distinguish whether the underlying cause is product quality, pricing, customer support, stronger competitors, or changes in customer expectations.
Diagnostic evidence looks different. It changes the balance between explanations.
Imagine three competing hypotheses:
HypothesisDiagnostic evidence that would strongly favour itProduct quality has declinedComplaints are concentrated around a recent software release, while customers on older versions remain stable.Price is the problemChurn rises immediately after price increases but remains unchanged among customers on grandfathered pricing.Competitors are attracting customersChurn is highest where a new competitor has entered, despite unchanged product quality and pricing.
Each observation would reduce confidence in some explanations while increasing confidence in another. That is what makes it diagnostic.
The important question therefore becomes:
Which observation would make me abandon my preferred explanation?
If the answer is “none”, the analysis has probably become advocacy rather than investigation. [pages.ucsd.edu]pages.ucsd.eduConfirmation Bias: A Ubiquitous Phenomenon in ManyOctober 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation bias, as the term is typically used in the psychological literat…
Why persuasive evidence often misleads
People naturally seek information that confirms what they already suspect. Psychologists refer to this tendency as confirmation bias. Instead of comparing competing explanations, people often build increasingly elaborate cases for the first explanation that seems reasonable. [pages.ucsd.edu]pages.ucsd.eduConfirmation Bias: A Ubiquitous Phenomenon in ManyOctober 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation bias, as the term is typically used in the psychological literat…
This happens because supportive evidence feels informative even when it is not.
Suppose a sales manager believes poor training explains declining sales. Evidence such as:
- lower confidence among new staff,
- requests for additional coaching,
- weaker product knowledge,
all appear persuasive.
However, every one of those findings might also occur if:
- sales territories were redistributed poorly,
- market demand weakened,
- pricing became less competitive,
- experienced staff left the company.
The evidence strengthens the preferred explanation emotionally without separating it analytically from rival explanations.
Diagnostic evidence would instead ask questions such as:
- Do experienced staff in the same territories also show declining sales?
- Did sales fall equally across products requiring very different levels of expertise?
- Did comparable competitors experience similar declines?
Those answers discriminate among explanations rather than merely reinforcing one.
Examples that genuinely change the answer
Customer retention
A company notices subscription renewals have fallen.
The first explanation is that customers dislike a recent redesign.
Supportive evidence includes negative online comments and increased complaints after launch.
[Diagnostic evidence goes further.]monashhealth.orgCognitive Bias Scoping Review 2019 FINALBest practice to identify and prevent cognitive bias in…by M Garrubba · 2019 · Cited by 7 — Their findings suggest that cognitive bias…
If cancellations occur almost exclusively among customers who actively use the redesigned features, the redesign becomes a much stronger explanation.
If cancellations instead occur mainly among customers who never adopted the redesign, another cause becomes more likely.
The same complaints now carry very different weight because they are interpreted alongside evidence capable of separating alternatives.
Sales performance
Suppose quarterly sales decline by 15%.
Possible explanations include:
- weaker demand,
- ineffective sales staff,
- inventory shortages,
- increased competition.
Average sales figures alone support all four possibilities.
More diagnostic questions include:
- Did competitors also experience similar declines?
- Did regions with full inventory outperform regions experiencing shortages?
- Did experienced and inexperienced representatives decline equally?
- Were declines concentrated in one product category?
Each answer eliminates possibilities rather than simply accumulating more evidence.
Team performance
A software team suddenly misses deadlines.
One manager believes motivation has fallen.
Another believes technical complexity has increased.
Employee surveys reporting higher stress support both explanations.
More diagnostic evidence would include:
- whether delays are concentrated in technically novel projects,
- whether experienced engineers are affected as much as junior staff,
- whether productivity recovered after architectural problems were resolved,
- whether projects with unchanged technical demands also slowed.
Those observations discriminate between workload, capability, morale and process problems instead of treating all signs of stress as equivalent.
Designing questions that separate rival causes
One practical way to generate diagnostic evidence is to write competing explanations before gathering more information.
Instead of asking:
“How can I prove my explanation?”
ask:
“What result would look different if each explanation were true?”
This leads naturally to stronger investigative questions.
For example:
Weak questionDiagnostic questionAre customers unhappy?Which customer groups are unhappy, and does that pattern uniquely fit one explanation?Are employees stressed?Which teams are stressed, and does the distribution distinguish workload from poor leadership?Is quality declining?Did quality change before or after the suspected cause appeared?Are competitors improving?Are losses concentrated where competitors made changes or across the entire market?
The goal is to identify observations that would not be equally expected under every hypothesis.
Look for evidence that could prove you wrong
Scientific reasoning places special value on evidence capable of falsifying a hypothesis rather than endlessly confirming it. While everyday decisions rarely allow perfectly controlled experiments, the same principle improves practical thinking.
Before collecting more information, ask:
- What finding would clearly weaken my current explanation?
- Which competing explanation makes a different prediction?
- What evidence would surprise me?
- Which observation would force me to revise my conclusion?
These questions encourage active comparison rather than passive accumulation of confirming facts.
Medical decision-making illustrates this principle well. Research on diagnostic reasoning shows that clinicians become more accurate when they deliberately consider alternative diagnoses and continue searching even after finding an apparently satisfactory explanation. Structured reflection and “cognitive forcing” strategies help reduce premature closure—the tendency to stop investigating once an early answer seems plausible. American Medical Association+2Royal College of Physicians of Edinburgh [ama-assn.org]ama-assn.orgLearn with the AMA about strategies to neutralize themAmerican Medical Association4 widespread cognitive biases and how doctors can…4 Feb 2021 — Cognitive biases can cause diagnostic and t…
Practical habits for finding diagnostic evidence
Analytical thinkers can make their evidence more diagnostic by adopting a few consistent habits.
- List competing explanations first. Evidence is only diagnostic relative to alternatives.
- Predict what each explanation would uniquely imply. Different causes should produce different observable patterns.
- Prefer comparisons over isolated facts. Differences between groups, places or time periods often reveal more than overall averages.
- Actively search for disconfirming evidence. A finding that challenges your current belief is often more informative than another supporting example.
- Revise confidence gradually. One diagnostic observation may substantially change your judgement, but rarely settles complex problems on its own.
These habits shift analysis away from collecting persuasive stories and towards gathering information that genuinely distinguishes among competing explanations.
The key takeaway
The quality of analytical thinking depends less on how much evidence you collect than on how informative that evidence is. Supportive evidence can make an explanation feel increasingly convincing while leaving competing explanations equally plausible. Diagnostic evidence earns its value by changing the balance between those alternatives.
Whenever facing an important judgement, replace the question “What supports my view?” with “What observation would clearly favour one explanation over the others?” That simple change transforms evidence gathering from confirmation into genuine analysis.
Amazon book picks
Further Reading
Books and field guides related to What evidence would actually change your mind?. Use these as the next step if you want deeper reading beyond the article.
Superforecasting
Emphasizes updating beliefs, testing rival hypotheses, and identifying evidence that should change forecasts.
Thinking, Fast and Slow
Directly supports the page’s focus on avoiding confirmation bias and improving analytical judgement.
The Scout Mindset
Closely matches the idea of seeking truth rather than defending a preferred explanation.
Decisive
Gives practical methods for widening options, reality-testing assumptions, and improving business decisions.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: pages.ucsd.edu
Title: Confirmation Bias: A Ubiquitous Phenomenon in Many
Link: https://pages.ucsd.edu/~mckenzie/nickersonConfirmationBias.pdfSource snippet
October 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation bias, as the term is typically used in the psychological literat...
Published: October 6, 2004
-
Source: ama-assn.org
Title: Learn with the AMA about strategies to neutralize them
Link: https://www.ama-assn.org/about/ethics/4-widespread-cognitive-biases-and-how-doctors-can-overcome-themSource snippet
American Medical Association4 widespread cognitive biases and how doctors can...4 Feb 2021 — Cognitive biases can cause diagnostic and t...
-
Source: rcpe.ac.uk
Title: jrcpe 48 3 osullivan
Link: https://www.rcpe.ac.uk/sites/default/files/jrcpe_48_3_osullivan.pdfSource snippet
Royal College of Physicians of EdinburghCognitive bias in clinical medicineby ED O’Sullivan · 2018 · Cited by 474 — This paper describes...
-
Source: Wikipedia
Title: Confirmation bias
Link: https://en.wikipedia.org/wiki/Confirmation_biasSource snippet
Confirmation biasConfirmation bias is the tendency to search for, interpret, favor and recall information in a way that confirms or su...
Additional References
-
Source: researchgate.net
Title: 366323093 Influences of early diagnostic suggestions on clinical reasoning
Link: https://www.researchgate.net/publication/366323093_Influences_of_early_diagnostic_suggestions_on_clinical_reasoningSource snippet
(PDF) Influences of early diagnostic suggestions on clinical...22 May 2026 — Previous research has highlighted the importance of physici...
Published: May 2026
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=Mg0iUm3ralwSource snippet
This CIA Manual Trains the World's Sharpest Analytical Minds explains structured analytic techniques developed to identify diagnostic evi...
-
Source: codex.ucsf.edu
Title: primer 3 role clinical reasoning diagnostic excellence
Link: https://codex.ucsf.edu/primer-3-role-clinical-reasoning-diagnostic-excellenceSource snippet
Primer 3: The Role of Clinical Reasoning in Diagnostic...25 Jun 2025 — This primer focuses on the role of cognition in diagnostic excell...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=Y-J0FYOQRMYSource snippet
Applying structured analytic techniques to a cyber problem: Lori Cole & Liz Fulp, Financial Firm...
-
Source: monashhealth.org
Title: Cognitive Bias Scoping Review 2019 FINAL
Link: https://monashhealth.org/wp-content/uploads/2020/03/Cognitive-Bias_Scoping-Review_2019_FINAL.pdfSource snippet
Best practice to identify and prevent cognitive bias in...by M Garrubba · 2019 · Cited by 7 — Their findings suggest that cognitive bias...
-
Source: youtube.com
Title: Analysis of Competing Hypotheses (ACH): Finding Plausible Answers
Link: https://www.youtube.com/watch?v=xt4EnzvGA4wSource snippet
Analysis of Competing Hypotheses (ACH): A Structured Analytic Technique (SAT) for FinCrime...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8520040/Source snippet
biases in diagnosis and decision making during...by CS Webster · 2021 · Cited by 85 — Cognitive bias is typically defined as flaws or di...
-
Source: youtube.com
Title: Diagnostic Tests & Studies: How to Use & Evaluate in Practice
Link: https://www.youtube.com/watch?v=ra_eS_Vn4csSource snippet
Analysis of Competing Hypotheses (ACH): Finding Plausible Answers...
-
Source: youtube.com
Title: This CIA Manual Trains the World’s Sharpest Analytical Minds
Link: https://www.youtube.com/watch?v=NMElghTG_kISource snippet
Diagnostic Tests & Studies: How to Use & Evaluate in Practice...
-
Source: themedicon.com
Title: MCMS 07 242
Link: https://themedicon.com/pdf/medicalsciences/MCMS-07-242.pdfSource snippet
Confirmation Bias Effects on Healthcare & Patientsby S Nandennagari · 2024 · Cited by 2 — Confirmation bias is the tendency to search for...
Topic Tree