Within Better Questions
What Are You Really Comparing?
Evidence only becomes useful when the real alternatives are visible, whether the choice is a job, policy, habit, or tool.
On this page
- Why isolated pros and cons mislead
- How to identify the live alternatives
- Comparison traps in everyday decisions
Page outline Jump by section
Introduction
Good thinking begins before you search for evidence. One of the most common mistakes is asking whether something is “good”, “worth it”, or “effective” without first deciding what it is supposed to be better than. Evidence only becomes meaningful against a comparison. A study showing that a new method improves performance tells you little if the real alternative is another method that performs even better, costs less, or carries fewer risks. Defining the comparison before searching prevents wasted effort, reduces misleading conclusions, and makes it much easier to recognise genuinely relevant evidence. This is why evidence-based research frameworks such as PICO explicitly require a comparator rather than treating it as an optional detail. [Cochrane+2Innovation Service]cochrane.orgChapter 2: Determining the scope of the review and…The review PICO defines the broad scope of the review, and the PICO for com…
What Are You Really Comparing?
Many everyday questions hide an unstated comparison.
- “Is this job good?” compared with staying in your current role, accepting another offer, or continuing your search?
- “Should we adopt this software?” compared with keeping the existing system, buying a competitor’s product, or changing the workflow without new software?
- “Is daily exercise worthwhile?” compared with complete inactivity, occasional exercise, or spending the same time sleeping more?
These questions appear simple but actually contain several different decision problems. Searching before identifying the live alternative often produces evidence that answers a different question from the one you actually face.
This is why systematic review methods insist on defining the comparator before searching the literature. The comparison determines which evidence is relevant and which studies should be excluded. Without that discipline, researchers risk combining findings that answer different questions entirely. [Cochrane+2Innovation Service]cochrane.orgChapter 2: Determining the scope of the review and…The review PICO defines the broad scope of the review, and the PICO for com…
Why Isolated Pros and Cons Mislead
Lists of advantages and disadvantages feel balanced, but they often disguise the absence of a meaningful comparison.
For example, suppose someone asks whether working remotely improves productivity. A list of benefits might include flexibility, reduced commuting and higher job satisfaction. A list of disadvantages might mention weaker informal communication and fewer mentoring opportunities.
Neither list answers the practical decision.
The real question might be:
- fully remote versus hybrid working;
- remote work for experienced staff versus new recruits;
- remote work over six months versus five years;
- remote work measured by output, retention, wellbeing or promotion.
Each comparison may produce a different answer.
The same problem appears in product reviews, political debates and health advice. A claim that one option has disadvantages tells you almost nothing unless you know whether the alternatives perform better or worse on the outcomes that matter.
In decision-making, the comparison—not merely the evidence—defines what counts as success.
How to Identify the Live Alternatives
A useful comparison is not every imaginable possibility. It is the realistic choice you could actually make.
A practical way to uncover the real comparison is to ask:
- What happens if I do nothing? Doing nothing is often the default option, yet it is frequently omitted from discussions.
- What is my most realistic alternative? Compare against the option you would genuinely choose if the preferred option disappeared.
- Would another decision-maker choose a different baseline? Different stakeholders often assume different comparisons without noticing.
- Which options remain feasible after considering cost, time and constraints? Impossible alternatives should not dominate the analysis.
These questions narrow attention to decisions that are genuinely available rather than hypothetical ideals.
Evidence-based search frameworks formalise this idea. In the PICO structure, the comparator is intended to represent current practice, another intervention or another realistic option, ensuring that searches retrieve studies capable of answering the actual decision rather than a loosely related one. [Innovation Service+2NCCMT]innovation.nhs.ukInnovation ServiceThe population, intervention, comparator and outcomes…The population, intervention, comparator and outcomes (PICO) f…
Comparison Traps in Everyday Decisions
Several recurring mistakes distort thinking before the search even begins.
Comparing against an unrealistic ideal. A new public policy may be criticised because it is imperfect rather than compared with the practical alternatives that governments could actually implement.
Comparing with the past instead of the available future. A person considering a career move may compare the new role with the best moments of their current job rather than with its likely future.
Comparing different outcomes. One option may be judged on cost while another is judged on convenience or prestige, making the comparison inconsistent.
Changing the comparison mid-discussion. A conversation may begin by asking whether one option is better than another but gradually shift to whether it has any flaws at all. Once the comparison changes, the evidence no longer addresses the same question.
Ignoring the status quo. People often evaluate proposed changes without asking whether the existing approach already performs adequately. The status quo may not be optimal, but it remains the real benchmark unless another option is genuinely available.
These errors often create the illusion of disagreement when people are actually comparing different alternatives.
Why the Comparator Shapes the Evidence You Find
Search results are not neutral collections of facts. They are filtered by the question you ask.
Suppose you search for:
“Does online learning work?”
You will retrieve a broad mixture of studies comparing online learning with classroom teaching, blended learning, correspondence courses, self-study and many other educational settings.
If instead you ask:
“For experienced software engineers, does structured online training improve certification pass rates compared with self-directed study over six months?”
The search becomes dramatically more focused because the comparator has been specified.
Research on question formulation consistently shows that structured questions improve the relevance and precision of evidence retrieval by making the search strategy more specific. The comparator is a central part of that improvement because it limits the search to evidence capable of distinguishing between genuine alternatives. [PMC+2NCCMT]pmc.ncbi.nlm.nih.govThe impact of patient, intervention, comparison, outcome…by MB Eriksen · 2018 · Cited by 1699 — This review aimed to determine if t…
Choosing Comparisons Without Becoming Rigid
Selecting the right comparison does not mean assuming there is only one valid perspective.
Different decisions legitimately require different comparators.
A business deciding whether to automate a process may compare automation with current staffing levels. An employee may compare automation with retraining opportunities. A regulator may compare it with legal compliance requirements. Each comparison answers a different but valid question.
The key is to make the comparison explicit rather than leaving it hidden.
At the same time, structured frameworks such as PICO should be treated as thinking aids rather than rigid rules. Reviews of its use in evidence-based practice note that while the comparator is valuable for clarifying questions and improving searches, no single framework captures every type of inquiry equally well. Critical judgement remains necessary when defining meaningful alternatives. [ResearchGate+2jmla.pitt.edu]researchgate.netResearch Gate(PDF) PICO: What it is and what it is notResearchGate(PDF) PICO: What it is and what it is notSeptember 1, 2021 — Aim To assess the role and effectiveness of the mnemonic PICO (P…
A Simple Test Before You Search
Before looking for evidence, complete this sentence:
“I want to know whether A is better than B because I care about C.”
If you cannot clearly identify B, you are probably searching for answers to the wrong question.
Making the comparison visible transforms vague curiosity into a decision that evidence can genuinely inform. Instead of collecting isolated facts, you begin evaluating alternatives—a much stronger foundation for careful thinking and better judgement.
Amazon book picks
Further Reading
Books and field guides related to What Are You Really Comparing?. Use these as the next step if you want deeper reading beyond the article.
Decisive
Focuses on expanding options and avoiding narrow comparisons in decision-making.
Thinking, Fast and Slow
Explains common comparison errors, cognitive biases, and better decision processes.
Super Thinking
Helps readers frame decisions by identifying appropriate alternatives and comparisons.
Smart Choices
Provides structured methods for defining alternatives before evaluating evidence.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: cochrane.org
Link: https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-02Source snippet
Chapter 2: Determining the scope of the review and...The review PICO defines the broad scope of the review, and the PICO for com...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC6148624/Source snippet
The impact of patient, intervention, comparison, outcome...by MB Eriksen · 2018 · Cited by 1699 — This review aimed to determine if t...
-
Source: nccmt.ca
Link: https://www.nccmt.ca/resources/search/138Source snippet
Defining your question: PICO and PS | Resource DetailsThe PICO (Population, Intervention, Comparison, Outcome) strategy helps users devel...
-
Source: jmla.pitt.edu
Link: https://jmla.pitt.edu/ojs/jmla/article/view/739/1069Source snippet
A comparison of patient, intervention, comparison, outcome...by LA Kloda · 2020 · Cited by 95 — In educating students in the health prof...
-
Source: researchgate.net
Title: Research Gate(PDF) PICO: What it is and what it is not
Link: https://www.researchgate.net/publication/354318815_PICO_What_it_is_and_what_it_is_notSource snippet
ResearchGate(PDF) PICO: What it is and what it is notSeptember 1, 2021 — Aim To assess the role and effectiveness of the mnemonic PICO (P...
Published: September 1, 2021
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=QPXXTE8N4PYSource snippet
4 What is Problem [Framing]({{ 'framing/' | relative_url }})? The Key to Strategic Decisions...
-
Source: innovation.nhs.uk
Link: https://innovation.nhs.uk/innovation-guides/evidence/population-intervention-comparator-and-outcomes-framework/Source snippet
Innovation ServiceThe population, intervention, comparator and outcomes...The population, intervention, comparator and outcomes (PICO) f...
Additional References
-
Source: pubrica.com
Link: https://pubrica.com/insights/study-guide/guide-to-evidence-based-research-on-pico-framework/Source snippet
PICO Framework: A Guide to Evidence-based ResearchDesigned for simple clinical questions, to be specific, define your population, interve...
-
Source: casp-uk.net
Title: pico framework
Link: https://casp-uk.net/pico-framework/Source snippet
How to use the PICO Framework to Aid Critical Appraisal7 Jun 2023 — Discover what the PICO framework is, the benefits of using PICO tools...
-
Source: youtube.com
Title: What is Problem Framing? The Key to Strategic Decisions
Link: https://www.youtube.com/watch?v=CX3HeinRBtYSource snippet
5 Lecture 3.4 Creating and Comparing Alternatives...
-
Source: frontiersin.org
Link: https://www.frontiersin.org/journals/digital-health/articles/10.3389/fdgth.2026.1755598/fullSource snippet
PICO-based [assessment]({{ 'assessment/' | relative_url }}) and categorization of evidence...by U Buddrus · 2026 · Cited by 3 — With the development of the PICO-based Assessm...
-
Source: youtube.com
Title: Lecture 3.4 Creating and Comparing Alternatives
Link: https://www.youtube.com/watch?v=S2-Xp5M_VJA -
Source: youtube.com
Link: https://www.youtube.com/watch?v=BtPL1MWVqecSource snippet
2 How to Ask [Better Questions]({{ 'better-questions/' | relative_url }}): David Hackworth at TEDxOslo...
Topic Tree