Within Evidence Tests
How to Stop Moving the Goalposts
A written change-my-mind test makes it harder to reinterpret every result after your preferences are already engaged.
On this page
- Why evidence standards work best before searching
- Writing a one sentence change my mind test
- Reviewing results without rewriting the threshold
Page outline Jump by section
Introduction
A precommitment test is a rule you write before you look at evidence, stating what would make you change your mind. Its purpose is simple: stop yourself from quietly raising the standard for unwelcome evidence or lowering it for evidence you already wanted to believe. This habit addresses one of the most persistent problems in reasoning—moving the goalposts after the outcome is known. Research on confirmation bias shows that people often search for, interpret and remember information in ways that support existing beliefs, frequently without realising they are doing so. [UCSD Pages+2PMC]pages.ucsd.eduPages Confirmation Bias: A Ubiquitous Phenomenon in ManyUCSD PagesConfirmation Bias: A Ubiquitous Phenomenon in Many…October 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation…
Within a broader approach to evidence standards, a precommitment test acts as a personal contract. It does not guarantee perfect judgement, but it makes inconsistency visible. If your original standard was reasonable, the burden shifts from explaining away each new piece of evidence to asking one straightforward question: did the evidence meet the threshold I set in advance?
Why evidence standards work best before searching
The best time to decide what counts as convincing evidence is before you know what the evidence says. Once you have started reading articles, watching videos or debating with others, your preferences become engaged. At that point, it becomes much easier to reinterpret every result in a favourable direction.
Psychologists distinguish between genuinely evaluating evidence and building a case for a conclusion that has already become attractive. Confirmation bias is often subtle rather than deliberate: people may sincerely believe they are being objective while unconsciously applying stricter standards to unwelcome findings than to supportive ones. [UCSD Pages]pages.ucsd.eduPages Confirmation Bias: A Ubiquitous Phenomenon in ManyUCSD PagesConfirmation Bias: A Ubiquitous Phenomenon in Many…October 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation…
Scientific preregistration illustrates the same principle on a larger scale. Researchers record their hypotheses, methods and planned analyses before collecting data so that later readers can distinguish predictions made in advance from explanations invented afterwards. The value is not that every plan is perfect, but that departures from the original plan become transparent rather than invisible. [PMC+2Wharton Faculty Platform]pmc.ncbi.nlm.nih.govPMCThe preregistration revolutionNIHby BA Nosek · 2018 · Cited by 2585 — Preregistration is a solution that helps researchers maintain clarity between prediction an…
For everyday decisions, you do not need a formal protocol. A brief written commitment often provides enough friction to prevent your standards from shifting unnoticed.
Writing a one-sentence change-my-mind test
A useful precommitment test is short enough that you will actually write it before searching for evidence.
A practical template is:
“I currently believe X. I will reduce or reverse my confidence if I find Y.”
The important feature is specificity. Replace vague phrases such as “good evidence” or “convincing proof” with observable conditions.
For example:
- Buying a product: “If three independent long-term reviews consistently report the same reliability problem, I will choose a different model.”
- Health decision: “If high-quality clinical guidelines conclude the treatment has no meaningful benefit for my situation, I will not use it.”
- Work decision: “If the pilot project fails to achieve the agreed performance target after four weeks, I will abandon the proposal.”
- Political claim: “If multiple independent fact-checks supported by primary documents contradict this claim, I will lower my confidence.”
Notice that each statement identifies both the evidence and the action that follows. Without the second part, people often acknowledge inconvenient evidence while changing nothing.
Another useful refinement is to specify your confidence rather than forcing an all-or-nothing reversal. For instance:
- “I am about 80% confident now.”
- “If this evidence appears, I will reduce that to around 50%.”
- “If stronger evidence appears later, I may go lower.”
Thinking in degrees makes updating easier than demanding complete certainty before changing your position.
Reviewing results without rewriting the threshold
The difficult moment comes after the evidence arrives.
Suppose your precommitment said that two independent, well-conducted studies would be enough to reduce your confidence. When those studies appear, you may notice an urge to introduce new requirements:
- “The sample should have been larger.”
- “Those researchers probably had an agenda.”
- “This was the wrong population.”
- “I’d really like five studies instead.”
Sometimes these objections are legitimate. Research quality genuinely matters. The key question is whether you would have demanded the same additional standard if the evidence had supported your preferred conclusion.
A practical review process is:
- Read your original change-my-mind statement.
- Compare the evidence against that original threshold.
- Separate genuine methodological concerns from standards you are inventing only after seeing the result.
- If you believe the original threshold was poorly designed, revise it only for future decisions—not retroactively for the current one.
This preserves fairness. You are allowed to improve your decision rules over time, but not to rewrite them simply because today’s evidence is uncomfortable.
Common ways people move the goalposts
Most goalpost shifting follows a few recognisable patterns.
Demanding ever stronger evidence. What counted as “strong” before the search suddenly becomes “not quite enough” after an unfavourable result appears.
Changing the question. A discussion that began about whether a claim was true quietly becomes a discussion about whether it was perfectly stated, universally true or free from every exception.
Introducing new criteria. Standards that were never mentioned beforehand become mandatory only after inconvenient evidence emerges.
Accepting weak supporting evidence while rejecting stronger opposing evidence. This asymmetric treatment is a hallmark of motivated reasoning, where desirable conclusions receive less scrutiny than undesirable ones. [PMC+2Frontiers]pmc.ncbi.nlm.nih.govPMCStop Fooling Yourself!Diagnosing and Treating Confirmation…by RT Born · 2024 · Cited by 9 — Confirmation bias (CB) is a cognitive bias that allows us to fo…
Recognising these patterns in yourself is usually more valuable than spotting them in other people.
A precommitment test is a guide, not a prison
Writing evidence standards in advance does not mean ignoring better arguments or refusing to adapt.
In scientific research, preregistered plans sometimes need modification because unexpected practical problems arise or new information becomes available. The recommended practice is not rigid obedience but transparent explanation: distinguish what was planned from what changed and why. [PMC+2Wharton Faculty Platform]pmc.ncbi.nlm.nih.govPMCThe preregistration revolutionNIHby BA Nosek · 2018 · Cited by 2585 — Preregistration is a solution that helps researchers maintain clarity between prediction an…
The same principle applies in everyday reasoning. If you discover that your original threshold was unrealistic, acknowledge that openly:
- “My original standard was too weak because I overlooked an important source of bias.”
- “I’ll use a better standard next time.”
That response preserves intellectual honesty. What undermines good reasoning is not changing your standards for future decisions, but quietly changing them only after learning which answer you prefer.
Building the habit
The effectiveness of a precommitment test comes less from its complexity than from its timing. A single written sentence created before you search for evidence creates a reference point that your future self cannot easily ignore.
Over time, this habit produces two benefits. First, it reduces the temptation to defend beliefs at any cost. Second, it makes genuine learning easier because changing your mind becomes an expected outcome of following your own rules rather than a personal defeat.
The goal is not to predict correctly every time. It is to make sure that when the evidence really does justify changing your mind, your standards allow it to happen instead of moving the goalposts.
Amazon book picks
Further Reading
Books and field guides related to How to Stop Moving the Goalposts. Use these as the next step if you want deeper reading beyond the article.
The Scout Mindset
Directly supports precommitment, evidence thresholds, and avoiding motivated reasoning.
Thinking, Fast and Slow
Explains why people unconsciously move goalposts and reinterpret evidence.
Decisive
Provides practical techniques for preventing biased decision-making before committing to conclusions.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: pages.ucsd.edu
Title: Pages Confirmation Bias: A Ubiquitous Phenomenon in Many
Link: https://pages.ucsd.edu/~mckenzie/nickersonConfirmationBias.pdfSource snippet
UCSD PagesConfirmation Bias: A Ubiquitous Phenomenon in Many...October 6, 2004 — by RS Nickerson · 1998 · Cited by 12458 — Confirmation...
Published: October 6, 2004
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCStop Fooling Yourself!
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11495861/Source snippet
(Diagnosing and Treating Confirmation...by RT Born · 2024 · Cited by 9 — Confirmation bias (CB) is a cognitive bias that allows us to fo...
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCThe preregistration revolution
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC5856500/Source snippet
NIHby BA Nosek · 2018 · Cited by 2585 — Preregistration is a solution that helps researchers maintain clarity between prediction an...
-
Source: frontiersin.org
Link: https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2024.1430953/fullSource snippet
Processing of misinformation as motivational and cognitive...by Y Zhou · 2024 · Cited by 25 — Confirmation bias, is the tendency to seek...
-
Source: faculty.wharton.upenn.edu
Title: Wharton Faculty Platform Pre‐registration is a Game Changer
Link: https://faculty.wharton.upenn.edu/wp-content/uploads/2016/11/35-Simmons-Nelson-Simonsohn-2021b.pdfSource snippet
But, Like Random...by JP Simmons · 2021 · Cited by 57 — When studies are pre-registered, reviewers do not have to try to read between th...
Additional References
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/360798343_Motivated_Reasoning_and_Attitudes_Towards_Supreme_Court_Confirmation_Hearings_Evidence_from_Five_Nominations_and_an_ExperimentSource snippet
Evidence from Five Nominations and an ExperimentRelying on theories of motivated reasoning, I hypothesize that individuals who favor a no...
-
Source: developmental-psychopathology.lab.uiowa.edu
Title: Petersen et al, 2024, ICD
Link: https://developmental-psychopathology.lab.uiowa.edu/sites/developmental-psychopathology.lab.uiowa.edu/files/2024-02/Petersen%20et%20al%2C%202024%2C%20ICD.pdfSource snippet
Developmental Psychopathology LabAdapting open science and pre‐registration to longitudinal...by IT Petersen · 2022 · Cited by 30 — Typi...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2012.01663 -
Source: youtube.com
Title: Preregistration: Improve Research Rigor, Reduce Bias
Link: https://www.youtube.com/watch?v=PboPpcg6ik4Source snippet
Making Smarter Decisions & Why Great Investors Are Great Quitters w/ Annie Duke...
-
Source: youtube.com
Title: The Tool That Stops You From Holding Losers Too Long | Annie Duke
Link: https://www.youtube.com/watch?v=Nu9qgbRm6bsSource snippet
This will make you a better decision maker | Annie Duke...
-
Source: scholar.google.com.au
Title: google.com.au David Nickerson
Link: https://scholar.google.com.au/citations?hl=zh-CN&user=oHjxSVEAAAAJSource snippet
Nickerson - Google 学术搜索DW Nickerson. American political Science review 102 (1), 49-57, 2008. 905, 2008. Vote buying and social desirabili...
-
Source: youtube.com
Title: When should evidence change your mind?
Link: https://www.youtube.com/watch?v=5ycjezG1gH0Source snippet
Preregistration: Improve Research Rigor, Reduce Bias...
-
Source: youtube.com
Title: This will make you a better decision maker | Annie Duke
Link: https://www.youtube.com/watch?v=svQMODvIGAESource snippet
When should evidence change your mind?...
-
Source: youtube.com
Title: Making Smarter Decisions & Why Great Investors Are Great Quitters w/ Annie Duke
Link: https://www.youtube.com/watch?v=84Uisiv6VVI
Topic Tree