Within Predictions
When being right still teaches the wrong lesson
Being correct for the wrong reason can hide a weak mental model unless the prediction includes the reasoning behind it.
On this page
- Why bare guesses are less useful
- How to separate outcome accuracy from reasoning quality
- How to repair a lucky or misleading explanation
Page outline Jump by section
Introduction
A correct prediction is not always evidence of good thinking. Sometimes the answer is right only because of luck, a misleading cue, or a flawed chain of reasoning that happened to land on the correct outcome. If you only record whether you were right, you risk reinforcing an inaccurate mental model. The real value of making predictions before checking the answer is that they reveal not just what you expected, but why you expected it. That distinction allows you to separate genuine understanding from fortunate coincidence and avoid repeating reasoning that only worked by accident. Research on mental models, hindsight bias and pretesting all points to the same lesson: feedback is most useful when it improves the reasoning process rather than merely rewarding the final answer. [PMC+2Springer]pmc.ncbi.nlm.nih.govPMCMental models and human reasoningNIHby PN Johnson-Laird · 2010 · Cited by 987 — The consensus in psychology was that our ability to reason depends on a tacit mental…
Why bare guesses are less useful
If you predict an outcome without recording your reasoning, you only learn whether the outcome matched reality. You do not learn whether your thinking deserved to succeed.
Imagine predicting that a company will outperform the market. If the share price rises, you might conclude that your analysis was sound. But perhaps your actual reasoning relied on an incorrect assumption about interest rates, while the gain was driven by an unrelated acquisition announcement. Your prediction was correct, yet your explanation failed.
The same pattern appears in everyday learning:
- A student chooses the correct multiple-choice answer after eliminating options using faulty logic.
- A doctor identifies the correct diagnosis but for symptoms that are not actually diagnostic.
- A programmer fixes a bug by changing several unrelated lines, with only one change making the real difference.
In each case, rewarding yourself simply for being correct strengthens reasoning that may fail the next time the circumstances change.
This is one reason prediction exercises become far more informative when they include a brief explanation of the expected mechanism rather than only the expected result. Research on the pretesting effect suggests that attempting answers before instruction improves learning because corrective feedback updates knowledge—not merely because learners guessed correctly or incorrectly. [PMC]pmc.ncbi.nlm.nih.govMetacognitive awareness of the pretesting effect improves…by SC Pan · 2023 · Cited by 44 — The pretesting or prequestion effect ref…
How to separate outcome accuracy from reasoning quality
A useful review asks two independent questions:
- Was the prediction correct?
- Was the reasoning valid?
Keeping these separate prevents success from hiding weak thinking.
One practical approach is to classify each prediction into four categories:
OutcomeReasoningWhat it meansCorrectSoundYour mental model was supported. Continue testing it in new situations.CorrectFlawedTreat this as a warning rather than a success. The result may have been accidental.IncorrectSoundThe reasoning may still have value if an overlooked fact or unusual event changed the outcome.IncorrectFlawedThis is the clearest opportunity for revision because both conclusion and explanation need attention.
The second category—correct answer, flawed reasoning—is often the easiest to overlook because success feels like confirmation. Yet it is frequently the most dangerous for long-term improvement.
Psychological research on hindsight bias shows that once the outcome is known, people tend to reconstruct their earlier thinking as if they had understood more than they actually did. This can make a lucky success appear like evidence of expertise unless the original reasoning was written down beforehand. [Springer]link.springer.comMetacognitive hindsight bias | Memory & Cognitionby R Ackerman · 2020 · Cited by 29 — Hindsight bias (HB) is a robust phenomenon…
Why being right can reinforce the wrong mental model
Mental models are internal explanations of how a system works. They improve when feedback identifies which assumptions were responsible for success or failure. They stagnate when feedback rewards outcomes without examining the underlying process. [PMC]pmc.ncbi.nlm.nih.govPMCMental models and human reasoningNIHby PN Johnson-Laird · 2010 · Cited by 987 — The consensus in psychology was that our ability to reason depends on a tacit mental…
Several mechanisms contribute to this problem.
Outcome bias. People often judge the quality of a decision by its result instead of by the information and reasoning available when the decision was made. A lucky outcome makes a poor decision appear wiser than it was.
Hindsight reconstruction. After learning the answer, memory shifts. People often remember themselves as having believed something closer to the truth than they actually did, making it harder to detect faulty reasoning. [Springer]link.springer.comMetacognitive hindsight bias | Memory & Cognitionby R Ackerman · 2020 · Cited by 29 — Hindsight bias (HB) is a robust phenomenon…
Selective reinforcement. Correct outcomes naturally feel rewarding. Without inspecting the explanation, the brain may strengthen every cue that happened to accompany success, including irrelevant or false ones.
These mechanisms explain why prediction journals that record only wins and losses provide much less value than journals that also preserve the reasoning behind each prediction.
How to repair a lucky or misleading explanation
A correct prediction supported by weak reasoning should be treated as unfinished work rather than evidence of mastery.
A useful review process is:
- Compare your explanation with the actual causal story. Ask which parts of your reasoning genuinely matched the evidence and which merely happened to point in the right direction.
- Identify the decisive assumption. Which belief, if changed, would have changed your prediction? Focus on that rather than every minor detail.
- Replace vague explanations with specific mechanisms. “It seemed likely” is difficult to evaluate later. “I expected demand to increase because prices fell” creates a testable claim.
- Test the explanation on a different example. If the reasoning only works for one isolated case, it may have been fitted to the outcome rather than reflecting a general principle.
The aim is not to erase lucky successes but to prevent them from becoming misleading lessons.
A simple habit that exposes wrong reasons
One sentence added before checking an answer often provides enough information to diagnose whether success came from understanding or luck:
“I predict X because Y.”
The first half records the prediction. The second records the mechanism you believe connects the evidence to the outcome.
After checking the answer, avoid asking only, “Was I right?” Instead ask:
- Which part of my explanation actually matched reality?
- Which assumption turned out to be irrelevant?
- Would I make the same prediction if one key fact changed?
- Could someone using the same reasoning reliably solve similar problems?
These questions shift attention from celebrating isolated successes to improving the mental models that generate future predictions.
Over time, this habit produces a more accurate picture of your thinking. You become less interested in collecting correct answers and more interested in developing explanations that remain reliable across new situations. That is the difference between being occasionally right by chance and becoming consistently right for the right reasons.
Amazon book picks
Further Reading
Books and field guides related to When being right still teaches the wrong lesson. Use these as the next step if you want deeper reading beyond the article.
Superforecasting
Strong focus on evaluating predictions by reasoning quality rather than outcomes.
The Scout Mindset
Emphasizes updating beliefs based on evidence instead of defending conclusions.
Thinking, Fast and Slow
Explains cognitive biases, luck, and flawed reasoning behind correct answers.
The Art of Thinking Clearly
Illustrates how cognitive mistakes can still produce correct outcomes.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCMental models and human reasoning
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC2972923/Source snippet
NIHby PN Johnson-Laird · 2010 · Cited by 987 — The consensus in psychology was that our ability to reason depends on a tacit mental...
-
Source: link.springer.com
Link: https://link.springer.com/article/10.3758/s13421-020-01012-wSource snippet
Metacognitive hindsight bias | Memory & Cognitionby R Ackerman · 2020 · Cited by 29 — Hindsight bias (HB) is a robust phenomenon...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC9839203/Source snippet
Metacognitive awareness of the pretesting effect improves...by SC Pan · 2023 · Cited by 44 — The pretesting or prequestion effect ref...
-
Source: pmc.ncbi.nlm.nih.gov
Title: Omission bias is the preference
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8763848/Source snippet
Impact of Cognitive Biases on Professionals' Decision...by V Berthet · 2022 · Cited by 306 — Hindsight bias is a propensity to perceive...
Additional References
-
Source: arxiv.org
Link: https://arxiv.org/html/2604.12046v1Source snippet
Improving Long-Form Generation Factuality via Reasoning...13 Apr 2026 — Accurate [uncertainty]({{ 'uncertainty/' | relative_url }}) estimation, or calibration...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/316486755_Cognitive_Biases_and_Their_Influence_on_Critical_Thinking_and_Scientific_Reasoning_A_Practical_Guide_for_Students_and_TeachersSource snippet
Cognitive Biases and Their Influence on [Critical Thinking]({{ 'critical-skills/' | relative_url }})...21 Jul 2023 — Researchers have discovered 200 cognitive biases that result i...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/371009673_Just_Ask_for_Calibration_Strategies_for_Eliciting_Calibrated_Confidence_Scores_from_Language_Models_Fine-Tuned_with_Human_FeedbackSource snippet
Strategies for Eliciting Calibrated Confidence Scores from...24 May 2023 — In this paper, we conduct a broad evaluation of computational...
Published: May 2023
-
Source: nirandfar.com
Link: https://www.nirandfar.com/hindsight-bias/Source snippet
Hindsight Bias: Why You Make Terrible Life ChoicesHindsight bias occurs when people feel that they “knew it all along” – when they believ...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/13072910_Falsifying_mental_models_Testing_the_predictions_of_theories_of_syllogistic_reasoningSource snippet
Testing the predictions of theories of syllogistic reasoningThe results are interpreted as showing that falsification of the kind propose...
-
Source: thedecisionlab.com
Link: https://thedecisionlab.com/biases/hindsight-biasSource snippet
Hindsight BiasThe hindsight bias describes our tendency to look back at an unpredictable event and think it was easily predictable.Read more...
-
Source: ml-site.cdn-apple.com
Link: https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdfSource snippet
analysis reveals that as problem complexity increases, correct solutions systematically emerge at later positions in thinking compared to...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=eYXh3ZCW9fs -
Source: medium.com
Link: https://medium.com/mind-cafe/mental-models-how-to-reason-better-be-less-irrational-17e095dcd2aeSource snippet
(Mental models require working memory, which is both limited and varies between individuals.) Why...Read more...
-
Source: verywellmind.com
Title: cognitive biases distort thinking 2794763
Link: https://www.verywellmind.com/cognitive-biases-distort-thinking-2794763Source snippet
12 Types of Cognitive Bias That Influence Your Thinking1 May 2026 — hindsight bias is a common cognitive bias that involves the tendency...
Published: May 2026
Topic Tree