How Do You Learn From Wrong Calls?

Introduction

Feedback loops improve judgement by turning a decision into a learning cycle: make the call, record the reasoning, observe what happened, compare outcome with expectation, and change the next decision process. Without that loop, experience can simply reinforce confidence. With it, errors become data. The core lesson from research on learning, forecasting, team reviews and hindsight bias is that feedback works best when it is specific, timely enough to be useful, tied to the original expectation, and focused on the reasoning process rather than on blame or embarrassment. Error alone is not enough; error followed by corrective feedback is what tends to produce learning, especially when people were confident in the wrong answer.[PubMed]pubmed.ncbi.nlm.nih.govLearning from Errorsby J Metcalfe · 2017 · Cited by 899 — Experimental investigations indicate that errorful learning followed by c…

This matters for improving thinking and analytical skills because many important judgements do not produce clean, immediate signals. A doctor may not hear about a missed diagnosis, a manager may confuse a lucky success with a sound decision, and a political forecaster may remember vague impressions rather than exact probabilities. Better judgement therefore depends on deliberately designing feedback: write down the prediction, define what would count as success or failure, review the result, and protect the review from hindsight excuses.

Why Feedback Matters

Feedback is the difference between merely having experience and learning from experience. Daniel Kahneman and Gary Klein’s influential discussion of intuitive expertise argued that skilled intuition is most trustworthy in environments with valid cues and opportunities for rapid, unambiguous feedback. Chess, firefighting and some clinical pattern-recognition tasks can offer repeated cues and correction; many strategic, financial, political and organisational decisions do not. In those weaker learning environments, people may feel experienced while receiving little reliable evidence about whether their judgement is improving.[PubMed]pubmed.ncbi.nlm.nih.govOpen source on nih.gov.

The practical implication is uncomfortable: confidence is not proof of learning. A person can make hundreds of decisions and still learn the wrong lessons if outcomes are noisy, delayed, selectively observed or interpreted defensively. This is why feedback loops should capture not just “what happened” but “what did we expect, how confident were we, and why?” The loop creates a record against which judgement can be tested rather than reconstructed.

Research on learning from errors supports this point. Janet Metcalfe’s review of experimental work found that making mistakes can aid learning when followed by corrective feedback, and that high-confidence errors can be especially memorable once corrected. The sting of being wrong is not the mechanism by itself; the useful part is the contrast between the mistaken belief and the corrective information.[PubMed]pubmed.ncbi.nlm.nih.govLearning from Errorsby J Metcalfe · 2017 · Cited by 899 — Experimental investigations indicate that errorful learning followed by c…

Feedback also needs good design. Kluger and DeNisi’s major meta-analysis of feedback interventions found that feedback improved performance on average, but more than a third of interventions reduced performance. One reason is that feedback can shift attention away from the task and towards ego, self-defence or social comparison. For judgement improvement, the safest rule is to make feedback about the decision model: information quality, assumptions, probability estimates, alternatives considered and triggers for revision.[ResearchGate]researchgate.netResearch Gate(PDF) The Effects of Feedback Interventions on PerformanceResearch Gate(PDF) The Effects of Feedback Interventions on Performance

What a Good Judgement Loop Looks Like

A useful judgement loop has four parts: a forecast, a decision record, an outcome check and a process review. Each part protects against a different failure.

First, the judgement must be stated clearly enough to be wrong. “This project feels risky” is too vague to teach much later. “There is a 40% chance the supplier misses the August deadline” is much better, because it can be compared with reality. Probability estimates are not just for professional forecasters; they force a thinker to separate “possible”, “likely” and “almost certain”.

Second, the reasoning must be recorded before the outcome is known. This can be a short decision journal: the options, the chosen path, the evidence available, the main assumptions, the confidence level, and what would change the decision. A record matters because memory is a poor audit trail. Once the result is known, people tend to remember the past as more predictable than it was. Hindsight bias includes memory distortion, changed beliefs about likelihood and inflated belief in one’s own foresight.[Carlson School of Management]carlsonschool.umn.eduvohs et al 2012 hindsight biasvohs et al 2012 hindsight bias

Third, the outcome must be checked against the original expectation. This is where many informal feedback loops fail. People review only dramatic failures, forget quiet near-misses, or stop at “it worked” and “it didn’t”. A better review distinguishes four cases:

Decision processGood outcomeBad outcomeGood processRepeat the process, but do not assume perfectionTreat as possible bad luck; check whether any warning signs were missedBad processTreat as good luck; do not reward the reasoningDiagnose the error and change the routine

This distinction helps avoid outcome bias: the tendency to judge a decision mainly by its result rather than by the quality of the reasoning given what was knowable at the time. A good bet can lose, and a reckless bet can win. Feedback improves judgement only when it can separate luck from skill.[The Decision Lab]thedecisionlab.comOpen source on thedecisionlab.com.

Finally, the loop must change behaviour. A review that produces no new threshold, checklist, question, trigger or habit is not a feedback loop; it is a conversation about the past. The smallest useful output is a rule for the next similar case: “Before approving a project plan, ask which assumption would make the timeline fail,” or “For any forecast above 80%, write down the strongest reason it might be wrong.”

Error Review Routines

The most effective error review routines are simple enough to use repeatedly. They do not require a long post-mortem for every decision. They require a stable habit of capturing judgement before reality rewrites the story.

A compact routine can work like this:

After-action reviews are a team version of this loop. They are commonly framed around a small set of questions: what was supposed to happen, what actually happened, why was there a difference, and what should change next time? BetterEvaluation describes an after-action review as a simple method for assessing organisational performance by bringing a team together to discuss a task, while guidance for humanitarian and operational settings stresses that an AAR is not a complaint session or a full evaluation report but a learning opportunity after successes and setbacks.[Better Evaluation]betterevaluation.orgOpen source on betterevaluation.org.

The evidence suggests that structured reviews can improve team performance when they are implemented well. A study of after-action review meetings notes that AARs can be used to reduce errors and cites evidence that team effectiveness can improve when teams conduct them. Patient-safety discussions have also treated AARs as a way to improve individual and team learning after clinical events, though the quality of the review depends heavily on psychological safety, relevant facts and follow-through.[PMC]pmc.ncbi.nlm.nih.govOpen source on nih.gov.

For individual judgement, the equivalent is a lightweight decision journal. It is most useful for decisions that are important, repeated or uncertain: hiring, investments, medical differentials, project approvals, forecasts, negotiations and strategic choices. The journal should be short enough that the person will actually use it. A five-minute entry before the decision is often more valuable than a polished essay afterwards, because it freezes the real state of knowledge before hindsight takes over.

Forecasting as a Training Ground

Forecasting is one of the cleanest ways to train judgement because it forces feedback into the open. A forecast has a question, a probability, a closing date and an outcome. This structure makes it harder to hide behind vague language such as “could”, “might”, “soon” or “likely”.

The Good Judgment Project is the best-known example. It used geopolitical forecasting tournaments in which forecasters made probabilistic predictions and received scores based on accuracy. Research associated with the project emphasised training, teaming, aggregation and the identification of especially accurate forecasters. Good Judgment’s own summary describes four keys to accurate forecasting: talent-spotting, training, teaming and aggregation.[lukemuehlhauser.com]lukemuehlhauser.comForecasting TournamentsForecasting Tournaments

The Brier score is a practical tool in this setting. It measures the squared error of a probabilistic forecast: if you say there is a 70% chance of an event and it happens, your error is smaller than if you said 20%. Good Judgment Open explains the score by converting forecast percentages into probabilities, coding reality as 0 or 1, and adding the squared differences. This makes overconfidence visible. A person who often says 90% for events that happen only 60% of the time cannot easily explain away the pattern.[Good Judgment Open]gjopen.comOpen source on gjopen.com.

Forecasting also shows why feedback should be repeated, not occasional. One prediction teaches little because any single outcome contains luck. Many scored predictions reveal patterns: overconfidence, underconfidence, poor base-rate use, slow updating, or a habit of treating preferred outcomes as likely. In business contexts, Tetlock and Schoemaker argued that the Good Judgment Project showed that even brief training could improve forecasting accuracy, and that organisations can strengthen judgement by turning vague expectations into explicit forecasts that are later reviewed.[Corporate Research Forum]crforum.co.uk2016 Schoemaker Tetlock Superforecasting HBR May 20162016 Schoemaker Tetlock Superforecasting HBR May 2016Published: May 2016

The broader lesson is not that every decision should become a formal tournament. It is that analytical skill improves when people practise making claims precise enough to score. Even a personal spreadsheet of twenty forecasts can reveal more about judgement than years of unrecorded impressions.

Avoiding Hindsight Excuses

The enemy of a feedback loop is the story people tell after they know the ending. Hindsight bias makes past events feel more foreseeable than they were, while outcome bias makes good results look like proof of good reasoning. Together, they let people protect their self-image instead of improving their judgement.

A common excuse is “we could not have known”. Sometimes that is true. A good review should not punish people for failing to predict genuinely unknowable events. But the phrase can also hide avoidable omissions: no one checked the base rate, the team ignored a dissenting expert, the decision relied on a single optimistic assumption, or no one defined what warning sign would trigger a rethink. The review question should therefore be: “What was knowable at the time, and did our process make proper use of it?”

The opposite excuse is equally dangerous: “it was obvious”. Once the outcome is known, people can overstate how clear the signals were. Hoch and Loewenstein’s work on outcome feedback and hindsight found that people can extract diagnostic information from feedback while also overestimating what they would have known beforehand. This is the central tension: outcome feedback is necessary for learning, but it can also distort memory unless the original judgement was recorded.[Carnegie Mellon University]cmu.eduCarnegie Mellon University Outcome Feedback: Hindsight and InformationCarnegie Mellon University Outcome Feedback: Hindsight and Information

A strong review separates three questions:

This structure prevents the lazy conclusion that every failure proves foolishness and every success proves wisdom. It also reduces blame. People are more likely to admit errors when the review asks how the system of judgement can improve rather than who should be embarrassed.

When Feedback Misleads

Feedback loops can make judgement worse when the signal is biased, delayed or self-reinforcing. In hiring, a manager may never observe the candidates they rejected, so the feedback loop only contains people they selected. In medicine, a clinician may not receive follow-up on patients who went elsewhere. In investing, a lucky gain can reward a poor process. In social media, engagement feedback can train people to favour attention over accuracy.

Algorithmic decision-making offers a vivid version of the same problem. Research on feedback loops in automated decision systems shows that when predictions affect the environment, the resulting data can reinforce the original pattern. For example, if policing resources are repeatedly sent to the same neighbourhoods, recorded incidents may rise there partly because more observation is taking place, not necessarily because underlying offending is uniquely higher. Pagan and colleagues classify such feedback dynamics and argue that they can perpetuate, reinforce or sometimes reduce bias depending on the structure of the loop.[arXiv]arxiv.orgOpen source on arxiv.org.

The human equivalent is selective feedback. If a manager only hears from happy customers, only reviews wins, or only studies visible failures, the loop teaches a distorted lesson. Better judgement requires asking: “What feedback are we not seeing?” Rejected options, near-misses, silent users, delayed harms and counterfactual outcomes are often the missing data.

Feedback can also mislead when it is too personal. The Kluger and DeNisi findings are important here: feedback interventions can backfire when they direct attention away from the task and towards the self. “You are bad at analysis” is less useful than “your forecast ignored the base rate and used only the most recent example.” The second statement gives the thinker a lever to change.[ResearchGate]researchgate.netResearch Gate(PDF) The Effects of Feedback Interventions on PerformanceResearch Gate(PDF) The Effects of Feedback Interventions on Performance

Building the Loop Into Everyday Decisions

The best feedback loop is the one that survives contact with a busy life. For most people, that means using different levels of review for different stakes.

For small repeated judgements, use quick scoring. Estimate how long tasks will take, how likely a meeting is to resolve an issue, or how often a chosen habit will happen this week. Review the prediction later. The point is not the topic; it is practising calibration.

For medium decisions, use a short decision journal. Record the decision, options, assumptions, confidence and review date. This is useful for hiring, purchases, project plans, negotiations and health-related choices where uncertainty is real but a full review would be excessive.

For high-stakes team decisions, use a pre-mortem before action and an after-action review afterwards. Gary Klein’s pre-mortem asks a team to imagine that a project has failed and then work backwards to identify plausible reasons. Its value is that it makes dissent safer before people are publicly committed to a plan.[Harvard Business Review]hbr.orgHarvard Business Review Performing a Project PremortemHarvard Business Review Performing a Project Premortem

The review should then close the loop after reality has supplied evidence. Wharton’s guidance on after-action reviews emphasises scheduling them consistently, including after successes as well as failures, and grounding the discussion in relevant facts and figures. That consistency matters because reviewing only failures teaches people that feedback is punishment. Reviewing successes too helps identify good processes worth repeating and lucky outcomes that should not be overlearned.[Wharton Executive Education]executiveeducation.wharton.upenn.eduWharton Executive Education After-Action Reviews: A Simple Yet Powerful ToolWharton Executive Education After-Action Reviews: A Simple Yet Powerful Tool

A practical feedback culture asks a few plain questions again and again:

These questions are simple, but they change the unit of learning. Instead of learning from an outcome in isolation, the thinker learns from the gap between expectation, reasoning and reality.

The Real Payoff

Feedback loops make better judgement less mysterious. They replace the fantasy of flawless intuition with a humbler and more effective process: make reasoning visible, let reality test it, and revise the next call. They also make improvement fairer. A person is no longer judged only by whether the last outcome was good or bad, but by whether their reasoning process is becoming better calibrated, better evidenced and less defensive.

The most important shift is from self-protection to error use. Wrong calls are not automatically proof of incompetence; they are opportunities to discover whether the failure came from missing data, poor assumptions, weak reasoning, bad incentives or ordinary uncertainty. Good calls are not automatically proof of wisdom; they may be the result of luck. A feedback loop keeps both lessons alive.

For anyone trying to improve analytical skill, the essential habit is therefore not simply “seek feedback”. It is to design feedback that cannot be easily bent into a flattering story. Record the call before the result, score it against reality, review the process without excuses, and convert the lesson into a changed routine. That is how judgement becomes trainable.

Amazon book picks

Further Reading

Books and field guides related to How Do You Learn From Wrong Calls?. Use these as the next step if you want deeper reading beyond the article.

Book

Thinking, Fast and Slow

By Daniel Kahneman

Explains biases, feedback, intuition and learning from errors.

See on Amazon

Book

The Scout Mindset

By Julia Galef

Focuses on updating beliefs from feedback rather than defending them.

See on Amazon

Book

Decisive

By Chip Heath, Dan Heath

Provides practical routines for improving decision quality.

See on Amazon

Book

How to Measure Anything

By Douglas W. Hubbard

Encourages measurable thinking and evidence-based learning.

See on Amazon

Browse more on Amazon: Thinking, Fast and Slow The Scout Mindset Decisive

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Example eBay listing

Motivational Quote Poster Wall Art Gym Office Bedroom Home Decor A4 A3 A5

SearcheBay.co.uk: office motivational poster

Browse similar oneBay.co.uk

Example eBay listing

A4 Laminated SUCCESS Motivational Poster | Minimalist Wall Art | Home Office Stu

SearcheBay.co.uk: office motivational poster

Browse similar oneBay.co.uk

Example eBay listing

Motivational Quote Poster Wall Art Gym Office Bedroom Home Decor A4 A3 A5

SearcheBay.co.uk: office motivational poster

Browse similar oneBay.co.uk

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: researchgate.net
Title: Research Gate(PDF) The Effects of Feedback Interventions on Performance
Link:https://www.researchgate.net/publication/232458848_The_Effects_of_Feedback_Interventions_on_Performance_A_Historical_Review_a_Meta-Analysis_and_a_Preliminary_Feedback_Intervention_Theory

2. Source: betterevaluation.org
Link:https://www.betterevaluation.org/methods-approaches/methods/after-action-review

3. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC6020850/

4. Source: lukemuehlhauser.com
Title: Forecasting Tournaments
Link:https://lukemuehlhauser.com/wp-content/uploads/Tetlock-et-al-Forecasting-tournaments-tools-for-increasing-transparency-and-improving-the-quality-of-debate.pdf

5. Source: arxiv.org
Link:https://arxiv.org/abs/2305.06055

6. Source: researchgate.net
Link:https://www.researchgate.net/publication/3229642_Performing_a_Project_Premortem

7. Source: researchgate.net
Link:https://www.researchgate.net/publication/375085139_A_Classification_of_Feedback_Loops_and_Their_Relation_to_Biases_in_Automated_Decision-Making_Systems

8. Source: researchgate.net
Link:https://www.researchgate.net/profile/Philip-Tetlock

9. Source: researchgate.net
Link:https://www.researchgate.net/publication/26798603_Conditions_for_Intuitive_Expertise

10. Source: researchgate.net
Title: 385386278 Scoring Rules and Calibration for Imprecise Probabilities
Link:https://www.researchgate.net/publication/385386278_Scoring_Rules_and_Calibration_for_Imprecise_Probabilities

11. Source: researchgate.net
Title: 318013212 The Premortem Technique
Link:https://www.researchgate.net/publication/318013212_The_Premortem_Technique

12. Source: researchgate.net
Link:https://www.researchgate.net/publication/292477080_Implementing_after-action_review_systems_in_organizations_Key_principles_and_practical_considerations

13. Source: researchgate.net
Link:https://www.researchgate.net/publication/373114533_Immediate_Versus_Delayed_Feedback_on_Learning_Do_People%27s_Instincts_Really_Conflict_with_Reality

14. Source: arxiv.org
Link:https://arxiv.org/abs/2410.23001

15. Source: medium.com
Link:https://medium.com/%40shreyashere/how-to-use-[pre-mortems

16. Source: medium.com
Link:https://medium.com/%40quixotic_scholar/not-so-fast-the-hidden-value-of-delaying-educational-feedback-b2282caa04f5

17. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/27648988/

18. Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/19739881/

19. Source: carlsonschool.umn.edu
Title: vohs et al 2012 hindsight bias
Link:https://carlsonschool.umn.edu/sites/carlsonschool.umn.edu/files/2026-01/vohs-et-al-2012-hindsight-bias.pdf

20. Source: thedecisionlab.com
Link:https://thedecisionlab.com/biases/hindsight-bias

21. Source: thedecisionlab.com
Link:https://thedecisionlab.com/biases/outcome-bias

22. Source: goodjudgment.com
Link:https://goodjudgment.com/about/the-science-of-superforecasting/

23. Source: gjopen.com
Link:https://www.gjopen.com/faq

24. Source: crforum.co.uk
Title: 2016 Schoemaker Tetlock Superforecasting HBR May 2016
Link:https://www.crforum.co.uk/wp-content/uploads/2016/10/2016_Schoemaker_Tetlock_Superforecasting-HBR-May-2016.pdf
Published: May 2016

25. Source: cmu.edu
Title: Carnegie Mellon University Outcome Feedback: Hindsight and Information
Link:https://www.cmu.edu/dietrich/sds/docs/loewenstein/OutcomeFeedback.pdf

26. Source: hbr.org
Title: Harvard Business Review Performing a Project Premortem
Link:https://hbr.org/2007/09/performing-a-project-premortem

27. Source: executiveeducation.wharton.upenn.edu
Title: Wharton Executive Education After-Action Reviews: A Simple Yet Powerful Tool
Link:https://executiveeducation.wharton.upenn.edu/thought-leadership/wharton-at-work/2021/07/after-action-reviews-simple-tool/

28. Source: research-collection.ethz.ch
Link:https://www.research-collection.ethz.ch/server/api/core/bitstreams/5cc66b9a-8013-499b-90f9-7aeba05c0195/content

29. Source: research-collection.ethz.ch
Link:https://www.research-collection.ethz.ch/server/api/core/bitstreams/a56c57ea-9f4a-4340-a9f6-61016712cd02/content

30. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC11281873/

31. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC7333631/

32. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC12818272/

33. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC12330140/

34. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC8763826/

35. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10149772/

36. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC8763848/

37. Source: Wikipedia
Title: Brier score
Link:https://en.wikipedia.org/wiki/Brier_score

38. Source: Wikipedia
Title: The Good Judgment Project
Link:https://en.wikipedia.org/wiki/The_Good_Judgment_Project

39. Source: frontiersin.org
Link:https://www.frontiersin.org/journals/applied-mathematics-and-statistics/articles/10.3389/fams.2021.669546/full

40. Source: frontiersin.org
Link:https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2021.706821/full

41. Source: cognitivebiaslab.com
Title: Hindsight Bias
Link:https://www.cognitivebiaslab.com/bias/bias-hindsight/

42. Source: thedecisionlab.com
Title: thinking fast when intuition isnt all bad according to its biggest critic
Link:https://thedecisionlab.com/insights/society/thinking-fast-when-intuition-isnt-all-bad-according-to-its-biggest-critic

43. Source: dataopsschool.com
Title: brier score
Link:https://dataopsschool.com/blog/brier-score/

44. Source: jstor.org
Link:https://www.jstor.org/stable/20182507

45. Source: ink.library.smu.edu.sg
Link:https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=5536&context=soss_research

46. Source: decision-mastery.com
Title: decision journal
Link:https://www.decision-mastery.com/articles/decision-journal

Additional References

47. Source: columbia.edu
Title: Learning from errorsAnnual ReviewMetcalfe2016
Link:https://www.columbia.edu/cu/psychology/metcalfe/PDFs/Learning%20from%20errorsAnnual%20ReviewMetcalfe2016.pdf

48. Source: youtube.com
Link:https://www.youtube.com/watch?v=V8cAYSyg724

49. Source: youtube.com
Title: Intuition Is Experience Speaking — with Gary Klein
Link:https://www.youtube.com/watch?v=BR3kfG6STZI

50. Source: aiimpacts.org
Link:https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project/

51. Source: thebsps.org
Link:https://www.thebsps.org/short-reads/accuracy-and-calibration-williams-pettigrew/

52. Source: prplayers.co.uk
Link:https://www.prplayers.co.uk/case-studies/medical/after-action-review-aar

53. Source: england.nhs.uk
Link:https://www.england.nhs.uk/improvement-hub/wp-content/uploads/sites/44/2015/08/learning-handbook-after-action-review.pdf

54. Source: academia.edu
Link:https://www.academia.edu/109849319/Cognitive_forcing_strategies_in_clinical_decisionmaking

55. Source: alliancefordecisioneducation.org
Link:https://alliancefordecisioneducation.org/resources/conducting-a-pre-mortem/

56. Source: kirschnered.nl
Link:https://www.kirschnered.nl/2025/07/15/deliberate-learning-and-errors/