Within Feedback
Was It Skill, Luck or Bad Feedback?
Good outcomes can reward bad reasoning, while bad outcomes can punish sound decisions that simply ran into bad luck.
On this page
- Why results can mislead reviews
- The four cell process and outcome check
- How to reward reasoning without ignoring results
Page outline Jump by section
Introduction
A good decision can produce a bad result, and a poor decision can produce a good one. This simple fact is one of the biggest obstacles to improving judgement. When people evaluate decisions by their outcomes instead of by the quality of the reasoning available at the time, they fall into outcome bias. The consequence is a distorted feedback loop: luck gets mistaken for skill, while bad luck is mistaken for poor judgement. Over time, this encourages risky habits that happened to succeed and discourages careful thinking that happened to fail. Classic psychological research first demonstrated this effect experimentally, and more recent replication studies show that it remains a robust feature of human judgement.[PubMed]pubmed.ncbi.nlm.nih.govPub Med Outcome bias in decision evaluationOutcome bias in decision evaluation - PubMed - NIHby J Baron · 1988 · Cited by 1449 — In 5 studies, undergraduate subjects were giv…
Within a feedback system, the central question is therefore not merely, “Did this work?” but “Given what was known beforehand, was this a sound decision?” Separating process quality from outcome quality creates feedback that teaches rather than merely rewards or punishes.
Why results can mislead reviews
Outcomes contain two ingredients: the quality of the decision and the role of chance. Reviews become misleading when they assume the outcome reflects only the first.
Suppose two investors analyse identical information and make equally well-supported decisions. One benefits from an unexpected market rally, while the other is hit by an unforeseen geopolitical shock. If their reasoning was equally sound, the difference in returns says little about who made the better decision. Yet observers—and often the decision-makers themselves—naturally treat the winner as more competent.
Jonathan Baron and John Hershey demonstrated this tendency in a series of influential experiments. Participants evaluated medical and financial decisions under uncertainty. Although the information available to the decision-maker remained identical, merely changing the eventual outcome caused participants to judge the quality of the original decision differently. Positive outcomes led to higher ratings of competence and reasoning, while negative outcomes led to harsher evaluations—even when participants explicitly recognised that outcomes should not matter.[PubMed]pubmed.ncbi.nlm.nih.govPub Med Outcome bias in decision evaluationOutcome bias in decision evaluation - PubMed - NIHby J Baron · 1988 · Cited by 1449 — In 5 studies, undergraduate subjects were giv…
A large pre-registered replication published in 2023 found the same basic effect using modern methods and a substantially larger sample. Participants continued to judge identical decisions differently depending on whether the outcome was favourable or unfavourable, confirming that outcome bias remains a reliable feature of human evaluation.[rips-irsp.com]rips-irsp.comOutcomes Affect Evaluations of Decision Qualityby S Aiyer · 2023 · Cited by 4 — Research by Baron and Hershey (1988) demonstrated that ou…
The practical danger is that noisy environments produce misleading lessons:
- A reckless strategy that succeeds once appears brilliant.
- A cautious strategy that fails because of rare bad luck appears foolish.
- Organisations begin rewarding visible success rather than disciplined reasoning.
- Individuals become overconfident after lucky wins and excessively cautious after unlucky losses.
In environments such as investing, medicine, business strategy or public policy, where uncertainty is unavoidable, this distortion can accumulate into systematic errors.
The four-cell process and outcome check
A useful review separates decision quality from outcome quality. Rather than treating success and failure as a single scale, it helps to classify decisions into four possibilities.
Decision processOutcomeInterpretationGood processGood outcomeDesired result, but success should not automatically validate every detail of the reasoning.Good processBad outcomeOften bad luck. The process may still deserve to be repeated in similar circumstances.Poor processGood outcomeLucky success. This is the most dangerous cell because it encourages repeating flawed reasoning.Poor processBad outcomeBoth the reasoning and the result require correction.
The most valuable learning often comes from the two mixed cases.
A good process with a bad outcome asks whether the uncertainty was recognised beforehand. If the risk was known and acceptable, the decision may still have been correct. For example, a surgeon choosing the treatment with the highest probability of success cannot guarantee survival in every individual case.
A poor process with a good outcome is more deceptive. Someone who ignored evidence, skipped analysis or relied on intuition may still succeed because chance happened to favour them. Without careful review, that success reinforces precisely the habits that should be changed.
The four-cell approach encourages reviewers to ask two separate questions:
- Was the reasoning appropriate given the information available at the time?
- How much of the outcome was influenced by uncertainty beyond the decision-maker’s control?
Only after answering both questions should performance be evaluated.
How outcome bias weakens learning
Outcome bias damages learning because it corrupts the feedback signal that experience provides.
Instead of identifying which thinking patterns consistently produce better decisions, people begin associating emotional reactions with isolated outcomes. A painful failure creates the illusion that the underlying reasoning was flawed. An exciting success creates the illusion that every judgement leading to it was insightful.
This produces several recurring problems:
- False confidence. Lucky successes increase confidence without increasing skill.
- False self-doubt. Sound reasoning abandoned after an unlucky outcome may never demonstrate its long-term value.
- Inconsistent standards. Identical decisions receive praise or criticism depending solely on events outside the decision-maker’s control.
- Poor experimentation. Teams become reluctant to test sensible innovations if any unsuccessful trial is treated as evidence of incompetence.
The distortion is particularly severe in low-feedback environments where important decisions are rare and outcomes unfold slowly. A single visible success or failure may dominate memory despite containing little information about the underlying quality of judgement.
How to reward reasoning without ignoring results
Avoiding outcome bias does not mean ignoring results. Outcomes remain essential because they reveal whether assumptions matched reality. The challenge is to use outcomes as evidence rather than as the sole measure of decision quality.
Several practices help preserve this distinction.
Record reasoning before acting. Decision journals capture assumptions, forecasts, confidence levels and expected risks before hindsight can reshape memory. Later reviews compare the original reasoning with what actually happened rather than relying on reconstructed recollections.
Judge decisions using information available at the time. Reviewers should deliberately ignore facts that emerged only after the decision. This recreates the original uncertainty and produces a fairer assessment.
Look for repeated patterns rather than isolated wins or losses. A single outcome is often dominated by randomness. Consistent performance across many decisions provides much stronger evidence of skill.
Reward disciplined processes as well as successful outcomes. Organisations that recognise careful analysis, explicit assumptions and thoughtful risk management reduce the temptation to chase lucky victories.
Separate accountability from luck. People remain responsible for the quality of their reasoning, preparation and risk management, but not for every random event that occurs afterwards.
These practices align incentives with long-term judgement rather than short-term fortune.
The balance between process and results
Focusing on process alone would also be a mistake. A beautifully reasoned decision that repeatedly produces poor outcomes eventually signals that important assumptions are wrong. Outcomes contain valuable information; they simply cannot be interpreted in isolation.
The most reliable feedback loop therefore combines both perspectives. Process reviews ask whether the reasoning was sound when the decision was made. Outcome reviews ask whether repeated experience suggests the model itself needs revision. Neither question replaces the other.
By consistently distinguishing skill from luck and reasoning from results, decision-makers avoid rewarding fortunate mistakes or punishing sensible risk-taking. That distinction is one of the foundations of better judgement in any uncertain environment.
Amazon book picks
Further Reading
Books and field guides related to Was It Skill, Luck or Bad Feedback?. Use these as the next step if you want deeper reading beyond the article.
Black Box Thinking
Encourages process-focused learning rather than outcome-focused blame.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
1.
Source: rips-irsp.com
Link:https://rips-irsp.com/articles/10.5334/irsp.751
Source snippet
Outcomes Affect Evaluations of Decision Qualityby S Aiyer · 2023 · Cited by 4 — Research by Baron and Hershey (1988) demonstrated that ou...
2.
Source: rips-irsp.com
Link:https://rips-irsp.com/articles/751/files/64d085974dca9.pdf
Source snippet
Outcomes Affect Evaluations of Decision Qualityby S Aiyer · 2023 · Cited by 4 — Outcome bias is the phenomenon whereby decisions which re...
3.
Source: pubmed.ncbi.nlm.nih.gov
Title: Pub Med Outcome bias in decision evaluation
Link:https://pubmed.ncbi.nlm.nih.gov/3367280/
Source snippet
Outcome bias in decision evaluation - PubMed - NIHby J Baron · 1988 · Cited by 1449 — In 5 studies, undergraduate subjects were giv...
4.
Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC12372742/
Source snippet
In their Experiment 1, participants were presented with 15 cases of medical decisions...Read more...
5.
Source: Wikipedia
Title: Outcome bias
Link:https://en.wikipedia.org/wiki/Outcome_bias
Source snippet
Outcome biasThe outcome bias is an error made in evaluating the quality of a decision when the outcome of that decision is already kno...
Additional References
6.
Source: newristics.com
Link:https://newristics.com/heuristics-biases/outcome-bias
Source snippet
Definition, Example & How Outcome Bias WorksOutcome bias refers to the tendency to judge a decision or action based on its outcome rather...
7.
Source: thedecisionlab.com
Link:https://thedecisionlab.com/biases/outcome-bias
Source snippet
Outcome bias: Why we blame bad results, not bad reasoningOutcome bias is a cognitive bias where we evaluate decisions primarily by how th...
8.
Source: merriam-webster.com
Link:https://www.merriam-webster.com/thesaurus/resulting
Source snippet
RESULTING Synonyms: 81 Similar and Opposite WordsSynonyms for RESULTING: resultant, consequent, related, subsequent, associated, followin...
9.
Source: researchgate.net
Link:https://www.researchgate.net/publication/19789598_Outcome_Bias_in_Decision_Evaluation
Source snippet
(PDF) Outcome Bias in Decision EvaluationThis research finds that outcomes impact employee evaluations, even when the evaluator has acces...
10.
Source: hermes.cde.state.co.us
Link:https://hermes.cde.state.co.us/islandora/object/co%3A31690/datastream/OBJ/download/Outcome_bias_in_decision_evaluation.pdf
Source snippet
Bias in Decision EvaluationAccording the study, people rated the quality of thinking better, the decision maker as more competent, and we...
11.
Source: ixdf.org
Title: outcome bias not all outcomes are created equal
Link:https://ixdf.org/literature/article/outcome-bias-not-all-outcomes-are-created-equal
Source snippet
Outcome Bias – Not All Outcomes are Created Equal2 Dec 2015 — The first mention of Outcome Bias was probably in Baron and Hershey's paper...
12.
Source: bear.warrington.ufl.edu
Title: baron hershey jpsp1988
Link:https://bear.warrington.ufl.edu/brenner/mar7588/Papers/baron-hershey-jpsp1988.pdf
Source snippet
Bias in Decision Evaluationby J Baron · 1988 · Cited by 1449 — In 5 studies, undergraduate subjects were given descriptions and outcomes...
13.
Source: commonweb.unifr.ch
Link:https://commonweb.unifr.ch/artsdean/pub/gestens/f/as/files/4660/21931_171009.pdf
Source snippet
Bias in Decision Evaluationby JC Hershey · 1988 · Cited by 1449 — Subjects rated the quality of thinking of the decisions, the competence...
14.
Source: lawprof.co
Title: Resulting Trust
Link:https://lawprof.co/definition/resulting-trust/
Source snippet
English Law DefinitionA resulting trust is an implied trust that arises by operation of law when beneficial ownership of property returns...
15.
Source: psychologytoday.com
Title: why you shouldnt judge decisions by results alone
Link:https://www.psychologytoday.com/gb/blog/decisions-and-the-brain/202509/why-you-shouldnt-judge-decisions-by-results-alone
Source snippet
Baron, J., & Hershey, J. C. (1988).Read more...
Topic Tree