Within Real Practice
Why feedback turns practice into better judgement
Real-problem practice improves fastest when predictions, decisions, outcomes and reasoning reviews are connected in a repeatable loop.
On this page
- Make predictions before the answer is known
- Compare outcomes with the original reasoning
- Separate lucky results from sound decisions
Page outline Jump by section
Introduction
Real-problem practice improves analytical skill only when it includes a feedback loop. Solving realistic problems is not enough on its own. People become better judges when they make predictions before the answer is known, compare those predictions with what actually happened, and examine where their reasoning succeeded or failed. Without that cycle, experience often produces confidence rather than accuracy.
Learning research and decision science reach a similar conclusion. Expertise develops most reliably in environments where people receive timely, meaningful feedback that allows them to connect decisions with outcomes. When feedback is weak, delayed or misleading, experience alone is a poor teacher because memory, hindsight and luck distort what people think they have learned. [PubMed]pubmed.ncbi.nlm.nih.govConditions for intuitive expertise: a failure to disagreeby D Kahneman · 2009 · Cited by 4056 — This article reports on an effort t…
Why feedback changes the quality of practice
The key advantage of real-world practice is not realism for its own sake. It is that reality can answer back.
A logic exercise usually has a predefined correct answer. Real decisions rarely do. Instead, they generate evidence over time: a forecast proves accurate or inaccurate, a project finishes earlier or later than expected, a hiring decision succeeds or disappoints, or a policy creates intended and unintended effects. Those outcomes provide information that can improve future judgement—if they are deliberately reviewed.
The crucial distinction is between experience and learning from experience. Many people repeat similar decisions for years without becoming substantially more accurate because they never compare what they believed beforehand with what later occurred. Kahneman and Klein argue that genuine expertise requires both a sufficiently predictable environment and repeated opportunities to receive informative feedback. Without those conditions, repeated experience can reinforce overconfidence instead of skill. [PubMed]pubmed.ncbi.nlm.nih.govConditions for intuitive expertise: a failure to disagreeby D Kahneman · 2009 · Cited by 4056 — This article reports on an effort t…
Make predictions before the answer is known
The strongest feedback loop begins before any outcome exists.
Writing down a prediction forces analytical reasoning into a form that can later be tested instead of reconstructed from memory. A useful prediction includes:
- the expected outcome
- the main reasons supporting that expectation
- the assumptions that matter most
- factors that would prove the judgement wrong
- a confidence estimate rather than simple certainty
This transforms vague impressions into falsifiable claims.
For example, instead of saying, “I think this product launch will go well,” a better prediction would state:
- launch date likely to remain unchanged
- customer complaints expected below a specified threshold
- strongest assumption is manufacturing stability
- confidence estimated at roughly 70%
Months later, every element can be reviewed independently.
Prediction also exposes hidden assumptions. Many analytical mistakes occur not because reasoning is illogical but because important assumptions remain implicit and therefore escape later evaluation.
Compare outcomes with the original reasoning
The review stage is where improvement actually occurs.
Many people naturally compare outcomes only with expectations:
“Was I right?”
A more useful question is: [linkedin.com]linkedin.comA summary of Daniel Kahneman and Gary Klein's article…The article is based on one simple question: Under what conditions are the intui…
“Which part of my reasoning survived contact with reality?”
The difference matters because predictions consist of multiple components:
- evidence used
- assumptions made
- interpretation of uncertainty
- weighting of competing explanations
- final judgement
Sometimes the conclusion is correct for the wrong reasons. Sometimes the conclusion is wrong even though most of the reasoning was sound.
A structured review therefore asks questions such as:
- Which assumptions proved true?
- Which evidence turned out to be misleading?
- Which warning signs did I overlook?
- What information became available later that genuinely could not have been known beforehand?
- What information was available but ignored?
This separates flaws in thinking from unavoidable uncertainty.
Separate lucky results from sound decisions
Perhaps the hardest lesson in analytical practice is that outcomes do not always reveal decision quality.
A poor decision can produce an excellent result through luck.
A careful, well-reasoned decision can produce a disappointing outcome because uncertainty remained unavoidable.
Decision researchers refer to judging decisions primarily by their outcomes as outcome bias or, in Annie Duke’s terminology, “resulting”. Experiments consistently show that people rate identical decisions differently once they know whether the eventual outcome was favourable or unfavourable, even when the outcome depended partly on chance. [Heidelberg University]uni-heidelberg.deHeidelberg UniversityOutcome bias in the evaluation of financial agentsDecember 21, 2017 — by C König-Kersting · 2017 · Cited by 74 — Abs…
A useful review therefore separates two questions:
QuestionPurposeWas the reasoning sound given the evidence available at the time?Evaluates analytical quality.Did events later produce a favourable result?Evaluates outcome, including luck.
Keeping these questions separate prevents random success from reinforcing poor habits.
For example:
- Good reasoning + good outcome: repeat the process.
- Good reasoning + bad outcome: examine whether uncertainty rather than poor analysis explains the result.
- Poor reasoning + good outcome: treat success cautiously because luck may be disguising weak thinking.
- Poor reasoning + bad outcome: identify which parts of the process require improvement.
The third case is often the most dangerous because success discourages learning.
Protect the feedback loop from hindsight
Once outcomes become known, memory changes surprisingly quickly.
People commonly remember having been more certain than they really were or believe they “knew it all along”. This hindsight bias makes honest learning difficult because the original uncertainty disappears from memory. [Wikipedia]WikipediaHindsight biasHindsight bias
The simplest defence is contemporaneous records.
Decision journals are effective because they preserve what was actually believed before events unfolded. Even brief notes recording predictions, assumptions and confidence levels make later reviews much more accurate than relying on memory alone. Decision scientists and experienced investors have long recommended this practice because it reveals systematic thinking errors that hindsight tends to erase. [Eva Keiffenheim]evakeiffenheim.substack.comEva Keiffenheim How to Engineer Skill in InvestingRead what you actually wrote six months ago, then compare it to what you “remember” thinking. The gap…Read more…
Shorten the learning cycle where possible
Some domains naturally provide rapid feedback.
Examples include:
- forecasting measurable events
- estimating project completion times
- analysing sports performance
- operational process improvement
- repeated business decisions
Other domains produce slow or ambiguous feedback. Strategic planning, hiring, policy decisions or long-term investments may take years before consequences become visible.
Where natural feedback is delayed, practitioners often create artificial checkpoints instead:
- interim forecasts
- milestone reviews
- assumption audits
- scenario updates
- pre-planned post-mortems
These intermediate comparisons provide more opportunities for correction before final outcomes arrive.
What a complete feedback loop looks like
The most effective cycle is simple but disciplined:
- Define the decision or prediction.
- Record the reasoning before the outcome is known.
- Estimate confidence rather than claiming certainty.
- Wait for sufficient evidence to emerge.
- Compare reality with the original reasoning, not reconstructed memory.
- Identify whether errors came from evidence, assumptions, interpretation or chance.
- Adjust future reasoning while keeping successful analytical habits.
Repeated dozens or hundreds of times, this loop gradually calibrates judgement. Confidence becomes better matched to actual accuracy, recurring mistakes become visible, and analytical skill becomes grounded in evidence rather than intuition alone.
Unlike isolated thinking exercises, real-problem practice supplies an external test. Feedback is the mechanism that closes the gap between what seemed persuasive at the time and what actually proved true, allowing experience to become genuine improvement rather than merely accumulated time.
Amazon book picks
Further Reading
Books and field guides related to Why feedback turns practice into better judgement. Use these as the next step if you want deeper reading beyond the article.
Superforecasting
Centers on prediction, outcome tracking and continuous feedback as routes to better judgement.
Thinking in Bets
Teaches separating decision quality from outcomes and learning through feedback.
Noise
Explores improving judgement by reducing inconsistency and evaluating decisions systematically.
Thinking, Fast and Slow
Explains why feedback is often misinterpreted because of cognitive biases.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: Wikipedia
Title: Outcome bias
Link: https://en.wikipedia.org/wiki/Outcome_bias -
Source: Wikipedia
Title: Hindsight bias
Link: https://en.wikipedia.org/wiki/Hindsight_bias -
Source: pubmed.ncbi.nlm.nih.gov
Link: https://pubmed.ncbi.nlm.nih.gov/19739881/Source snippet
Conditions for intuitive expertise: a failure to disagreeby D Kahneman · 2009 · Cited by 4056 — This article reports on an effort t...
-
Source: uni-heidelberg.de
Link: https://www.uni-heidelberg.de/md/awi/professuren/behfin/kppt_outcomebias_21dec2017.pdfSource snippet
Heidelberg UniversityOutcome bias in the evaluation of financial agentsDecember 21, 2017 — by C König-Kersting · 2017 · Cited by 74 — Abs...
Published: December 21, 2017
-
Source: evakeiffenheim.substack.com
Title: Eva Keiffenheim How to Engineer Skill in Investing
Link: https://evakeiffenheim.substack.com/p/how-to-engineer-skill-in-investingSource snippet
Read what you actually wrote six months ago, then compare it to what you “remember” thinking. The gap...Read more...
Additional References
-
Source: globalcognition.org
Link: https://www.globalcognition.org/articles/phillips_JDM04_training-decision-skills.pdfSource snippet
Expertise in judgment and decision makingThe naturalistic approach to conducting research around expert judgment is exemplified by the Na...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/26798603_Conditions_for_Intuitive_Expertise -
Source: linkedin.com
Link: https://www.linkedin.com/pulse/summary-daniel-kahneman-gary-kleins-article-called-failure-herbert-zq9beSource snippet
A summary of Daniel Kahneman and Gary Klein's article...The article is based on one simple question: Under what conditions are the intui...
-
Source: youtube.com
Title: Philosophy Roulette 534
Link: https://www.youtube.com/watch?v=99nL3mKs9OgSource snippet
Thinking, Fast and Slow By Daniel Kahneman-Intuition or deliberation? Where you can trust your brain Bite Sized Books · 102 views...
-
Source: kahneman.scholar.princeton.edu
Link: https://kahneman.scholar.princeton.edu/publicationsSource snippet
Daniel Kahneman - Princeton UniversityA select number of articles and book chapters, as well as the entire text of Dr. Kahneman's 1973...
-
Source: youtube.com
Title: Before You Make A Hard Decision, Use A Decision Journal (with example)
Link: http://www.youtube.com/watch?v=MbnfgDyz1tQSource snippet
Thinking, Fast and Slow By Daniel Kahneman-Intuition or deliberation? Where you can trust your brain...
-
Source: youtube.com
Link: http://www.youtube.com/watch?v=qktu0kyo4xkSource snippet
The Hidden Skill Behind Every Expert with Charles Good...
-
Source: youtube.com
Title: The Deliberate Practice Secret Top Performers Use Daily
Link: http://www.youtube.com/watch?v=MUep1zqt0bASource snippet
Philosophy Roulette 534 - Conditions for Intuitive Expertise. Daniel Kahneman & Gary Klein...
-
Source: eatonvance.com
Title: article feedback
Link: https://www.eatonvance.com/content/dam/im/assets/publication/thought-leadership/consilient-observer/article_feedback.pdf?1776816000215=Source snippet
Morgan Stanley Investment ManagementFeedback16 Mar 2022 — Introduction. All outcomes are the result of a process. The outcome alone is an...
-
Source: youtube.com
Title: The Hidden Skill Behind Every Expert with Charles Good
Link: http://www.youtube.com/watch?v=MSwfnoGGYNkSource snippet
The Deliberate Practice Secret Top Performers Use Daily...
Topic Tree