Within Feedback
When Experience Makes Judgement Worse
In noisy or delayed settings, repeated experience can build confidence without producing reliable learning.
On this page
- Why some domains do not teach cleanly
- Valid cues, fast feedback and intuition
- Designing safeguards for noisy decisions
Page outline Jump by section
Introduction
Experience does not automatically produce better judgement. In some domains, repeating the same task for years can increase confidence while leaving accuracy unchanged—or even making it worse. The reason is not a lack of intelligence or effort but the quality of the feedback the environment provides. When outcomes are delayed, noisy, selective or misleading, people struggle to tell whether a decision was genuinely good or merely fortunate. Over time, lucky successes can reinforce poor habits, while sound decisions that happen to end badly may be abandoned.
This distinction is central to improving thinking and analytical skills. Research on judgement, expertise and forecasting shows that intuition becomes reliable only when people repeatedly encounter valid cues and receive prompt, accurate feedback about whether those cues predicted reality. Where those conditions are absent, experience must be supplemented with deliberate safeguards rather than trusted on its own.[AdaptLX]preview-prod-production.gurooproducer.comAdaptLXThe Two Settings of Kind and Wicked Learning EnvironmentsAugust 13, 2021 — by RM Hogarth · 2015 · Cited by 255 — Hogarth (2001) us…
Why Some Domains Do Not Teach Cleanly
Psychologist Robin Hogarth described the contrast between kind and wicked learning environments. A kind environment has stable rules, recurring patterns and clear feedback. Chess is a classic example: every move produces visible consequences, mistakes can be analysed immediately and similar situations recur frequently.
A wicked environment has the opposite characteristics. Rules may change, many factors interact simultaneously, and feedback is delayed, incomplete or distorted. Financial markets, long-term business strategy, public policy, hiring decisions and geopolitical forecasting often fall into this category. Because cause and effect are difficult to separate, people receive little trustworthy information about which aspects of their judgement actually deserve credit or blame.[AdaptLX]preview-prod-production.gurooproducer.comAdaptLXThe Two Settings of Kind and Wicked Learning EnvironmentsAugust 13, 2021 — by RM Hogarth · 2015 · Cited by 255 — Hogarth (2001) us…
The problem is not simply uncertainty. Many uncertain environments still teach effectively if outcomes are observed quickly and consistently. The deeper issue is that the feedback itself is unreliable. A poor decision may succeed because of luck, while a well-reasoned decision may fail because of external events beyond anyone’s control.
How Experience Can Become Misleading
Weak feedback environments create several systematic traps.
Luck masquerades as skill. When outcomes depend heavily on chance, repeated success can convince people they possess expertise they have not actually demonstrated. Conversely, repeated bad luck can discourage effective methods before they have had enough opportunities to prove themselves.
Delayed consequences weaken learning. If months or years separate judgement from outcome, it becomes difficult to remember the original assumptions accurately. People naturally reconstruct their reasoning in ways that fit what eventually happened, making genuine learning much harder.
Selective feedback hides mistakes. Many professionals never discover the decisions that turned out poorly. A physician may never learn about a patient who sought treatment elsewhere. A recruiter rarely sees how rejected applicants would have performed. Investors only observe the investments they actually made, not the alternatives they ignored.
Changing environments invalidate experience. Experience acquired under one set of conditions may become misleading when technology, incentives or competitors change. Past success then reflects historical circumstances rather than enduring judgement.
Hogarth and colleagues argue that many judgement failures arise because the environment used for learning differs systematically from the environment in which decisions are later applied. Experience is not merely incomplete; it can become actively biased.[AdaptLX]preview-prod-production.gurooproducer.comAdaptLXThe Two Settings of Kind and Wicked Learning EnvironmentsAugust 13, 2021 — by RM Hogarth · 2015 · Cited by 255 — Hogarth (2001) us…
Valid Cues, Fast Feedback and Intuition
Not all intuition deserves equal trust. Daniel Kahneman and Gary Klein’s influential collaboration helped reconcile two traditions in psychology by identifying the conditions under which intuitive expertise develops.
Reliable intuition requires two ingredients:
- Valid environmental cues, where observable patterns genuinely predict future outcomes.
- Frequent, accurate feedback, allowing repeated correction of errors.
Without both conditions, intuition has little opportunity to calibrate itself. A firefighter repeatedly exposed to recognisable physical patterns can often develop highly accurate instinctive judgement. A long-term macroeconomic forecaster receives far fewer opportunities to connect specific judgements with unambiguous outcomes.
This explains why experts may display extraordinary performance in one domain while showing little advantage over informed novices in another. Years of experience alone do not guarantee expertise if the environment never provides the information needed to improve.[Range Widely]davidepstein.substack.comRange Widely"Kind" and "Wicked" Learning EnvironmentsIn what Hogarth called “kind” learning environments, experience led to predictable i…
Historical Examples of Misleading Experience
One striking illustration comes from medicine. Robin Hogarth recounts a physician renowned for diagnosing typhoid fever at an unusually early stage by examining patients’ tongues manually. His apparent success reinforced confidence in his diagnostic technique. In reality, he had unknowingly become an efficient carrier of the disease himself, infecting patients during examinations. Positive feedback consistently reinforced the wrong causal explanation.[AdaptLX]preview-prod-production.gurooproducer.comAdaptLXThe Two Settings of Kind and Wicked Learning EnvironmentsAugust 13, 2021 — by RM Hogarth · 2015 · Cited by 255 — Hogarth (2001) us…
Avalanche safety provides another example. Skiers can repeatedly make hazardous route choices without triggering an avalanche. Every safe outing appears to confirm the quality of their judgement, even though survival may largely reflect favourable conditions rather than sound decision-making. The first unmistakable feedback may arrive only when an avalanche finally occurs—far too late to support gradual learning. Avalanche educators increasingly describe this as a textbook wicked learning environment.[avalanchejournal.ca]avalanchejournal.caWicked Learning - Avalanche Journalby B Roach — This concept is that in avalanche terrain, we often lack the correct feedback to make eff…
Financial markets exhibit similar dynamics. Investors often experience extended winning streaks during favourable market conditions, encouraging excessive confidence. Later losses may reflect changing conditions rather than a sudden decline in skill, making it difficult to distinguish robust strategies from temporary good fortune.[ajinkyabhat.com]ajinkyabhat.comKind and Wicked Learning EnvironmentsTypically, these are also easier for…Read more…
Why Confidence Often Increases Faster Than Accuracy
Weak feedback environments produce an uncomfortable asymmetry: confidence accumulates more easily than knowledge.
Every successful outcome feels like evidence supporting existing beliefs. Failed predictions, however, can often be explained away as exceptional circumstances, unusual bad luck or unforeseeable events. Because many decisions lack clear counterfactuals—what would have happened had another option been chosen—people rarely observe decisive evidence against their preferred explanations.
This asymmetry helps explain why experienced professionals can sincerely disagree for decades while each believes experience has validated their judgement. Both individuals have accumulated stories consistent with their beliefs, but neither has received sufficiently diagnostic feedback to identify which interpretation is more accurate.
Designing Safeguards for Noisy Decisions
Although weak feedback environments cannot be transformed into perfect teachers, decision-makers can improve the quality of the feedback they receive.
Record predictions before outcomes. Writing down expected probabilities, assumptions and confidence levels creates an objective reference point that later outcomes cannot rewrite.
Evaluate the reasoning, not only the result. A good decision process sometimes produces bad outcomes because of chance. Likewise, poor reasoning occasionally succeeds through luck. Reviewing process separately from outcome reduces outcome bias.
Seek faster proxy measures. Where ultimate outcomes take years, identify earlier indicators that genuinely correlate with long-term success. Care is needed to ensure these proxies themselves remain valid.
Use large samples instead of memorable anecdotes. Individual experiences are often dominated by randomness. Aggregating many decisions provides a more reliable picture of calibration and accuracy.
Invite independent review. Colleagues who examine the reasoning before outcomes are known can identify assumptions and overlooked alternatives without hindsight contamination.
Look for missing data. Ask not only what happened, but what could not be observed. Decisions often generate systematic blind spots because rejected options, avoided risks or unchosen alternatives never produce directly comparable evidence.
These safeguards do not eliminate uncertainty, but they help replace misleading experience with more informative feedback.
Recognising When to Distrust Experience
A useful diagnostic question is not, “How much experience do I have?” but, “How well has this environment taught me?”
Experience deserves greater trust when:
- the same patterns recur regularly;
- outcomes follow reasonably quickly;
- mistakes become visible;
- alternative explanations can be tested; and
- rules remain relatively stable.
Greater scepticism is warranted when:
- outcomes depend heavily on chance;
- consequences emerge only after long delays;
- failures remain hidden;
- successful outcomes have many possible causes; or
- the environment changes faster than experience accumulates.
The practical lesson is that experience is evidence, not proof. In weak feedback environments, the quality of learning depends less on the number of years spent making decisions than on the quality of the feedback loop those years actually provided.[AdaptLX+2Range Widely]preview-prod-production.gurooproducer.comAdaptLXThe Two Settings of Kind and Wicked Learning EnvironmentsAugust 13, 2021 — by RM Hogarth · 2015 · Cited by 255 — Hogarth (2001) us…
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
1.
Source: avalanchejournal.ca
Link:https://avalanchejournal.ca/wicked-learning/
Source snippet
Wicked Learning - Avalanche Journalby B Roach — This concept is that in avalanche terrain, we often lack the correct feedback to make eff...
2.
Source: ajinkyabhat.com
Title: Kind and Wicked Learning Environments
Link:https://ajinkyabhat.com/blog/kind-and-wicked-learning-environments
Source snippet
Typically, these are also easier for...Read more...
3.
Source: preview-prod-production.gurooproducer.com
Link:https://preview-prod-production.gurooproducer.com/learntolead2021/hogarth-et-al-2015.pdf
Source snippet
AdaptLXThe Two Settings of Kind and Wicked Learning EnvironmentsAugust 13, 2021 — by RM Hogarth · 2015 · Cited by 255 — Hogarth (2001) us...
Published: August 13, 2021
4.
Source: davidepstein.substack.com
Link:https://davidepstein.substack.com/p/kind-and-wicked-learning-environments
Source snippet
Range Widely"Kind" and "Wicked" Learning EnvironmentsIn what Hogarth called “kind” learning environments, experience led to predictable i...
5.
Source: davidepstein.substack.com
Link:https://davidepstein.substack.com/p/kind-and-wicked-learning-environments/comments
Source snippet
"Kind" and "Wicked" Learning Environments16 May 2024 — People struggle to navigate wicked environments not just because of heuristics and...
Published: May 2024
Additional References
6.
Source: researchgate.net
Link:https://www.researchgate.net/publication/283828207_The_Two_Settings_of_Kind_and_Wicked_Learning_Environments
Source snippet
The Two Settings of Kind and Wicked Learning EnvironmentsKind learning environments involve close matches between the informational eleme...
7.
Source: thinklikesocrates.com
Title: decision making under uncertainty
Link:https://thinklikesocrates.com/2020/03/02/decision-making-under-uncertainty/
Source snippet
feedback loop is often poor, missing or worse, misleading at times. These are referred to as 'wicked learning environments'. Contrast tha...
8.
Source: youtube.com
Link:https://www.youtube.com/watch?v=jH2ONhN0y-0
Source snippet
Why Generalists Triumph: Key Lessons from Range by David Epstein...
9.
Source: pmc.ncbi.nlm.nih.gov
Title: Learning to learn about uncertain feedback
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC4749830/
Source snippet
We should ignore noise but shift strategy after rule changes...
10.
Source: youtube.com
Title: The Myth of Experience: Kind and Wicked Learning Environments
Link:https://www.youtube.com/watch?v=KDWfsxfRKXc
Source snippet
Productivity Expert: The Counterintuitive Secret to Getting More Done | David Epstein...
11.
Source: youtube.com
Title: Why Generalists Triumph: Key Lessons from Range by David Epstein
Link:https://www.youtube.com/watch?v=mmwoQ79lZH8
Source snippet
What's irrelevant? - The Myth of Experience #3...
12.
Source: youtube.com
Title: Kind vs. Wicked
Link:https://www.youtube.com/watch?v=uFbxL2JTa6c
Source snippet
The Myth of Experience: Kind and Wicked Learning Environments...
13.
Source: youtube.com
Title: What’s irrelevant?
Link:https://www.youtube.com/watch?v=iUXiLMlDSJg
Topic Tree