Keep Score on Your Own Judgement

Introduction

A decision journal is a simple way to keep score on your own judgement. Before making a real choice, you record what you believe, why you believe it, what you expect to happen, how uncertain you are, and when you will review the result. Later, when the outcome is known, you compare the original reasoning with what actually happened. The value is not that the journal guarantees better decisions immediately. Its value is that it protects the original state of uncertainty before memory tidies it up.

This matters because many important choices give weak feedback. A hiring decision, investment, product launch, career move or policy change may look brilliant or foolish months later, but the outcome alone cannot tell you whether the judgement was sound. Research on hindsight bias shows that once people know an outcome, they tend to see it as more predictable than it was beforehand; research on outcome bias shows that people also judge decision quality too heavily by results, even when luck and uncertainty were involved. A decision journal is a practical intervention against both errors: it turns vague memory into a record that can be reviewed. [Massachusetts Institute of Technology]web.mit.edu13 Org Behavior and Human Perf 13 1975 FischhoffMassachusetts Institute of TechnologyI knew it would happen: Remembered probabilities of onceOctober 30, 2009 — Requests for reprints sho…Published: October 30, 2009

Why real choices need a scorecard

Most people review decisions informally. They ask, “Did it work?” and then build a story around the answer. That is understandable, but it is a poor learning system. A good outcome can come from a bad process, and a bad outcome can come from a good process exposed to bad luck. Poker player and decision writer Annie Duke calls the common error “resulting”: treating the result as proof that the choice was wise or stupid. Her broader point is especially relevant to everyday analytical skill: in uncertain situations, decision quality and outcome quality are related, but they are not the same thing. [MarketWatch]marketwatch.comShe introduces a three-step method for assessing speculative markets: establishing a historical base rate for context, assessing whether…

The classic evidence is not just anecdotal. Baruch Fischhoff’s early work on hindsight bias found that outcome knowledge changes how predictable events seem in retrospect. People who already know what happened tend to overestimate how knowable it was beforehand. Fischhoff’s later reflections describe hindsight bias as a persistent problem for learning from the past because it can make uncertainty disappear after the fact. [Massachusetts Institute of Technology]web.mit.edu13 Org Behavior and Human Perf 13 1975 FischhoffMassachusetts Institute of TechnologyI knew it would happen: Remembered probabilities of onceOctober 30, 2009 — Requests for reprints sho…Published: October 30, 2009

Outcome bias adds a second trap. Jonathan Baron and John Hershey’s 1988 studies found that people rated the quality of decisions differently when they knew whether the outcome was good or bad, even when they had the same information that was available to the original decision-maker. A 2023 replication and extension again found that outcomes affect evaluations of decision quality. This is exactly the confusion a decision journal is designed to interrupt: it preserves the ex ante view, meaning the view from before the result was known. [PubMed]pubmed.ncbi.nlm.nih.govPub Med Outcome bias in decision evaluationOutcome bias in decision evaluation - PubMed - NIHby J Baron · 1988 · Cited by 1463 — In 5 studies, undergraduate subjects were giv…

A useful decision journal therefore does two jobs. First, it improves the decision before commitment by making assumptions visible. Secondly, it improves learning after the fact by making the review fairer. It asks, “Given what I knew then, was this a good decision process?” before asking, “Did I like what happened?”

What to record before deciding

The journal entry should be short enough that you will actually use it, but specific enough that future-you cannot wriggle away from it. Farnam Street’s widely used decision-journal template frames the practice as “quality control” for thinking and recommends writing down the situation, the decision, the variables, the expected outcomes, probabilities and later results. The most important design principle is precision: vague words create escape hatches, while clear predictions create reviewable evidence. [Farnam Street]fs.blogFarnam Street Decision Journal: Template and Example IncludedFarnam Street Decision Journal: Template and Example Included

For consequential decisions, record these items before acting: [cognitivebiaslab.com]cognitivebiaslab.comHindsight BiasHindsight Bias

1. The decision and deadline. State the choice in a sentence: “Hire candidate A for the operations role,” “Continue the project for one more quarter,” or “Move savings from cash into a diversified index fund.” Include the date and the point at which the decision becomes hard to reverse.
2. The live alternatives. List the serious options you rejected, not just the one you chose. This prevents the review from becoming a lazy comparison between the chosen path and an imaginary perfect path.
3. The reasons for the choice. Write the main argument in plain language. Include the evidence that would support the choice and the evidence that made you hesitate.
4. The assumptions that matter most. Identify what must be true for the decision to work. For a product launch, that might be demand, distribution, pricing or team capacity. For a career move, it might be learning curve, manager quality, commute, salary stability or family strain.

1. Predictions and probabilities. Convert expectations into checkable forecasts where possible: “There is a 60% chance this role will improve my income within 12 months,” or “I expect customer churn to fall from 8% to below 6% by the end of quarter two.” Forecasting research uses probability scores such as the Brier score to compare predicted probabilities with outcomes; an ordinary personal journal does not need tournament-level maths, but it does need numbers often enough to reveal overconfidence. Good Judgment Open

2. Confidence and emotional state. Note whether you feel rushed, defensive, excited, afraid of missing out, or under social pressure. This is not therapy language for its own sake; it is data about the conditions under which your judgement may drift.
3. A review date. Decide in advance when the outcome will be informative. Some decisions should be reviewed in two weeks; others need three months, a year, or staged reviews.

A strong entry does not need to be long. The habit works because it captures the forecast while the uncertainty is still alive. Once the outcome arrives, the mind is tempted to replace the messy original situation with a cleaner story.

A practical entry for a real choice

Imagine a manager deciding whether to promote a strong individual contributor into a team-lead role. A weak review would simply ask six months later, “Did the promotion work?” If the person succeeds, the manager may remember the choice as obvious. If the person struggles, the manager may remember the warning signs as obvious. Neither memory is reliable.

[A decision-journal entry would make the review sharper:](#endnote-7 “

Decision: Promote Maya to team lead from 1 September rather than hiring externally.

Alternatives considered: External hire with more management experience; delay promotion for six months while giving Maya a trial leadership assignment.

Reasons: Maya has strong technical credibility, high trust in the team, and has informally mentored two colleagues. External hiring would take longer and risks cultural mismatch.

Main uncertainties: Maya has not handled performance conflict; the team is growing quickly; she may struggle to move from peer to manager.

Predictions: 70% chance the team’s delivery reliability improves or stays stable over six months; 50% chance Maya reports high stress in the first quarter; 30% chance an external coach or formal management training becomes necessary.

Review date: Three months for early stress and support needs; six months for team outcomes.

This entry changes the later conversation. If Maya struggles with conflict, the review should not simply say, “Bad promotion.” The risk was identified in advance. The better question is whether the manager gave enough support for a known risk. If the team performs well despite early stress, the review should not erase the uncertainty either. The journal keeps the lesson specific: the judgement about trust and technical credibility may have been sound, while the support plan may have been underbuilt.

How to review outcomes later

A decision review should separate three questions that people often collapse into one:

• Process: Did I use the information available at the time well?
• Prediction: Which expectations were accurate, too optimistic, too pessimistic or too vague?
• Outcome: What actually happened, and what role did luck, timing or outside factors play?

This distinction is crucial because feedback from real life is often noisy. In the Good Judgment Project, forecasters improved through a system that made predictions explicit, scored them, and used training, teaming and aggregation to improve accuracy. A personal decision journal borrows the same spirit at a smaller scale: make forecasts visible, revisit them, and learn from patterns rather than isolated wins or losses. Good Judgment

A good review begins by rereading the original entry before writing any new interpretation. Then mark each prediction as broadly right, broadly wrong, too vague to assess, or not yet mature. The “too vague” category is not a failure; it is feedback that the next entry needs sharper wording.

Next, classify the outcome:

• Good process, good outcome: The decision probably deserves reinforcement, though not hero worship.
• Good process, bad outcome: Look for bad luck, hidden variables, or a risk that was known but accepted.
• Bad process, good outcome: Treat this as a warning, not a triumph. You may have been lucky.
• Bad process, bad outcome: Identify the preventable failure, but avoid pretending it was all obvious at the time.

The review should end with one or two changes to future practice. For example: “For senior hires, I will always include a work-sample test,” or “For investments, I will write the sell conditions before buying.” Without a behavioural change, a decision journal becomes a diary of regret rather than a tool for improving judgement.

Avoiding hindsight bias in reviews

Hindsight bias is powerful because it feels like understanding. Once an outcome is known, the mind can easily produce a story in which the clues were there all along. The problem is not that hindsight is useless; it is that hindsight must be disciplined. Outcome knowledge can contaminate memory of what was known, what was believed and what was realistically foreseeable. Cambridge University Press & Assessment

[A decision journal reduces that contamination in three ways.](#endnote-7 “

First, it anchors memory. Instead of asking, “What do I think I thought?”, you can see what you actually wrote. This is especially useful for decisions that unfold over months, where confidence and reasons change gradually.

Secondly, it protects uncertainty. A 55% forecast that comes true was not a certainty. A 70% forecast that fails was not necessarily foolish. The journal keeps probability language alive after the event.

Thirdly, it exposes recurring patterns. One bad call may be noise. Ten entries may reveal that you repeatedly underestimate implementation time, overtrust charismatic people, ignore base rates, or become too cautious after a visible failure.

Gary Klein’s premortem method is a useful companion before major projects: the team imagines that a project has failed and works backwards to identify plausible reasons. McKinsey’s discussion of premortems frames the method as a way to counter overconfidence and surface risks early. A decision journal serves a different but compatible purpose: the premortem improves the decision before launch; the journal preserves the reasoning so the later review does not become fiction. McKinsey & Company

What makes decision journals work in practice

The practice fails when it is too elaborate, too moralistic or too focused on proving who was right. It works best when it is treated as a lightweight feedback system for judgement.

Choose decision clusters, not every choice. Journal decisions that repeat often enough to teach you something: hiring, investments, project approvals, negotiations, medical choices, major purchases, career moves, team strategy or public commitments. Recording every minor choice creates fatigue and noise.

Write before the decision is final. A journal written after commitment is already contaminated by justification. The entry should happen before the action, ideally when alternatives are still live.

Use plain language. “Strategic alignment risk” is less useful than “The sales team may not prioritise this because their bonuses reward the old product.” Clear wording makes later review harder to evade.

Include disconfirming evidence. Kahneman, Lovallo and Sibony’s work on decision quality control for organisations stresses the need to challenge proposals, examine alternatives and test whether the numbers are grounded. A personal journal should do the same in miniature: write down what would make the choice wrong, not just what supports it. execed.hkubs.hku.hk

Review on a calendar, not when emotion peaks. If you review only after triumphs or disasters, the sample will be distorted. A scheduled review date makes learning less reactive.

Look for calibration, not perfection. The aim is not to predict everything correctly. It is to become better calibrated: to have 70% confidence mean something like “right about seven times in ten” across many entries. Forecasting platforms such as Good Judgment Open use scoring systems to create feedback on probabilistic judgement; a personal decision journal can use simpler tallies while preserving the same principle of keeping score. Good Judgment Open

Common failure modes

The most common mistake is turning the journal into a justification memo. If the entry reads like a lawyer defending a choice, it will not teach much later. The better tone is curious and provisional: “Here is what I think, here is why, and here is where I might be wrong.”

Another failure is reviewing too soon. Some decisions produce early signals that are misleading. A new hire may look slow in the first month and excellent by month six. A project may look successful at launch but fail to retain users. The review date should match the decision’s feedback cycle.

A third failure is scoring only the final result. Suppose an investor buys a stock because they expect a product release to drive revenue growth. The stock rises because of an unrelated acquisition rumour. The outcome is good, but the thesis was not validated. The journal should record that distinction. The useful lesson is not “I am a good investor”; it may be “I was lucky, and my original causal story remains unproven.”

A fourth failure is ignoring base rates. If a decision journal repeatedly records inside-view stories — “this project is different”, “this candidate is exceptional”, “this market will respond faster than usual” — but never compares them with reference-class evidence, it may preserve bias rather than reduce it. The journal should ask: what usually happens in choices like this?

When a decision journal is worth the effort

A decision journal is most valuable when decisions are consequential, uncertain, repeatable and reviewable. It is less useful for one-off choices with no clear feedback or trivial choices where the administrative cost outweighs the learning.

It is especially useful in four settings:

• Career and hiring decisions, where outcomes are delayed and personality impressions can dominate evidence.
• Investing and financial choices, where luck, market movement and overconfidence can blur the link between process and result.
• Product and strategy decisions, where teams often remember the final narrative more clearly than the original uncertainty.
• Personal life choices with repeated patterns, such as commitments, relocations, major purchases or health routines.

The best reason to keep one is not self-criticism. It is self-honesty. Better thinking depends on feedback, but real life rarely provides clean feedback by itself. A decision journal creates a small feedback loop around real choices: what you knew, what you expected, what happened, and what that should change next time.

Amazon book picks

Further Reading

Books and field guides related to Keep Score on Your Own Judgement. Use these as the next step if you want deeper reading beyond the article.

Book

Thinking in Bets

By Annie Duke

Explains decision quality, uncertainty, resulting, and decision journals.

See on Amazon

Book

Superforecasting

By Philip Tetlock, Dan Gardner

Focuses on tracking predictions, calibration, and improving judgment.

See on Amazon

Book

Noise

By Daniel Kahneman, Olivier Sibony et al.

Covers decision quality, judgment errors, and better evaluation processes.

See on Amazon

Book

The Signal and the Noise

By Nate Silver

Shows how probabilistic thinking improves real-world decisions.

See on Amazon

Browse more on Amazon: Thinking in Bets Superforecasting Noise

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

Using USA USA USAUKCanadaAustraliaIreland

Example eBay listing

A Truly Amazing English Teacher Notebook // Journal Thank You Gift for Englis...

Search eBay.co.uk: decision journal notebook

Browse similar on eBay.co.uk

Example eBay listing

Debate It! - Decision Journal | Make better decisions | Structured and logical

Search eBay.co.uk: decision journal notebook

Browse similar on eBay.co.uk

Example eBay listing

A Truly Amazing English Teacher: Notebook // Journal: Thank You Gift for English

Search eBay.co.uk: decision journal notebook

Browse similar on eBay.co.uk

Browse more on eBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: marketwatch.com
   Link: https://www.marketwatch.com/story/hold-stocks-or-fold-em-former-world-series-of-poker-champion-annie-duke-tells-you-how-to-decide-6ded9e52
   Source snippet

   She introduces a three-step method for assessing speculative markets: establishing a historical base rate for context, assessing whether...

2. Source: resolve.cambridge.org
   Title: hindsight bias
   Link: https://resolve.cambridge.org/core/services/aop-cambridge-core/content/view/D1116404C680ED2CA1F9286FF1183483/9780511574894c4_p62-77_CBO.pdf/hindsight-bias.pdf

3. Source: mckinsey.com
   Title: Mc Kinsey & Company Bias Busters: Premortems: Being smart at the start
   Link: https://www.mckinsey.com/capabilities/strategy-and-corporate-finance/our-insights/bias-busters-premortems-being-smart-at-the-start

4. Source: execed.hkubs.hku.hk
   Title: Before You Make That Big Decision
   Link: https://execed.hkubs.hku.hk/wp-content/uploads/2023/04/Harvard-Business-Review-Before-You-Make-That-Big-Decision.pdf

5. Source: mckinsey.de
   Link: https://www.mckinsey.de/~/media/McKinsey/Business%20Functions/Strategy%20and%20Corporate%20Finance/Our%20Insights/Making%20great%20decisions/Making%20great%20decisions.pdf

6. Source: cambridge.org
   Title: recalibrating probabilistic forecasts to improve theiraccuracy
   Link: https://www.cambridge.org/core/services/aop-cambridge-core/content/view/5D8F0DFC90FC1E9B5C38C12992A6C6A7/S1930297500009049a.pdf/recalibrating_probabilistic_forecasts_to_improve_theiraccuracy.pdf

7. Source: youtube.com
   Title: KEEP A DECISION JOURNAL
   Link: https://www.youtube.com/watch?v=gsSkijSm0YE
   Source snippet

   Hindsight Bias...

8. Source: web.mit.edu
   Title: 13 Org Behavior and Human Perf 13 1975 Fischhoff
   Link: https://web.mit.edu/curhan/www/docs/Articles/biases/13_Org_Behavior_and_Human_Perf_13_1975_Fischhoff.pdf
   Source snippet

   Massachusetts Institute of TechnologyI knew it would happen: Remembered probabilities of onceOctober 30, 2009 — Requests for reprints sho...

   Published: October 30, 2009

9. Source: pubmed.ncbi.nlm.nih.gov
   Title: Pub Med Outcome bias in decision evaluation
   Link: https://pubmed.ncbi.nlm.nih.gov/3367280/
   Source snippet

   Outcome bias in decision evaluation - PubMed - NIHby J Baron · 1988 · Cited by 1463 — In 5 studies, undergraduate subjects were giv...

10. Source: pubmed.ncbi.nlm.nih.gov
    Link: https://pubmed.ncbi.nlm.nih.gov/39913490/

11. Source: fs.blog
    Title: Farnam Street Decision Journal: Template and Example Included
    Link: https://fs.blog/decision-journal/

12. Source: pmc.ncbi.nlm.nih.gov
    Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8763848/

13. Source: pubmed.ncbi.nlm.nih.gov
    Link: https://pubmed.ncbi.nlm.nih.gov/14645895/

14. Source: pmc.ncbi.nlm.nih.gov
    Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8763826/

15. Source: pmc.ncbi.nlm.nih.gov
    Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC12372742/

16. Source: pmc.ncbi.nlm.nih.gov
    Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC12818272/

17. Source: pmc.ncbi.nlm.nih.gov
    Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC10189590/

18. Source: Wikipedia
    Title: Outcome bias
    Link: https://en.wikipedia.org/wiki/Outcome_bias

19. Source: Wikipedia
    Title: The Good Judgment Project
    Link: https://en.wikipedia.org/wiki/The_Good_Judgment_Project

20. Source: Wikipedia
    Title: Brier score
    Link: https://en.wikipedia.org/wiki/Brier_score

21. Source: decision-mastery.com
    Title: decision journal
    Link: https://www.decision-mastery.com/articles/decision-journal

22. Source: dataopsschool.com
    Title: brier score
    Link: https://dataopsschool.com/blog/brier-score/

23. Source: fs.blog
    Link: https://fs.blog/smart-decisions/

24. Source: coopilots.io
    Title: decision journal
    Link: https://www.coopilots.io/free-resources/decision-journal

25. Source: lesswrong.com
    Title: hindsight bias
    Link: https://www.lesswrong.com/lw/il/hindsight_bias

26. Source: atlassian.com
    Title: decision journal
    Link: https://www.atlassian.com/blog/productivity/decision-journal

27. Source: elearningspecialist.com
    Title: Premortem method
    Link: https://elearningspecialist.com/en/blog/premortem-method-a-simple-and-probably-effective-approach-to-strengthening-the-safety-culture/

28. Source: cognitivebiaslab.com
    Title: Hindsight Bias
    Link: https://www.cognitivebiaslab.com/bias/bias-hindsight/

29. Source: theinnovationshow.io
    Title: Annie Duke
    Link: https://theinnovationshow.io/episode/annie-duke-thinking-in-bets/

Additional References

1. Source: gq.com
   Link: https://www.gq.com/story/annie-duke-thinking-in-bets-how-to-be-wrong-interview
   Source snippet

   Duke also discusses how humans struggle with probabilities and the concept of agency, as exemplified by the harsh criticisms of decision...

2. Source: psnet.ahrq.gov
   Link: https://psnet.ahrq.gov/issue/hindsight-foresight-effect-outcome-knowledge-judgment-under-uncertainty

3. Source: researchgate.net
   Link: https://www.researchgate.net/publication/51453002_Before_you_make_that_big_decision

4. Source: researchgate.net
   Link: https://www.researchgate.net/publication/316470642_Cognitive_Biases_in_Decision_Making_in_Post-Bureaucratic_Organizations

5. Source: academia.edu
   Link: https://www.academia.edu/31130287/Investment_Decision_Making_and_Hindsight_Bias

6. Source: researchgate.net
   Link: https://www.researchgate.net/publication/3229642_Performing_a_Project_Premortem

7. Source: researchgate.net
   Link: https://www.researchgate.net/publication/314081281_Reducing_Biases_of_Decision-Making_Processes_in_Complex_Organizations

8. Source: openreview.net
   Link: https://openreview.net/forum?id=iFF-zKCgzS

9. Source: hbs.edu
   Link: https://www.hbs.edu/faculty/Pages/item.aspx?num=39958

10. Source: merriam-webster.com
    Link: https://www.merriam-webster.com/dictionary/decision