Within Decision Journal
How sure were you, really?
Putting probabilities beside predictions makes confidence testable and helps expose repeated overconfidence or vagueness.
On this page
- Why numbers beat vague confidence words
- Simple probability entries for ordinary decisions
- What repeated forecast errors reveal
Page outline Jump by section
Introduction
A decision journal becomes much more valuable when every important prediction includes a numerical probability. Instead of writing “I’m confident this project will succeed”, you write “I think there is a 70% chance this project will finish on time.” That small change turns a feeling into something that can be tested. Over time, your journal stops being a collection of stories and becomes a record of measurable judgement.
This matters because people are often poor judges of how certain they should be. Research on probabilistic forecasting consistently finds that people tend to express more confidence than their long-run accuracy justifies, particularly on difficult questions. Recording numerical probabilities makes that mismatch visible, allowing repeated review and gradual calibration rather than relying on hindsight or intuition. Warrington College of Business+2Cambridge University Press & Assessment [bear.warrington.ufl.edu]bear.warrington.ufl.eduWarrington College of BusinessOverconfidence in Probability and Frequency Judgmentsby LA BRENNER · 1996 · Cited by 439 — Finally, the ver…
Why numbers beat vague confidence words
Words such as “likely”, “probably” and “almost certain” sound precise, but different people attach very different probabilities to them. Even the same person may use “likely” to mean 60% in one situation and 90% in another. A numerical estimate removes much of this ambiguity. It forces you to state what you actually believe before events unfold.
Numbers also expose overconfidence in a way that qualitative language cannot. Suppose your journal contains fifty predictions marked as 90% likely. If only thirty-five of them come true, your stated confidence was far higher than your actual accuracy. Without numerical probabilities, that pattern would remain hidden because every prediction could later be reinterpreted as merely expressing optimism.
Forecasting researchers describe this relationship between stated confidence and observed accuracy as calibration. A well-calibrated forecaster who assigns 70% probability to many events should find that roughly seven out of ten actually occur. Perfect calibration is difficult, but the concept provides a practical learning target for anyone keeping a decision journal. [Statistical Consulting Services+2Pure]sites.stat.washington.eduProbabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions.Rea…
Simple probability entries for ordinary decisions
Numerical forecasting is useful far beyond finance or geopolitics. Everyday decisions can be recorded with simple percentages that are easy to review later.
For example:
- “There is a 60% chance this client signs the contract before the end of the month.”
- “I estimate a 75% chance that I complete this training course.”
- “There is only a 25% chance that this product feature increases customer retention by at least 5%.”
The exact number matters less than committing to one before the outcome is known. Many experienced forecasters avoid defaulting to round extremes such as 0% or 100%, recognising that genuine certainty is rare outside logical facts or already-observed events. Forecasting tournaments likewise encourage participants to express uncertainty rather than pretending confidence they do not possess. [Sage Journals+2ResearchGate]journals.sagepub.comSage Journals Forecasting TournamentsTetlock, Barbara A….4 Aug 2014 — Forecasting tournaments are level-playing-field competitions that reveal which individuals, teams, or…
Over time, your journal accumulates dozens or hundreds of predictions. That collection is far more informative than any single success or failure because random luck averages out and systematic tendencies become easier to detect.
What repeated forecast errors reveal
The real value appears during review.
Imagine that over one year you recorded:
Probability assignedNumber of predictionsCorrect outcomes90%201470%302250%2011
The pattern suggests consistent overconfidence. Events labelled 90% occurred only 70% of the time, while the 70% forecasts proved correct about 73% of the time. Your judgement is not necessarily poor, but your confidence scale needs adjustment.
This kind of feedback is much harder to dismiss than vague impressions. Instead of asking, “Do I think I’m overconfident?”, you can ask, “When I say 90%, how often am I actually right?”
Repeated reviews also distinguish different kinds of mistakes. You may discover that:
- you are consistently overconfident on unfamiliar topics;
- you underestimate uncertainty in long-term projects;
- your estimates improve when you gather outside information first;
- you become excessively certain after early success.
These are behavioural patterns rather than isolated failures, making them more useful targets for improvement. Studies of forecasting show that repeated outcome feedback helps people improve calibration and reduce overconfidence, although improvement is gradual rather than automatic. Wiley Online Library+2Cambridge University Press & Assessment [onlinelibrary.wiley.com]onlinelibrary.wiley.comWiley Online LibraryCalibration training for improving probabilistic judgments…by R Gruetzemacher · 2024 · Cited by 4 — We describe an…
Measuring whether your confidence is improving
You do not need sophisticated statistics to benefit from probability estimates. Simply grouping past forecasts by confidence level already reveals whether your confidence matches reality.
If you enjoy more quantitative feedback, forecasting research often evaluates predictions with proper scoring rules, especially the Brier score. This score rewards assigning high probabilities to events that occur while penalising unwarranted certainty when predictions fail. Importantly, it encourages forecasters to report their genuine beliefs rather than exaggerating confidence. Lower scores indicate better probabilistic judgement. [American Psychological Association]apa.orgBrier scoring rule is “strictly proper” in the sense that it incentiv- izes forecasters to report their true beliefs—and avoid making fal…
For most personal decision journals, however, calibration charts and simple success rates are sufficient. The objective is not mathematical perfection but honest feedback about how well your expressed confidence matches reality.
Why this improves analytical thinking
Assigning probabilities changes how decisions are made before outcomes occur. Instead of asking whether something will happen, you begin asking how likely it is and what evidence would justify moving from, say, 55% to 75%.
Research from forecasting tournaments led by Philip E. Tetlock and colleagues suggests that better forecasters tend to think in probabilities, update their estimates as new evidence arrives, and treat forecasting as an iterative process rather than a declaration of certainty. Their advantage comes less from extraordinary knowledge than from better calibration and disciplined revision. [Sage Journals+2ResearchGate]journals.sagepub.comSage Journals Forecasting TournamentsTetlock, Barbara A….4 Aug 2014 — Forecasting tournaments are level-playing-field competitions that reveal which individuals, teams, or…
A decision journal that records numerical probabilities therefore serves two purposes simultaneously. Before the decision, it forces clearer thinking about uncertainty. Afterwards, it provides objective evidence about whether your confidence was deserved. That feedback loop is one of the most practical ways to identify recurring overconfidence and gradually replace it with better-calibrated judgement.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: cambridge.org
Link: https://www.cambridge.org/core/journals/judgment-and-decision-making/article/outcome-feedback-reduces-overforecasting-of-inflation-andoverconfidence-in-forecasts/F185AC13F8CDAB43C8105CDF1FC5036FSource snippet
Cambridge University Press & AssessmentOutcome feedback reduces over-forecasting of inflation...by X Niu · 2022 · Cited by 17 — Much res...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/277087515_Identifying_and_Cultivating_Superforecasters_as_a_Method_of_Improving_Probabilistic_PredictionsSource snippet
(PDF) Identifying and Cultivating Superforecasters as a...25 May 2015 — Across a wide range of tasks, research has shown that people mak...
Published: May 2015
-
Source: onlinelibrary.wiley.com
Link: https://onlinelibrary.wiley.com/doi/abs/10.1002/ffo2.177Source snippet
Wiley Online LibraryCalibration training for improving probabilistic judgments...by R Gruetzemacher · 2024 · Cited by 4 — We describe an...
-
Source: researchgate.net
Link: https://www.researchgate.net/figure/Mean-standardized-Brier-scores-for-superforecasters-Supers-and-the-two-comparison_fig1_277087515Source snippet
hich reality is coded as 1 for the event and 0 otherwise), ranging from 0 (...Read more...
-
Source: researchgate.net
Title: 23789125 The disutility of the hard easy effect in choice confidence
Link: https://www.researchgate.net/publication/23789125_The_disutility_of_the_hard-easy_effect_in_choice_confidenceSource snippet
The disutility of the hard-easy effect in choice confidenceJan 6, 2026 — A common finding in confidence research is the hard-easy effect...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/358395522_Outcome_feedback_reduces_over-forecasting_of_inflation_and_overconfidence_in_forecastsSource snippet
January 2022; Judgment and Decision Making 17(1). DOI:10.1017/...Read more...
Published: January 2022
-
Source: bear.warrington.ufl.edu
Link: https://bear.warrington.ufl.edu/brenner/papers/brenner-obhdp1996.pdfSource snippet
Warrington College of BusinessOverconfidence in Probability and Frequency Judgmentsby LA BRENNER · 1996 · Cited by 439 — Finally, the ver...
-
Source: journals.sagepub.com
Title: Sage Journals Forecasting Tournaments
Link: https://journals.sagepub.com/doi/10.1177/0963721414534257Source snippet
Tetlock, Barbara A....4 Aug 2014 — Forecasting tournaments are level-playing-field competitions that reveal which individuals, teams, or...
-
Source: sites.stat.washington.edu
Link: https://sites.stat.washington.edu/raftery/Research/PDF/Gneiting2007jrssb.pdfSource snippet
Probabilistic forecasts of continuous variables take the form of predictive densities or predictive cumulative distribution functions.Rea...
-
Source: apa.org
Link: https://www.apa.org/pubs/journals/releases/xap-0000040.pdfSource snippet
Brier scoring rule is “strictly proper” in the sense that it incentiv- izes forecasters to report their true beliefs—and avoid making fal...
-
Source: mdpi.com
Link: https://www.mdpi.com/journal/forecastingSource snippet
asting published bimonthly online by MDPI...
Additional References
-
Source: goodjudgment.com
Link: https://goodjudgment.com/Source snippet
[Good Judgment]({{ 'good-judgment/' | relative_url }}): See the future sooner with SuperforecastingGood Judgment uses the science of Superforecasting to turn your strategic uncer...
-
Source: philarchive.org
Link: https://philarchive.org/archive/DOROIOv5Source snippet
Overconfidence in OverconfidenceWe believe that subjective probability judgments should be calibrated, whereas Gigerenzer appears unwilli...
-
Source: dlab.berkeley.edu
Link: https://dlab.berkeley.edu/news/predicting-future-harnessing-power-probabilistic-judgements-through-forecasting-tournamentsSource snippet
the Future: Harnessing the Power of...29 Apr 2025 — While the Brier Score remains the standard for many forecasting tournaments, other m...
-
Source: aiimpacts.org
Link: https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project/Source snippet
Brier scores mean less accuracy, so negative correlations are good. “Ravens” is...Read more...
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCCompromising improves forecasting
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC10189590/Source snippet
improves forecasting - PMC - NIHby DN Ferreiro · 2023 · Cited by 4 — Because higher Brier scores indicate lower prediction accuracy we re...
-
Source: learnmoore.org
Title: Overprecision in Judgmentby DA Moore · Cited by 193 —
Link: https://learnmoore.org/mooredata/HOC.pdfSource snippet
Keren, 1988;. Koriat, 2012). Hard questions tend to produce what appears to be overconfidence whereas easy questions produce underconfide...
-
Source: faculty.wharton.upenn.edu
Link: https://faculty.wharton.upenn.edu/wp-content/uploads/2016/11/Long-range-subjective-probability.pdfSource snippet
upenn.eduLong-Range Subjective-Probability Forecasts of Slow-Motion...by PE Tetlock · Cited by 14 — As with calibration, lower refinemen...
-
Source: forum.effectivealtruism.org
Link: https://forum.effectivealtruism.org/posts/pnpnqA4hijnr59p7d/efforts-to-improve-the-accuracy-of-our-judgments-andSource snippet
to Improve the Accuracy of Our Judgments and...25 Oct 2016 — A proper scoring rule, applied to a set of probabilistic judgments or forec...
-
Source: pure.tue.nl
Title: Pure On the calibration of probability judgements
Link: https://pure.tue.nl/ws/files/1436888/612384.pdfSource snippet
Some critical...Jan 1, 1997 — Calibration (or reliability) supposedly measures the accuracy of probability judgments whereas resolution...
-
Source: online.ucpress.edu
Title: Overprecision in the Survey of Professional
Link: https://online.ucpress.edu/collabra/article/10/1/92953/200113/Overprecision-in-the-Survey-of-ProfessionalSource snippet
UC Press OnlineOverprecision in the Survey of Professional Forecasters28 Feb 2024 — Practice with repeated forecasts over time, accompani...
Topic Tree