Why Probably Is Not Precise Enough

Introduction

Confidence calibration improves when you stop saying “probably”, “unlikely” or “I’m fairly sure” and instead assign explicit probabilities. Numbers force you to state how strongly you believe a claim and make it possible to compare that belief with what eventually happens. If you record that an event has a 70% chance of occurring, you can later ask whether events you rated around 70% actually happened about seven times out of ten. Vague language cannot be tested in the same way because different people often attach very different meanings to the same words. Research in psychology, forecasting and intelligence analysis consistently shows that numerical probabilities reduce ambiguity, improve communication and create better feedback for learning.[ResearchGate]researchgate.netResearch Gate(PDF) Words or Numbers?Communicating Probability in…July 23, 2020 — 8 Apr 2020 — We review the benefits and drawbacks of this approach, drawing on psychologi…Published: July 23, 2020

Within a prediction journal, replacing verbal confidence with numbers is one of the simplest changes that produces measurable improvement. It turns confidence from a feeling into something that can be evaluated, criticised and refined over time.

Why vague confidence words hide disagreement

Words that sound precise are often interpreted very differently by different people. If two colleagues both describe a project as “likely”, one may mean there is roughly a 60% chance of success while the other imagines something closer to 90%. They appear to agree while holding substantially different beliefs.

Studies of linguistic probability repeatedly find wide variation in how people interpret common expressions such as “likely”, “possible” and “unlikely”. Context matters as well. The same word can receive different numerical interpretations depending on whether it refers to a medical diagnosis, a business forecast or an intelligence assessment.[Sage Journals]journals.sagepub.comOpen source on sagepub.com.

This ambiguity creates several practical problems.

• False agreement. People think they share the same expectation when they do not.
• Difficult accountability. After the outcome is known, almost any verbal prediction can be reinterpreted.
• Poor feedback. You cannot easily evaluate whether “probably” was appropriately confident because no measurable prediction was ever recorded.
• Hidden overconfidence. Strong emotional conviction can hide behind soft language that sounds appropriately cautious.

Numerical probabilities expose these differences immediately. Saying “I think there is a 62% chance” communicates a noticeably different judgement from saying “I think there is an 88% chance”, even though both might otherwise have been described simply as “likely”.

How to choose practical probability ranges

The goal is not to pretend you can estimate every probability with scientific precision. Instead, probabilities should express the strength of your current belief as honestly as possible.

Many experienced forecasters avoid assigning arbitrary decimal precision unless they genuinely believe it is meaningful. Practical ranges often communicate uncertainty more faithfully than pretending to distinguish between, for example, 73% and 74%.

A useful working scale is:

ProbabilityPractical interpretation50%Evidence is balanced; either outcome seems equally plausible.60%Slightly more likely than not, but substantial doubt remains.70%A clear favourite, though failure is still common.80%Strong expectation with meaningful uncertainty remaining.90%Very confident, reserved for unusually strong evidence.95% or aboveShould be used sparingly because errors become especially informative.

Some people prefer fixed probability bands such as 5%, 20%, 40%, 60%, 80% and 95%. Others use increments of five percentage points. The exact system matters less than using it consistently enough that your forecasts can later be compared with reality.

When evidence is weak, saying “I don’t know” often corresponds to assigning a probability near 50%, not because both outcomes are equally attractive, but because your available information does not justify greater confidence.

When 60% and 85% mean different beliefs

The difference between 60% and 85% is much larger than it first appears.

A 60% prediction says that failure is expected almost two times in five. Someone hearing that forecast should prepare seriously for both outcomes.

An 85% prediction implies failure should occur only about once in every six or seven similar cases. Acting as though these two probabilities express the same confidence leads to poor decisions.

Imagine three project managers estimating whether a software release will meet its deadline.

All three might casually say the release will “probably” ship on time. Yet their expectations are dramatically different. Manager C is effectively saying that missing the deadline would be an unusual surprise, while Manager A expects delays to remain entirely plausible.

Recording numerical probabilities makes these differences visible before events unfold instead of allowing everyone to remember having held the “correct” opinion afterwards.

Why prediction journals benefit from numbers

A prediction journal works because it creates repeated opportunities for calibration.

Suppose you record one hundred forecasts over a year.

If your 90% predictions fail one-third of the time, you are systematically overconfident. If your 60% predictions succeed almost every time, you may be underconfident. Numerical forecasts reveal these patterns in a way verbal forecasts never can.

This is also why scoring methods such as the Brier score evaluate probabilistic forecasts rather than simple right-or-wrong answers. They reward assigning probabilities that accurately reflect real uncertainty instead of expressing excessive certainty.[Harvard App Extension]appext.hks.harvard.edu1964-1994 found that 96 percent of key judgments expressed probability in ways that…Read more…

Lessons from fields that depend on uncertainty

Several professions have adopted numerical probability systems because misunderstanding uncertainty can have serious consequences.

Intelligence organisations have spent decades studying “words of estimative probability” after discovering that policymakers interpreted expressions such as “likely” and “probable” very differently. Numerous organisations now pair verbal expressions with numerical ranges or encourage direct numerical estimates to reduce misunderstanding.[Gwern+2Sage Journals]gwern.netVerbal Probability Expressions In National IntelligenceVerbal Probability Expressions In National Intelligence…May 12, 2008 — 2 May 2008 — Sherman Kent's classic work “Words of Estimat…Published: May 12, 2008

Climate science follows a similar approach. Reports by the Intergovernmental Panel on Climate Change (IPCC) attach explicit probability ranges to phrases such as “very likely” or “likely” so that readers share a common interpretation rather than relying on personal intuition.[Wikipedia]WikipediaWords of estimative probabilityWords of estimative probability

These examples illustrate a broader principle: whenever important decisions depend on uncertain information, standardised probabilities communicate more reliably than ordinary language alone.

Practical habits that improve probability estimates

Using numerical probabilities becomes easier with regular practice.

Over time, this habit changes how uncertainty feels. Instead of asking whether something is “likely”, you begin asking exactly how likely it is and what evidence would justify moving that estimate higher or lower. That shift is the core mechanism through which replacing vague confidence words with probabilities improves judgement.

Amazon book picks

Further Reading

Books and field guides related to Why Probably Is Not Precise Enough. Use these as the next step if you want deeper reading beyond the article.

Book

Superforecasting

By Philip Tetlock, Dan Gardner

Emphasizes explicit probabilities rather than vague confidence language.

See on Amazon

Book

How to Measure Anything

By Douglas W. Hubbard

Teaches translating subjective judgments into numerical estimates.

See on Amazon

Book

Thinking in Bets

By Annie Duke

Shows how to express beliefs probabilistically and update them.

See on Amazon

Book

Thinking, Fast and Slow

By Daniel Kahneman

Provides the psychological foundation for improving probabilistic judgment.

See on Amazon

Browse more on Amazon: Superforecasting How to Measure Anything Thinking in Bets

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Top pick

critical thinking card game

Open a targeted eBay search for items related to this topic.

CriticalThinkingCard

Browse eBay

Related search

critical thinking game

Open a targeted eBay search for items related to this topic.

CriticalThinkingGame

Browse eBay

Related search

not poster

Open a targeted eBay search for items related to this topic.

NotPoster

Browse eBay

Related search

not memorabilia

Open a targeted eBay search for items related to this topic.

NotMemorabilia

Browse eBay

Browse more on eBay

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: researchgate.net
Title: Research Gate(PDF) Words or Numbers?
Link:https://www.researchgate.net/publication/340375795_Words_or_Numbers_Communicating_Probability_in_Intelligence_Analysis

Published: July 23, 2020

2. Source: appext.hks.harvard.edu
Link:https://appext.hks.harvard.edu/publications/getFile.aspx?Id=1326

3. Source: gwern.net
Title: Verbal Probability Expressions In National Intelligence
Link:https://gwern.net/doc/statistics/bayes/2008-kesselman.pdf

Published: May 12, 2008

4. Source: Wikipedia
Title: Words of estimative probability
Link:https://en.wikipedia.org/wiki/Words_of_estimative_probability

5. Source: youtube.com
Title: Words of estimative probability
Link:https://www.youtube.com/watch?v=XdojqZ1AerY

6. Source: journals.sagepub.com
Link:https://journals.sagepub.com/doi/10.1177/2372732215602907

7. Source: medium.com
Title: Words of Estimative Probability
Link:https://medium.com/%40chris.cooley/words-of-estimative-probability-an-analytical-language-reference-be84b8625628

8. Source: globalsecurity.org
Link:https://www.globalsecurity.org/intell/ops/probability.htm

Additional References

9. Source: youtube.com
Title: Superforcasting: How to [Predict]({{ ‘predict/’ | relative_url }}) the Future
Link:https://www.youtube.com/watch?v=rV5Gicb66WA

10. Source: youtu.be
Link:https://youtu.be/5mbX6ITznHk

11. Source: cyberwar.nl
Link:https://cyberwar.nl/d/fromForeknowledge/Analystsjudgementofprobability.pdf

12. Source: youtu.be
Link:https://youtu.be/NDY2fH1FitQ

13. Source: youtu.be
Link:https://youtu.be/UVvuwv-ne1I

14. Source: youtube.com
Title: ‘Superforecasting’: The people that predict the future – BBC REEL
Link:https://www.youtube.com/watch?v=SAzTP2A634g

15. Source: youtube.com
Title: Model Calibration
Link:https://www.youtube.com/watch?v=BiaebXlgfNQ