Within Calibration
When Likely Does Not Mean the Same Thing
Words like likely and possible often mask large differences in certainty until a team turns them into numbers.
On this page
- Why vague confidence words mislead teams
- How numeric probabilities expose hidden disagreement
- Simple translation rules for everyday forecasts
Page outline Jump by section
Introduction
Words such as likely, possible, unlikely and almost certain seem precise, but they often conceal substantial differences in what people actually believe. Two colleagues may both say that a project is “likely” to succeed while one privately means a 60% chance and the other means 90%. The discussion appears to end in agreement even though they would make very different decisions if money, time or safety depended on the outcome. Research across intelligence analysis, climate science, medicine and risk communication consistently shows that verbal probability expressions are interpreted far less consistently than most people expect. [PMC]pmc.ncbi.nlm.nih.govPMCVerbal probabilities: Very likely to be somewhat moreby BC Wintle · 2019 · Cited by 115 — People interpret verbal expressions of probabilities (eg 'very likely') in different ways, yet wo…
For anyone trying to improve analytical thinking, this matters because hidden disagreement is harder to detect than open disagreement. The solution is not to eliminate ordinary language but to recognise when vague probability words are carrying more weight than they can reliably support, then translate them into numerical estimates when important decisions depend on them.
Why vague confidence words mislead teams
The problem is not that words such as likely are meaningless. The problem is that they are elastic. Each listener unconsciously maps the word onto a personal probability scale shaped by experience, context and risk tolerance.
Suppose a product team discusses whether a feature will be finished before launch:
- One engineer says, “It’s likely we’ll finish on time.”
- The project manager hears an 85% chance.
- The engineer actually means around 60%.
Nobody notices the disagreement because everyone leaves the meeting believing they agreed.
Studies repeatedly find wide variation in how people interpret common probability words, even when they share the same language and professional background. In one large experiment, participants assigned strikingly different numerical values to expressions such as “very likely”, despite seeing the same wording in similar contexts. [PMC]pmc.ncbi.nlm.nih.govPMCVerbal probabilities: Very likely to be somewhat moreby BC Wintle · 2019 · Cited by 115 — People interpret verbal expressions of probabilities (eg 'very likely') in different ways, yet wo…
This creates a dangerous illusion of consensus. Teams often argue less than they should because ambiguous language hides differences instead of revealing them.
How numeric probabilities expose hidden disagreement
Replacing vague words with approximate numbers forces uncertainty into the open.
Imagine a meeting where each participant writes down a percentage before discussion.
Instead of hearing:
“I think it’s likely.”
You hear:
- 55%
- 65%
- 82%
- 95%
The conversation immediately changes. People can now ask why someone is much more optimistic or pessimistic instead of assuming everyone shares the same belief.
Numbers are not perfectly objective. A forecast of 70% is still a judgement. However, percentages have two important advantages:
- They are testable against future outcomes.
- They reveal disagreement that language can hide.
This is one reason why forecasting tournaments and evidence-based decision-making often require participants to express beliefs numerically rather than verbally. Explicit probabilities also make later calibration possible because forecasters can compare stated confidence with real-world results. [Wiley Online Library]onlinelibrary.wiley.comWiley Online LibraryCommunicating uncertainty in national security intelligence…by D Irwin · 2023 · Cited by 36 — Communicate judgment…
Lessons from organisations that depend on uncertainty
Some organisations face decisions where misunderstanding probability can have serious consequences. Rather than leaving terms open to interpretation, they define them explicitly.
The intelligence community has long wrestled with this problem. In the 1960s, intelligence analyst Sherman Kent argued that phrases such as probable and almost certain should correspond to agreed probability ranges rather than individual intuition. Modern intelligence guidance continues this idea by pairing approved verbal expressions with numerical ranges to reduce ambiguity. [SSRN+2ETH Zurich Files]papers.ssrn.comChapter 18 – VARIANTS OF VAGUE VERBIAGEUS Intelligence Community Directive (ICD) 203 provides two sets of verbal uncertainty expressi…
A similar approach appears in climate science. The Intergovernmental Panel on Climate Change (IPCC) assigns specific probability ranges to phrases such as:
- Virtually certain: greater than 99%
- Very likely: greater than 90%
- Likely: greater than 66%
- About as likely as not: 33% to 66%
- Very unlikely: less than 10% [Wikipedia]WikipediaWords of estimative probabilityWords of estimative probability
These standards do not eliminate uncertainty. They reduce disagreement about what the words themselves mean, allowing discussion to focus on the evidence instead.
Why context does not solve the problem
Many people assume that surrounding context naturally clarifies vague probability words. Research suggests this helps less than expected.
Even when participants read probability expressions within realistic scenarios, interpretations remain widely dispersed. Different readers continue assigning substantially different numerical values to the same phrase. Similar findings have appeared across different countries and languages, suggesting that the ambiguity is not merely a feature of English. [PMC+2Journal of Science Communication]pmc.ncbi.nlm.nih.govPMCVerbal probabilities: Very likely to be somewhat moreby BC Wintle · 2019 · Cited by 115 — People interpret verbal expressions of probabilities (eg 'very likely') in different ways, yet wo…
Context certainly matters for understanding what is being discussed. It is much less reliable for communicating how likely the speaker believes it is.
Simple translation rules for everyday forecasts
Most conversations do not require statistical precision. They do benefit from making uncertainty more explicit when the stakes are meaningful.
Useful habits include:
- Attach a number to important forecasts. Instead of “probably”, say “around a 70% chance”.
- Treat words as introductions, not conclusions. Follow “unlikely” with an approximate percentage.
- Check interpretations. Ask, “When you say likely, roughly what probability do you mean?”
- Record the number before outcomes are known. This supports later calibration instead of relying on memory.
- Distinguish probability from confidence in evidence. Two people may both assign a 70% chance while having very different reasons or evidence quality.
The goal is not false mathematical precision. Saying “roughly 65%” is usually more informative than insisting on “exactly 65.3%”. Approximate numbers communicate beliefs more clearly without pretending to know more than is justified.
A small change that improves analytical thinking
One of the simplest improvements to collective reasoning is replacing important probability words with approximate numerical estimates. The change seems minor, yet it transforms conversations.
Instead of debating whether something is “possible”, people begin discussing whether they mean 20%, 40% or 70%. Instead of assuming agreement, they discover where genuine differences lie. Instead of remembering that everyone “felt confident”, they leave behind forecasts that can later be evaluated against reality.
Better analytical thinking depends not only on estimating uncertainty well, but also on expressing it clearly enough that others know what you actually mean. Numbers do not eliminate uncertainty, but they make hidden disagreement visible, giving teams a better chance to reason together rather than merely sounding as though they already agree.
Amazon book picks
Further Reading
Books and field guides related to When Likely Does Not Mean the Same Thing. Use these as the next step if you want deeper reading beyond the article.
How to Measure Anything
Shows how to replace vague judgments with measurable estimates.
eBay marketplace picks
Marketplace Samples
Topic-anchored marketplace searches for visual, collectible, or second-hand items related to this page.
Endnotes
-
Source: pmc.ncbi.nlm.nih.gov
Title: PMCVerbal probabilities: Very likely to be somewhat more
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC6469752/Source snippet
by BC Wintle · 2019 · Cited by 115 — People interpret verbal expressions of probabilities (eg 'very likely') in different ways, yet wo...
-
Source: onlinelibrary.wiley.com
Link: https://onlinelibrary.wiley.com/doi/full/10.1111/risa.14009Source snippet
Wiley Online LibraryCommunicating uncertainty in national security intelligence...by D Irwin · 2023 · Cited by 36 — Communicate judgment...
-
Source: papers.ssrn.com
Link: https://papers.ssrn.com/sol3/Delivery.cfm/SSRN_ID3441269_code2173996.pdf?abstractid=3441269&mirid=1Source snippet
Chapter 18 – VARIANTS OF VAGUE VERBIAGEUS Intelligence Community Directive (ICD) 203 provides two sets of verbal uncertainty expressi...
-
Source: Wikipedia
Title: Words of estimative probability
Link: https://en.wikipedia.org/wiki/Words_of_estimative_probability -
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC9987346/Source snippet
and Use in the Expression of Estimative Probabilityby B van Tiel · 2022 · Cited by 13 — Words of estimative probability (WEPs), such as '...
-
Source: jcom.sissa.it
Link: https://jcom.sissa.it/article/pubid/JCOM_1902_2020_A03/Source snippet
Journal of Science CommunicationVariability in the interpretation of probability phrases used in...by S Willems · 2020 · Cited by 59 — I...
-
Source: files.ethz.ch
Link: https://www.files.ethz.ch/isn/55739/kesselman_thesis_final.pdfSource snippet
ETH Zurich FilesVerbal Probability Expressions in National Intelligence...by RF KESSELMAN · Cited by 28 — Sherman Kent's classic work “W...
Additional References
-
Source: files.osf.io
Link: https://files.osf.io/v1/resources/kuyhb/providers/osfstorage/5ec48c36f3e87e007a576dcc?action=download&direct=&version=2Source snippet
AND US POLICIES FOR COMMUNICATING...by MK Dhami — Despite the various options for communicating probability available to them, both the...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/335676014_Assessing_and_Communicating_Uncertainty_Effectively_in_a_Rapidly_Changing_WorldSource snippet
Recent research suggests that communicating probabilities numerically rather than verbally benefits forecasters' credibility.Read more...
-
Source: globalsecurity.org
Link: https://www.globalsecurity.org/intell/ops/probability.htmSource snippet
Intelligence analysts often resist explicit probability expressions, whether verbal or numerical...Read more...
-
Source: research.monash.edu
Link: https://research.monash.edu/en/publications/verbal-probabilities-ivery-likelyi-to-be-isomewhati-more-confusin/Source snippet
probabilities: >very likely> to be >somewhat> more...by BC Wintle · 2019 · Cited by 115 — People interpret verbal expressions of probabi...
-
Source: appext.hks.harvard.edu
Link: https://appext.hks.harvard.edu/publications/getFile.aspx?Id=1326Source snippet
ric percentages or “words of estimative probability.” We administered this survey to...Read more...
-
Source: lirias.kuleuven.be
Title: be Towards Effective Management of Verbal Probability
Link: https://lirias.kuleuven.be/retrieve/e54d95f3-0d8c-4508-8f3d-8f6b1f491a91Source snippet
by C FLEINER · 2024 · Cited by 3 — In this paper, we propose a co-learning approach with example to efficiently and effectively com...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/340555055Variability_in_the_interpretation_of_probability_phrases_used_in_Dutch_news_articles-_a_risk_for_miscommunicationSource snippet
In this study we look at how...Read more...
-
Source: publications.sto.nato.int
Link: https://publications.sto.nato.int/publications/STO%20Meeting%20Proceedings/STO-MP-SAS-OCS-ORA-2019/MP-SAS-OCS-ORA-2019-M-02-1.pdfSource snippet
and Communicating Uncertainty Effectively in a...by DR Mandel · Cited by 2 — This prompted Kent and his colleague, Max Foster, to develo...
-
Source: youtube.com
Title: A three-step approach to solve probability word problems
Link: https://www.youtube.com/watch?v=PwxrigxzaxsSource snippet
Words of estimative probability - YouTube Words of estimative probability - YouTube...
-
Source: substack.com
Title: p 186628752
Link: https://substack.com/home/post/p-186628752Source snippet
Analytic Tradecraft Standard 2: UncertaintyICD 203 prescribes a specific set of probability terms tied to numerical ranges (ICD 203 2015)...
Topic Tree