What Superforecasting Teaches Everyday Thinkers

Introduction

The Good Judgment Project showed that better judgement is not mainly about having exceptional intelligence or privileged information. Instead, it comes from making explicit probabilistic forecasts, updating them as evidence changes, and learning systematically from feedback. Originally developed as part of a large forecasting tournament for the US intelligence community, the project demonstrated that ordinary people could substantially improve their predictive accuracy through training, disciplined thinking, teamwork, and careful measurement. These lessons translate surprisingly well to everyday decisions, from planning projects and assessing business risks to judging whether a news story will develop as expected. Rather than trying to predict everything perfectly, the project teaches people how to become better calibrated: more confident when the evidence is strong, less certain when it is weak, and quicker to revise beliefs when reality changes.[Stanford University+2iarpa.gov]web.stanford.eduIntelligence Community sponsored a series of forecasting tournaments designed to explore the best… The Good Judgment Project (GJP). In…

Why forecasting tournaments matter

The Good Judgment Project emerged from the Intelligence Advanced Research Projects Activity (IARPA) Aggregative Contingent Estimation (ACE) programme, which asked competing research teams to improve geopolitical forecasting. Participants answered hundreds of clearly defined questions such as whether a country would hold an election by a particular date or whether a political agreement would be reached within a specified period. Every forecast received a numerical probability and was later scored against reality.[iarpa.gov]iarpa.govACEThe goal of the ACE Program is to dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad r…

This structure matters because it removes many of the excuses that normally protect poor judgement. Forecasts must be:

• specific enough to be resolved;
• probabilistic rather than vague;
• made before outcomes are known;
• scored objectively after the event.

Unlike ordinary opinions, forecasts cannot quietly evolve in memory after the fact. The continual cycle of prediction, outcome and feedback allows genuine improvement.

Perhaps the project’s most important finding was that forecasting ability varied far more than many researchers expected. Some participants consistently outperformed both experts and other volunteers over multiple years. These “superforecasters” were not simply lucky. Their advantage persisted across different questions and forecasting seasons, suggesting that better judgement reflected repeatable habits rather than isolated successes.[Stanford University+2Wharton Faculty Platform]web.stanford.eduIntelligence Community sponsored a series of forecasting tournaments designed to explore the best… The Good Judgment Project (GJP). In…

Training, updating, teamwork, and aggregation

The Good Judgment Project identified several practical behaviours that repeatedly distinguished stronger forecasters. None require participation in a forecasting tournament.

Think in probabilities, not certainties

Everyday conversations encourage words such as “probably”, “unlikely” or “almost certain”. Superforecasters instead attached explicit numerical probabilities to claims.

Doing so forces clearer thinking. Saying there is a 65% chance that a supplier will deliver on time communicates much more information than simply saying the delivery is “likely”. It also makes later evaluation possible.

Repeated use of numerical probabilities gradually improves calibration because forecasters discover whether their 70% predictions really occur about seven times out of ten.[Stanford University]web.stanford.eduIntelligence Community sponsored a series of forecasting tournaments designed to explore the best… The Good Judgment Project (GJP). In…

Update beliefs frequently

One of the strongest findings from the project was that successful forecasters revised their estimates more often than weaker participants.

Updating did not mean constantly changing one’s mind for trivial reasons. Instead, it meant responding proportionately to genuinely new evidence. Rather than treating an initial prediction as something to defend, superforecasters viewed it as a temporary estimate that should improve over time.

This contrasts with a common psychological tendency to anchor on first impressions and adjust too little afterwards.[Stanford University+2Good Judgment]web.stanford.eduIntelligence Community sponsored a series of forecasting tournaments designed to explore the best… The Good Judgment Project (GJP). In…

Break difficult questions into smaller ones

Large uncertain questions become easier when divided into manageable components.

Instead of asking:

“Will this new product succeed?”

a forecaster might separately estimate:

Thinking through each component often produces better overall judgement than relying on intuition about the whole problem. This “outside view plus inside view” style of reasoning became a recurring theme in the project’s training.[Corporate Research Forum]crforum.co.ukIntelligence Advanced Research Projects Activity tournament.Read morerporate Research ForumSuperforecasting: How to Upgrade Your Company's…by PJH Schoemaker · 2016 · Cited by 71 — The Good Judgment Pro…

Use base rates before unique stories

Superforecasters typically began with historical frequencies before considering special circumstances.

Suppose someone wants to estimate whether a software project will finish on time. A useful first question is not “How motivated is this team?” but “How often do comparable software projects finish on schedule?”

Starting with similar past cases helps prevent compelling narratives from overwhelming statistical reality. Individual details still matter, but they are interpreted relative to an evidence-based baseline rather than replacing it altogether.[Good Judgment]goodjudgment.comGood JudgmentTen Commandments for Aspiring SuperforecastersThese commandments describe behaviors that have been “experimentally demonstra…

Learn collaboratively without seeking consensus

An interesting finding from the Good Judgment Project was that carefully designed teamwork improved forecasting.

The most successful teams did not simply vote or compromise. Members challenged one another’s assumptions, shared evidence and independently revised their probabilities. Diversity of perspective proved valuable when combined with respectful disagreement and independent judgement.

Aggregation algorithms then combined many individual forecasts, usually producing predictions that outperformed most single forecasters. This illustrates a refined version of the “wisdom of crowds”: crowds become especially effective when individuals think independently and update from shared information rather than merely copying one another.[Stanford University+2Wharton Faculty Platform]web.stanford.eduIntelligence Community sponsored a series of forecasting tournaments designed to explore the best… The Good Judgment Project (GJP). In…

What individuals can borrow without a tournament

Most people do not need sophisticated software or hundreds of forecasting questions to benefit from these ideas. The habits can be incorporated into an ordinary prediction journal.

Useful questions include:

Each prediction should include:

After the outcome becomes known, compare the prediction with reality rather than with memory.

Over time, patterns emerge. Some people discover they consistently assign probabilities that are too high. Others realise they update too slowly when new evidence arrives. These recurring errors are far more useful than isolated mistakes because they reveal systematic biases in judgement.

The mindset behind superforecasting

Research associated with the Good Judgment Project suggests that successful forecasters shared several psychological habits.

They tended to be intellectually curious rather than ideologically rigid. They were comfortable expressing uncertainty instead of pretending certainty. They actively searched for evidence that could prove them wrong. They separated their identity from individual predictions, making it easier to admit error without treating it as personal failure.

This combination has sometimes been described as holding strong opinions lightly: working hard to reach the best current estimate while remaining willing to revise it whenever better evidence appears.[Good Judgment+2Esoteric Library]goodjudgment.comGood JudgmentTen Commandments for Aspiring SuperforecastersThese commandments describe behaviors that have been “experimentally demonstra…

Limits and misunderstandings

The Good Judgment Project should not be interpreted as showing that anyone can reliably predict complex events far into the future.

Several important limitations deserve attention.

First, the project focused largely on questions that were clearly defined, objectively resolvable and typically answered within months or a few years. Many everyday uncertainties are less sharply defined.

Second, forecasting skill improves decision quality but does not eliminate randomness. Even excellent forecasters make many incorrect predictions because uncertainty is inherent rather than merely a consequence of poor reasoning.

Third, some later research has questioned how easily superforecasting methods transfer to environments with limited data, smaller participant pools or rapidly changing conditions. Structured forecasting still helps, but gains may be smaller than those observed in the original large-scale tournaments.[PMC+2Stanford University]pmc.ncbi.nlm.nih.govThe superforecasting hypothesis is challenged under real-life scarcitySuperforecasting reality check: Evidence from a small pool of…by I Katsagounos · 2020 · Cited by 23 — The study contributes to the…

Finally, forecasting is only one part of good decision-making. A well-calibrated prediction does not determine what someone ought to do. Decisions also involve values, costs, risk tolerance and practical constraints.

The enduring lesson for everyday thinkers

The lasting contribution of the Good Judgment Project is not that a small group of “superforecasters” possess extraordinary intuition. It is that many elements of better judgement can be learned.

For someone already keeping a prediction journal, the project offers a practical framework:

Together, these practices create a continuous learning loop. Instead of treating judgement as a fixed personal trait, they encourage viewing it as a skill that improves through repeated forecasting, honest feedback and disciplined updating.[Stanford University+2Good Judgment]web.stanford.eduIntelligence Community sponsored a series of forecasting tournaments designed to explore the best… The Good Judgment Project (GJP). In…

Amazon book picks

Further Reading

Books and field guides related to What Superforecasting Teaches Everyday Thinkers. Use these as the next step if you want deeper reading beyond the article.

Book

Thinking, Fast and Slow

By Daniel Kahneman

Provides the cognitive bias and judgment framework that underpins better probabilistic thinking.

See on Amazon

Book

The Signal and the Noise

By Nate Silver

Explores why forecasts succeed or fail across many domains, complementing superforecasting concepts.

See on Amazon

Book

The Scout Mindset

By Julia Galef

Emphasizes updating beliefs, intellectual humility, and evidence-based reasoning central to good forecasting.

See on Amazon

Book

How to Measure Anything

By Douglas W. Hubbard

Shows practical methods for estimating uncertainty and improving decisions using measurable evidence.

See on Amazon

Browse more on Amazon: Thinking, Fast and Slow The Signal and the Noise The Scout Mindset

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Top pick

critical thinking card game

Open a targeted eBay search for items related to this topic.

CriticalThinkingCard

Browse eBay

Related search

critical thinking game

Open a targeted eBay search for items related to this topic.

CriticalThinkingGame

Browse eBay

Related search

your poster

Open a targeted eBay search for items related to this topic.

YourPoster

Browse eBay

Related search

your memorabilia

Open a targeted eBay search for items related to this topic.

YourMemorabilia

Browse eBay

Browse more on eBay

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: web.stanford.edu
Link:https://web.stanford.edu/~knutson/jdm/mellers15.pdf

2. Source: iarpa.gov
Link:https://www.iarpa.gov/research-programs/ace

3. Source: pmc.ncbi.nlm.nih.gov
Title: The superforecasting hypothesis is challenged under real-life scarcity
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC7333631/

4. Source: goodjudgment.com
Link:https://goodjudgment.com/

5. Source: faculty.wharton.upenn.edu
Title: 2015 superforecasters
Link:https://faculty.wharton.upenn.edu/wp-content/uploads/2015/07/2015—superforecasters.pdf

6. Source: esotericlibrary.weebly.com
Link:https://esotericlibrary.weebly.com/uploads/5/0/7/7/5077636/philip_e.tetlock-_superforecasting_the_art_and_science_of_prediction.pdf

7. Source: goodjudgment.com
Link:https://goodjudgment.com/philip-tetlocks-10-commandments-of-superforecasting/

8. Source: crforum.co.uk
Title: Intelligence Advanced Research Projects Activity tournament.Read more
Link:https://www.crforum.co.uk/wp-content/uploads/2016/10/2016_Schoemaker_Tetlock_Superforecasting-HBR-May-2016.pdf

9. Source: goodjudgment.com
Title: common questions good judgment superforecasters
Link:https://goodjudgment.com/common-questions-good-judgment-superforecasters/

10. Source: goodjudgment.com
Link:https://goodjudgment.com/resources/the-superforecasters-track-record/

11. Source: Wikipedia
Title: The Good Judgment Project
Link:https://en.wikipedia.org/wiki/The_Good_Judgment_Project

12. Source: gjopen.com
Link:https://www.gjopen.com/

13. Source: managementboek.nl
Link:https://www.managementboek.nl/magazine/recensie/11945/superforecasting-manpo187?srsltid=AfmBOoo2Wwm0nBtsMoyaIcQ1lmG7SUI86_S8FRw8dZuMx64oM-je8tPR

14. Source: youtube.com
Link:https://www.youtube.com/watch?v=pedNak4S9IE

15. Source: goodjudgment.substack.com
Link:https://goodjudgment.substack.com/p/a-primer-on-good-judgment-inc-and

Additional References

16. Source: thedecisionlab.com
Link:https://thedecisionlab.com/thinkers/political-science/philip-tetlock

17. Source: thejennyproject.com
Link:https://www.thejennyproject.com/blog/the-good-judgment-project-revolutionizing-forecasting

18. Source: edge.org
Link:https://www.edge.org/conversation/philip_tetlock-edge-master-class-2015-a-short-course-in-superforecasting-class-ii

19. Source: aiimpacts.org
Title: evidence on good forecasting practices from the good judgment project
Link:https://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project/

20. Source: floriswolswijk.com
Link:https://floriswolswijk.com/superforecasting/

21. Source: askeladdencapital.com
Link:https://askeladdencapital.com/philip-tetlocks-superforecasting-book-review-notes-analysis/

22. Source: soa.org
Link:https://www.soa.org/globalassets/assets/Library/Newsletters/Forecasting-Futurism/2015/July/ffn-2015-iss11-campbell.pdf

23. Source: forum.effectivealtruism.org
Link:https://forum.effectivealtruism.org/posts/pnpnqA4hijnr59p7d/efforts-to-improve-the-accuracy-of-our-judgments-and

24. Source: lesswrong.com
Title: is there an intuitive way to [explain]({{ ‘explain/’ | relative_url }}) how much better
Link:https://www.lesswrong.com/posts/yDdBLdTSAEcbjkbGj/is-there-an-intuitive-way-to-explain-how-much-better

25. Source: reddit.com
Link:https://www.reddit.com/r/slatestarcodex/comments/cvoi4y/the_tenants_of_superforecast_accuracy_are/