Within Transfer
Does Better Thinking Change Real Choices?
The most useful test of thinking instruction is whether people handle new decisions differently, not whether they can define concepts.
On this page
- Negative life events as outcome measures
- Decision processes versus test scores
- What the evidence can and cannot prove
Page outline Jump by section
Introduction
The strongest test of whether thinking skills matter is not whether someone can define concepts such as confirmation bias or correlation versus causation. It is whether they make better decisions when facing unfamiliar, messy situations where nobody tells them which thinking skill to use. Researchers increasingly distinguish between performance on thinking tests and real-world outcomes: financial choices, health decisions, interpersonal problems, workplace judgement and other consequences of everyday reasoning.
Evidence suggests that stronger critical thinking is associated with fewer costly mistakes across many areas of life, even after accounting for intelligence. At the same time, the evidence is more nuanced than simple claims that “critical thinking causes success”. Most studies are correlational rather than experimental, meaning they show relationships rather than definitive proof of cause and effect. The most useful conclusion is that real-world outcomes provide an important validity check on thinking instruction, but they are an imperfect one.[UMass Amherst]umass.eduUMass AmherstPredicting real-world outcomes_ Critical thinking ability is a…August 28, 2017 — by HA Butler · Cited by 2 — Despite evid…
Why Real-World Outcomes Matter More Than Test Scores
Many educational programmes judge success by improvements on classroom assessments. While useful, these measures answer only a limited question: can participants perform well on the kinds of problems they were trained to solve?
A more demanding question is whether improved reasoning transfers into everyday decisions. Someone who genuinely thinks more effectively should, over time, be more likely to:
- evaluate evidence before acting;
- avoid preventable financial and administrative mistakes;
- recognise misleading claims;
- revise beliefs when evidence changes;
- make decisions that reduce avoidable negative consequences.
This shifts evaluation from knowledge about thinking to behaviour influenced by thinking. A person who scores highly on a reasoning assessment but repeatedly makes poor financial, medical or interpersonal decisions has not demonstrated the kind of transfer that matters most outside education.[PMC]pmc.ncbi.nlm.nih.govby HA Butler · 2024 · Cited by 97 — The researchers found that those who scored higher on the critical thinking assessment experienced…
Negative Life Events as Outcome Measures
One influential line of research asked a practical question: do better critical thinkers experience fewer avoidable negative life events?
Researchers developed inventories covering everyday problems across multiple domains rather than relying solely on examination scores. These inventories included events such as:
- paying avoidable late fees;
- allowing food to spoil unnecessarily;
- excessive consumer debt;
- serious financial errors;
- interpersonal conflicts;
- poor business decisions.[workplaceinsight.net]workplaceinsight.netcritical thinking ability better predictor life decisions intelligenceCritical thinking ability is a better predictor of life decisions…28 Jul 2017 — The avoidance of poor outcomes gave an indirect measur…
The assumption was not that every bad outcome reflects poor thinking. Illness, accidents and bad luck happen. Instead, across many different situations, people with stronger reasoning skills should accumulate fewer preventable errors if those skills genuinely influence behaviour.
Studies using the Halpern Critical Thinking Assessment found exactly this pattern. Individuals with higher critical thinking scores reported fewer negative life events across diverse areas of daily life. Importantly, this relationship appeared in multiple samples and was later examined across different countries, suggesting the finding was not confined to one educational setting.[Gredos+2UMass Amherst]gredos.usal.esThe Halpern Critical Thinking Assessment and real-world…February 15, 2024 — by HA Butler · 2012 · Cited by 217 — The purpose of…
Decision Processes Versus Test Scores
Perhaps the most interesting finding is that critical thinking and intelligence are not interchangeable.
Intelligence tests measure abilities such as reasoning speed, memory and pattern recognition. These predict many important educational and occupational outcomes. However, they do not necessarily capture whether someone routinely:
- seeks disconfirming evidence;
- questions first impressions;
- weighs alternative explanations;
- notices when confidence exceeds evidence.
In a study of community adults and university students, both IQ and critical thinking related to fewer negative life events. However, critical thinking explained additional variation beyond intelligence alone and proved the stronger predictor of real-world decision outcomes. In other words, being intellectually capable is not identical to consistently applying good judgement.[UMass Amherst]umass.eduUMass AmherstPredicting real-world outcomes_ Critical thinking ability is a…August 28, 2017 — by HA Butler · Cited by 2 — Despite evid…
This distinction aligns with a broader literature arguing that successful reasoning depends not only on cognitive capacity but also on habits of reflective thinking—choosing to pause, evaluate evidence and override intuitive but potentially misleading responses. Reviews of everyday critical thinking similarly conclude that measures of applied reasoning predict meaningful behavioural differences outside educational settings.[PMC]pmc.ncbi.nlm.nih.govby HA Butler · 2024 · Cited by 97 — The researchers found that those who scored higher on the critical thinking assessment experienced…
What Real-World Measures Capture Better Than Classroom Tests
Real-life outcomes have several advantages over traditional assessments.
First, they involve unprompted application. Everyday life does not announce that a situation requires critical thinking. People must recognise this themselves.
Second, they include competing pressures. Time limits, emotions, social influence and uncertainty all affect judgement in ways that simplified classroom exercises often cannot reproduce.
Third, they evaluate long-term behavioural patterns rather than isolated performances. One excellent answer on an examination says little about hundreds of decisions made over months or years.
For these reasons, outcome-based measures often provide stronger evidence that thinking skills have transferred beyond instruction into everyday behaviour.[PMC]pmc.ncbi.nlm.nih.govby HA Butler · 2024 · Cited by 97 — The researchers found that those who scored higher on the critical thinking assessment experienced…
What the Evidence Cannot Prove
The findings are encouraging but should not be overstated.
Most studies showing links between critical thinking and life outcomes are correlational. Several alternative explanations remain possible.
People with stronger critical thinking may also differ in ways that contribute to better outcomes, including:
- conscientiousness;
- self-control;
- educational opportunities;
- socioeconomic circumstances;
- accumulated domain knowledge.
Although researchers often control statistically for intelligence and some demographic variables, observational studies cannot eliminate every possible confounding factor. Higher critical thinking may contribute to better outcomes, but successful life experiences may also reinforce better thinking habits over time.
Another limitation concerns measurement. Many real-world outcome inventories rely on self-reported life events. Participants may forget events, interpret questions differently or under-report mistakes. Researchers have worked to validate these instruments across cultures, but self-report remains less objective than direct observation of behaviour.[Gredos+2SciELO]gredos.usal.esThe Halpern Critical Thinking Assessment and real-world…February 15, 2024 — by HA Butler · 2012 · Cited by 217 — The purpose of…
Why Decision Quality Is Not the Same as Decision Outcomes
Another important caution is that good decisions do not always produce good outcomes.
Even excellent reasoning sometimes leads to failure because of chance, incomplete information or unpredictable events. Conversely, poor reasoning can occasionally produce successful outcomes through luck.
Decision scientists therefore distinguish between:
- decision quality, meaning whether a choice was reasonable given the information available at the time; and
- decision outcome, meaning what eventually happened.
Real-world outcome measures become informative only when many decisions are considered together. Over numerous independent decisions, good reasoning should increase the probability of favourable outcomes even though individual cases remain influenced by uncertainty. Judging thinking solely by isolated successes or failures risks falling into outcome bias—evaluating decisions by what happened rather than by the quality of the reasoning that produced them.[Wikipedia]WikipediaOutcome biasOutcome bias
What This Means for Improving Thinking
From the perspective of improving analytical skills, the practical implication is straightforward: success should be measured by changed behaviour rather than vocabulary.
Someone has probably improved their thinking if, over time, they increasingly:
- compare multiple explanations before deciding;
- look for evidence against their preferred view;
- delay confident conclusions until evidence is sufficient;
- recognise when a familiar solution does not fit a new problem;
- avoid recurring categories of preventable mistakes.
Remembering definitions of logical fallacies or cognitive biases has value, but these are intermediate achievements. The ultimate question is whether those concepts become habits that alter everyday judgement.
Current evidence suggests they can—but proving that requires looking beyond test scores to the quality of decisions people make in ordinary life, while recognising that no single outcome can perfectly measure the quality of someone’s thinking.[UMass Amherst+2PMC]umass.eduUMass AmherstPredicting real-world outcomes_ Critical thinking ability is a…August 28, 2017 — by HA Butler · Cited by 2 — Despite evid…
Amazon book picks
Further Reading
Books and field guides related to Does Better Thinking Change Real Choices?. Use these as the next step if you want deeper reading beyond the article.
How to Decide
Targets real-life decision processes, uncertainty, feedback, and mistake reduction.
Predictably Irrational
Shows how systematic errors affect everyday decisions, even when people believe they are reasoning well.
The Scout Mindset
Emphasises truth-seeking habits that can affect practical choices and belief revision.
Decisive: How to Make Better Choices in Life and Work
Focuses on improving actual choices rather than merely knowing decision concepts.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
1.
Source: umass.edu
Link:https://www.umass.edu/preferen/You%20Must%20Read%20This/ThinkingSkills.pdf
Source snippet
UMass AmherstPredicting real-world outcomes_ Critical thinking ability is a...August 28, 2017 — by HA Butler · Cited by 2 — Despite evid...
Published: August 28, 2017
2.
Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10890380/
Source snippet
by HA Butler · 2024 · Cited by 97 — The researchers found that those who scored higher on the critical thinking assessment experienced...
3.
Source: scielo.br
Title: b96RMs8Q9h3Zbjx Pwc WYM7w
Link:https://www.scielo.br/j/paideia/a/b96RMs8Q9h3ZbjxPwcWYM7w/?lang=en
Source snippet
Real-World Outcomes and Critical Thinking: Differential...by A Franco · 2015 · Cited by 22 — The Real-World Outcomes is an inventory tha...
4.
Source: Wikipedia
Title: Outcome bias
Link:https://en.wikipedia.org/wiki/Outcome_bias
5.
Source: gredos.usal.es
Link:https://gredos.usal.es/bitstream/10366/155816/1/DPBPMCC_The%20Halpern%20Critical%20Thinking%20Assessment.pdf
Source snippet
The Halpern Critical Thinking Assessment and real-world...February 15, 2024 — by HA Butler · 2012 · Cited by 217 — The purpose of...
Published: February 15, 2024
6.
Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC10300824/
Source snippet
by CP Dwyer · 2023 · Cited by 170 — The aim of this review is to discuss such barriers and evaluate their impact on critical thinking...
Additional References
7.
Source: oecd.org
Link:https://www.oecd.org/content/dam/oecd/en/topics/policy-issues/future-of-education-and-skills/learning-compass-constructs/Critical%20Thinking.pdf
8.
Source: workplaceinsight.net
Title: critical thinking ability better predictor life decisions intelligence
Link:https://workplaceinsight.net/critical-thinking-ability-better-predictor-life-decisions-intelligence/
Source snippet
Critical thinking ability is a better predictor of life decisions...28 Jul 2017 — The avoidance of poor outcomes gave an indirect measur...
9.
Source: app.dimensions.ai
Link:https://app.dimensions.ai/details/publication/pub.1090324414
Source snippet
Critical thinking more strongly predicted life events than...Read more...
10.
Source: researchgate.net
Link:https://www.researchgate.net/publication/318097450_Predicting_Real-World_Outcomes_Critical_Thinking_Ability_is_a_Better_Predictor_of_Life_Decisions_than_Intelligence
Source snippet
nd significantly added to the variance explained by IQ.Read more...
11.
Source: psychologytoday.com
Title: critical thinking and real world outcomes
Link:https://www.psychologytoday.com/gb/blog/in-one-lifespan/201210/critical-thinking-and-real-world-outcomes
Source snippet
Critical Thinking and Real-World Outcomes27 Oct 2012 — A series of recent studies have examined the relationship between critical thinkin...
12.
Source: michaelhoganpsychology.com
Title: Critical Thinking and Real-World Outcomes
Link:https://michaelhoganpsychology.com/2015/07/17/critical-thinking-and-real-world-outcomes-predicting-real-world-outcomes-of-critical-thinking/
Source snippet
Michael Hoganby M Hogan — A series of recent studies have examined the relationship between critical thinking and real-world outcomes of...
13.
Source: semanticscholar.org
Link:https://www.semanticscholar.org/paper/Halpern-Critical-Thinking-Assessment-Predicts-of-Butler/2e5d9b527f26a6b5de96cdcf6959b28121bf0b22
Source snippet
negative life events... Predicting real-world outcomes: Critical thinking ability is a better predictor of life decisions than intellige...
14.
Source: semanticscholar.org
Link:https://www.semanticscholar.org/paper/Predicting-real-world-outcomes%3A-Critical-thinking-a-Butler-Pentoney/5621af7c48076e7d9c33400c1ac8f4ed32ec339b
Source snippet
Critical thinking ability is a better predictor of life decisions...1 Sept 2017 — Predicting real-world outcomes: Critical thinking abil...
15.
Source: olj.onlinelearningconsortium.org
Link:https://olj.onlinelearningconsortium.org/index.php/olj/article/view/2725
Source snippet
Impact of Attitudes, Beliefs, and Cognitive Reflection...1 Jun 2022 — Predicting real-world outcomes: Critical thinking ability is a bet...
16.
Source: youtube.com
Link:https://www.youtube.com/watch?v=IhwovFgur-E
Source snippet
PSU's Dr. Keith Stanovich - What Intelligence Tests Miss...
Topic Tree