Within Teaching
What Physics Labs Reveal About Teaching Judgement
Explicit lab prompts can teach students to compare data, test models, and improve methods under uncertainty.
On this page
- Why routine labs may miss analytical skill
- Data comparison as a repeated decision
- What persisted after prompts were removed
Page outline Jump by section
Introduction
Physics laboratories are often assumed to teach analytical thinking simply because students collect data. Research suggests that this assumption is mistaken. Traditional “cookbook” laboratories, where students follow predetermined procedures to reproduce known results, can leave students with little opportunity to decide what evidence is trustworthy or how to improve an investigation. By contrast, redesigned physics laboratories that explicitly require students to compare datasets, test models against evidence, and revise their methods provide repeated practice in scientific judgement rather than procedural compliance. This approach offers one of the clearest demonstrations that analytical thinking can be taught directly within normal subject teaching, rather than as a separate critical-thinking course.[ResearchGate+2PNAS]researchgate.netResearch Gate(PDF) Measuring the impact of an instructional laboratoryWe have analyzed the impact of taking an associated lab course on the scores on final exam questions in two large introductory physics co…
Why routine labs may miss analytical skill
Many introductory physics laboratories have traditionally focused on verifying textbook laws. Students are given detailed instructions, expected outcomes and fixed procedures. Success is often measured by completing the experiment correctly rather than by making thoughtful decisions about uncertain evidence.
Research by Natasha Holmes, Carl Wieman and colleagues challenged this model after earlier work found that conventional physics laboratories produced little measurable improvement in students’ conceptual understanding of lecture material despite the considerable time invested in them. This prompted a shift in emphasis: instead of asking whether laboratories reinforce physics content, the researchers asked whether they could explicitly teach experimental judgement.[ResearchGate]researchgate.netResearch Gate(PDF) Measuring the impact of an instructional laboratoryWe have analyzed the impact of taking an associated lab course on the scores on final exam questions in two large introductory physics co…
The redesigned laboratories retained familiar experiments, such as pendulum measurements, but changed what students were expected to think about. Rather than confirming a known answer, students had to decide:
- whether two measurements genuinely differed or only appeared different because of uncertainty;
- whether a theoretical model adequately matched the observations;
- what additional measurements would most reduce uncertainty;
- how the experiment itself could be improved.
These decisions mirror the reasoning used by practising scientists, where evidence is rarely perfect and conclusions must be justified rather than simply reported.[PNAS]pnas.orgTeaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision…
Data comparison as a repeated decision
The distinctive feature of the transformed laboratories was not more sophisticated equipment but explicit prompts directing students to make evidence-based decisions repeatedly.
Instead of calculating uncertainty merely as a mathematical exercise, students were asked to use quantitative comparisons between datasets and between data and theoretical models. Whenever discrepancies appeared, they had to decide whether the difference was meaningful, identify possible explanations, and modify either their measurements or their experimental methods before collecting further data. This cycle of comparison, judgement and revision was repeated throughout the course.[PNAS]pnas.orgTeaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision…
The instructional framework deliberately embedded several thinking moves:
- Compare evidence quantitatively rather than relying on visual inspection or intuition.
- Judge competing explanations using statistical evidence rather than preference.
- Act on the judgement by redesigning measurements or collecting additional data.
- Evaluate the result before beginning another cycle.
The important point is that students were not merely told to “think critically”. They were repeatedly required to make concrete decisions under uncertainty and immediately experience the consequences of those decisions in the next stage of the experiment.[PNAS]pnas.orgTeaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision…
This design reflects an important distinction in thinking instruction. Analytical skill develops not from hearing explanations about scientific reasoning but from repeatedly practising the specific judgements that scientists routinely make.
What persisted after prompts were removed
One of the strongest findings from this work came after the explicit instructional support was gradually withdrawn.
During the early part of the transformed course, students received direct prompts telling them to compare data, evaluate models and improve their methods. Later laboratory activities removed many of these prompts. Despite this reduction in guidance, students continued making sophisticated evidence-based decisions on their own, while students in the conventional laboratory generally did not adopt these behaviours spontaneously.[PNAS]pnas.orgTeaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision…
The researchers also found that these students transferred their judgement skills to unfamiliar experiments that were structurally different from the earlier activities. Rather than simply repeating memorised procedures, they independently questioned assumptions, evaluated conflicting evidence and proposed improvements to experimental design. This persistence after scaffolding was removed is important because it suggests that students had learned a general reasoning habit rather than a scripted classroom routine.[PNAS]pnas.orgTeaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision…
Evidence that the approach measures real judgement
As physics laboratory instruction increasingly focused on reasoning rather than verification, researchers also needed better ways to measure improvement.
This led to the development of the Physics Lab Inventory of Critical Thinking (PLIC), an assessment specifically designed to evaluate how students judge data, evaluate methods and decide what to trust in experimental situations. Unlike traditional tests that reward factual recall, the PLIC presents students with realistic experimental scenarios requiring evidence-based decisions about data quality, uncertainty, models and methodological choices. Validation involved thousands of students across dozens of institutions, providing evidence that the instrument measures critical thinking within experimental physics rather than general physics knowledge.[APS Links]link.aps.orgAPS LinksQuantifying critical thinking: Development and validation of the…by C Walsh · 2019 · Cited by 148 — Introductory physics lab…
Subsequent research comparing different laboratory designs across more than 100 institutions found that laboratories explicitly focused on experimentation skills consistently outperformed laboratories primarily designed to reinforce lecture concepts. Activities encouraging student decision-making and scientific communication explained much of this advantage, whereas simply spending more time on modelling activities contributed comparatively little on its own.[ResearchGate]researchgate.netResearch Gate Skills-focused lab instruction improves critical thinkingSkills-focused lab instruction improves critical thinking…April 11, 2022 — 25 Feb 2026 — We show that labs focused on deve…
What this case reveals about teaching judgement
The transformed physics laboratory provides unusually strong evidence that analytical judgement can be embedded inside ordinary disciplinary work without creating a separate critical-thinking curriculum.
Three features appear especially important:
- Thinking is made explicit through prompts that identify the required judgement.
- Students repeatedly make authentic decisions using imperfect evidence rather than following fixed procedures.
- Instruction gradually removes support, allowing independent judgement to emerge.
The broader lesson is not that every subject should resemble a physics laboratory. Rather, the case demonstrates that analytical skill improves when learners repeatedly practise the actual decisions experts make in that discipline. In physics, those decisions revolve around comparing evidence, evaluating uncertainty, testing models and deciding how to improve an investigation. The research suggests these judgement routines become more durable when they are explicitly taught, repeatedly exercised and eventually performed without prompts.[PNAS+2ResearchGate]pnas.orgTeaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision…
Amazon book picks
Further Reading
Books and field guides related to What Physics Labs Reveal About Teaching Judgement. Use these as the next step if you want deeper reading beyond the article.
Thinking, Fast and Slow
Provides cognitive foundations for better experimental judgement.
How to Think Like a Scientist
Explains scientific judgement, evidence evaluation and experimental reasoning.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
1.
Source: researchgate.net
Title: Research Gate(PDF) Measuring the impact of an instructional laboratory
Link:https://www.researchgate.net/publication/278048597_Measuring_the_impact_of_an_instructional_laboratory_on_the_learning_of_introductory_physics
Source snippet
We have analyzed the impact of taking an associated lab course on the scores on final exam questions in two large introductory physics co...
2.
Source: pnas.org
Link:https://www.pnas.org/doi/10.1073/pnas.1505329112
Source snippet
Teaching critical thinkingby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decision...
3.
Source: link.aps.org
Link:https://link.aps.org/doi/10.1103/PhysRevPhysEducRes.15.010135
Source snippet
APS LinksQuantifying critical thinking: Development and validation of the...by C Walsh · 2019 · Cited by 148 — Introductory physics lab...
4.
Source: researchgate.net
Title: Research Gate Skills-focused lab instruction improves critical thinking
Link:https://www.researchgate.net/publication/359881672_Skills-focused_lab_instruction_improves_critical_thinking_skills_and_experimentation_views_for_all_students
Source snippet
Skills-focused lab instruction improves critical thinking...April 11, 2022 — 25 Feb 2026 — We show that labs focused on deve...
Published: April 11, 2022
5.
Source: link.aps.org
Link:https://link.aps.org/pdf/10.1103/PhysRevPhysEducRes.16.020150
Source snippet
APS LinksEvaluating instructional labs' use of [deliberate practice]({{ 'deliberate-practice/' | relative_url }}) to...by EM Smith · 2020 · Cited by 30 — In this paper, we describe a...
6.
Source: link.aps.org
Link:https://link.aps.org/doi/10.1103/PhysRevPhysEducRes.19.020146
Source snippet
changes in student views of measurement...by A Werth · 2023 · Cited by 6 — The transformed course was explicitly designed to prioritize...
7.
Source: link.aps.org
Title: 5np1 mccl
Link:https://link.aps.org/doi/10.1103/5np1-mccl
Source snippet
aps.orgStudents' views of experimental physics after participating in...by Q Liu · 2025 · Cited by 1 — Even a single week of open-ended...
8.
Source: researchgate.net
Link:https://www.researchgate.net/publication/407150006Introductory_physics_students%27_valued_features_in_AI[feedback
Source snippet
cation of student-generated prompts when seeking feedback from AI and (ii) to...Read more...
9.
Source: pmc.ncbi.nlm.nih.gov
Title: PMCTeaching critical thinking
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC4568696/
Source snippet
critical thinking - PMCby NG Holmes · 2015 · Cited by 479 — We present a simple learning framework that employs cycles of decisions about...
Additional References
10.
Source: lassp.cornell.edu
Link:https://www.lassp.cornell.edu/physics-lab-inventory-critical-thinking
Source snippet
Physics Lab Inventory of Critical thinkingThe Physics Lab Inventory of Critical thinking (PLIC) is a closed-response survey designed to a...
11.
Source: physics.utoronto.ca
Link:https://www.physics.utoronto.ca/~phy289/Holmes%2C%20Wieman%2C%20Bonn%20-%202015%20-%20Teaching%20critical%20thinking%285%29.pdf
Source snippet
critical thinkingby NG Holmesa · Cited by 479 — This study reports the results of applying that structure in an introductory physics labo...
12.
Source: ed.stanford.edu
Title: research shows how improve students critical thinking about scientific evidence
Link:https://ed.stanford.edu/news/research-shows-how-improve-students-critical-thinking-about-scientific-evidence
Source snippet
Research shows how to improve students' critical thinking...17 Aug 2015 — Encouraging students to repeatedly make decisions about data c...
13.
Source: pub.towardsai.net
Title: the physics of prompting why words dont matter but structure does 4c26cd69fc06
Link:https://pub.towardsai.net/the-physics-of-prompting-why-words-dont-matter-but-structure-does-4c26cd69fc06
Source snippet
LLMs follow mechanics. They react to: Placement (position bias); Density (entropy control); Patterns (in-context...Read more...
14.
Source: scribd.com
Link:https://www.scribd.com/document/477107376/PhysRevPhysEducRes-15-010135
Source snippet
is on. developing experimentation and critical thinking skills.Read more...
15.
Source: youtube.com
Title: Carl Wieman: Taking a Scientific Approach to Science Education
Link:https://www.youtube.com/watch?v=aBEPXfY7Elw
Source snippet
This selection of videos directly features the core researchers mentioned in your reference text, Dr. Natasha Holmes and Carl Wieman, as...
16.
Source: aip.org
Title: how lab courses can teach more than science
Link:https://www.aip.org/inside-science/how-lab-courses-can-teach-more-than-science
Source snippet
17 Aug 2015 — A research team that includes a Nobel Prize-winning physicist has developed a way to teach critical thinking that they foun...
17.
Source: pubmed.ncbi.nlm.nih.gov
Link:https://pubmed.ncbi.nlm.nih.gov/26283351/
Source snippet
critical thinkingby NG Holmes · 2015 · Cited by 479 — This study reports the results of applying that structure in an introductory physic...
18.
Source: strobe.colorado.edu
Link:https://strobe.colorado.edu/wp-content/uploads/Impact-of-a-course-transformation-on-students-reasoning-about-measurement-uncertainty.pdf
Source snippet
of a course transformation on students' reasoning...by B Pollard · 2020 · Cited by 56 — In this work, we focus on measurement uncertaint...
19.
Source: news.stanford.edu
Title: thinking holmes wieman 081715
Link:https://news.stanford.edu/stories/2015/08/thinking-holmes-wieman-081715
Source snippet
research shows how to improve students' critical...17 Aug 2015 — Students who gather their own data and make their own decisions in a si...
Topic Tree