Within Practice Tests
How to Build a Self Test That Actually Works
Good practice tests ask you to recall, explain, apply and contrast an idea, not just name it.
On this page
- Four layers of a useful self test
- Turning concepts into application questions
- Avoiding quizzes that are too easy or too narrow
Page outline Jump by section
Introduction
A good self-test does more than ask whether you can remember a definition. It asks whether you can recognise when an idea matters, explain it in your own words, apply it to an unfamiliar situation, and distinguish it from similar concepts. Those are the abilities that make knowledge usable in real thinking rather than merely familiar on the page.
This design principle follows directly from research on retrieval practice. Testing strengthens long-term learning most effectively when learners actively reconstruct knowledge instead of recognising it, and when practice resembles the way the knowledge will later be used. Simply naming a concept rarely prepares you to analyse evidence, solve problems, or make decisions. Well-designed self-tests therefore aim to build flexible mental models rather than isolated facts. [PsychNet+2Google Sites]psychnet.wustl.eduRoediger Karpicke 2006 PPSThe Power of Testing Memoryby HL Roediger III · Cited by 3306 — The experiment involves three phases— study, retrieval practice…
Four layers of a useful self-test
The simplest way to improve a self-test is to make it progressively more demanding. Rather than writing ten questions that all ask for definitions, design questions that exercise different forms of understanding.
- Recall the core idea What is confirmation bias? Define opportunity cost. State the difference between correlation and causation.
This establishes whether the concept can be retrieved without prompts. Recognition (“I know it when I see it”) is much weaker than producing the answer from memory. Short-answer questions generally produce stronger retrieval than recognition-only formats because they require reconstruction rather than guessing. [Andyʼs working notes]notes.andymatuschak.orgAndyʼs working notesTesting effectRetrieval practice and transfer learning. The simplest instantiation of the Testing effect is elicited…
- Explain the idea
Move beyond definitions.
Instead of asking, “What is survivorship bias?”, ask:
Explain survivorship bias to someone who has never heard the term.
Requiring explanation exposes whether the learner has an organised mental model or merely remembers wording.
- Apply the idea
Application is where concepts become thinking tools.
For example:
A company interviews only its most successful salespeople before changing its training programme. What important information might be missing?
Now retrieval must occur inside a realistic situation rather than in isolation. Research suggests retrieval practice can improve transfer—the ability to use knowledge in new contexts—especially when questions require application rather than verbatim repetition. [Andyʼs working notes]notes.andymatuschak.orgAndyʼs working notesTesting effectRetrieval practice and transfer learning. The simplest instantiation of the Testing effect is elicited…
- Contrast similar ideas
Many analytical mistakes arise from confusing neighbouring concepts.
Useful comparison questions include:
- How is availability bias different from confirmation bias?
- When would base rates matter more than anecdotal evidence?
- How does causation differ from prediction?
Contrasting concepts forces learners to organise knowledge rather than store disconnected definitions.
Turning concepts into application questions
The most useful self-tests resemble the situations in which you actually expect to think.
Suppose you are learning the idea of incentives.
A weak question asks:
What are incentives?
A stronger question asks:
A manager introduces bonuses that unexpectedly reduce cooperation between teams. How could incentives explain this outcome?
Instead of retrieving a sentence, you retrieve a framework and use it to interpret evidence.
The same pattern works across analytical concepts.
ConceptSimple recallBetter applicationBase ratesDefine base rates.Which missing statistic would change your judgement most?Opportunity costWhat is opportunity cost?What valuable alternative is being sacrificed here?Regression to the meanDefine regression.Was the improvement likely caused by the intervention or partly by natural variation?Correlation vs causationState the difference.What alternative explanations could produce this relationship?
Notice that every application question asks the learner to inspect evidence, identify relevant information, or explain reasoning rather than repeat memorised wording.
This mirrors the principle of transfer-appropriate processing: practice is more valuable when retrieval resembles future use. If your goal is analytical thinking, your self-tests should resemble analytical thinking. [Wikipedia]WikipediaTesting effectTesting effect
Build questions that require reasoning, not recognition
Many self-quizzes accidentally become recognition exercises.
For example:
- “Which of these is confirmation bias?”
- Flashcards that reveal most of the answer.
- Multiple-choice questions with obviously incorrect alternatives.
These can have value, particularly for beginners, but they often allow pattern recognition instead of genuine retrieval.
More demanding formats include:
- Answer before looking at any hints.
- Write three or four complete sentences.
- Draw a diagram from memory.
- Explain aloud without notes.
- Generate your own example.
Even open-book checking can be effective if retrieval comes first and consulting the material happens only afterwards. The important learning event is the attempt to retrieve before receiving the answer. [Andyʼs working notes]notes.andymatuschak.orgAndyʼs working notesTesting effectRetrieval practice and transfer learning. The simplest instantiation of the Testing effect is elicited…
Avoiding quizzes that are too easy or too narrow
Many learners unknowingly create tests that measure familiarity instead of understanding.
Common design mistakes include:
- Only testing vocabulary. Knowing the label “anchoring” matters less than recognising anchoring in a negotiation or estimate.
- Repeating identical wording. If every question copies the textbook, retrieval becomes tied to specific phrasing.
- Always using the same examples. Concepts become attached to one familiar story instead of becoming general reasoning tools.
- Testing immediately after study every time. Immediate success can be misleading because information is still active in working memory. Delayed retrieval usually provides a more realistic measure of durable learning. [PsychNet]psychnet.wustl.eduRoediger Karpicke 2006 PPSThe Power of Testing Memoryby HL Roediger III · Cited by 3306 — The experiment involves three phases— study, retrieval practice…
A useful rule is to vary both the wording and the context while keeping the underlying concept constant.
For example, if learning about sunk costs, one question might involve business investment, another personal relationships, and another government policy. The learner must identify the same principle across different settings.
Match the self-test to future use
Different goals require different question designs.
If you expect to:
- Explain ideas, include explanation prompts.
- Evaluate evidence, include case studies with incomplete information.
- Make decisions, include scenarios with competing alternatives.
- Spot reasoning errors, include flawed arguments for diagnosis.
- Teach others, include questions that require analogies and simple explanations.
The closer the retrieval task resembles future thinking, the more likely the concept will become available under real conditions. Research on transfer shows that retrieval practice is not limited to memorising exact answers; appropriately designed testing can improve the ability to apply knowledge beyond the original learning context. [Andyʼs working notes+2Wikipedia]notes.andymatuschak.orgAndyʼs working notesTesting effectRetrieval practice and transfer learning. The simplest instantiation of the Testing effect is elicited…
A practical template for building better self-tests
For each important concept, create four questions:
- Recall: What is it?
- Explain: How would I describe it in plain language?
- Apply: Where would this concept change my judgement in a realistic scenario?
- Contrast: What similar idea might I confuse it with, and how are they different?
A single concept tested this way develops a richer mental representation than ten definition questions alone. The learner practises retrieving knowledge, organising it, adapting it to new situations, and recognising its boundaries—all essential features of durable understanding and stronger analytical thinking.
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Endnotes
-
Source: sites.google.com
Link: https://sites.google.com/view/cynthia-brame/teaching-guides/test-enhanced-learning-using-retrieval-practice-to-help-students-learnSource snippet
Google SitesTest-enhanced learning: Using retrieval practice to help...In essence, test-enhanced learning is the idea that the process o...
-
Source: Wikipedia
Title: Henry L. Roediger III
Link: https://en.wikipedia.org/wiki/Henry_L._Roediger_III -
Source: Wikipedia
Title: Testing effect
Link: https://en.wikipedia.org/wiki/Testing_effect -
Source: psychnet.wustl.edu
Title: Roediger Karpicke 2006 PPS
Link: https://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Roediger-Karpicke-2006_PPS.pdfSource snippet
The Power of Testing Memoryby HL Roediger III · Cited by 3306 — The experiment involves three phases— study, retrieval practice...
-
Source: notes.andymatuschak.org
Link: https://notes.andymatuschak.org/zTpJdbe6ub7uhBFLuHkFsrTSource snippet
Andyʼs working notesTesting effectRetrieval practice and transfer learning. The simplest instantiation of the Testing effect is elicited...
Additional References
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=G6u9_lbW2hYSource snippet
The Testing Effect & Retrieval practice: the number 1 study hackIn this video I'm going to tell you about retrieval practice (or self-tes...
-
Source: yukaichou.com
Title: retrieval [practice testing]({{ ‘practice-tests/’ | relative_url }}) effect roediger karpicke learning
Link: https://yukaichou.com/gamification-analysis/retrieval-practice-testing-effect-roediger-karpicke-learning/Source snippet
Retrieval Practice: Testing Beats RereadingRetrieval practice means pulling knowledge out of memory instead of putting it in again...
-
Source: my.chartered.college
Title: college Optimising-learning-using-retrieval-practice
Link: https://my.chartered.college/wp-content/uploads/2018/02/Optimising-learning-using-retrieval-practice.docxSource snippet
by MA Sumeracki · 2018 — Retrieval practice can also improve learning compared to other study strategies thought to be beneficial, such a...
-
Source: structural-learning.com
Title: testing effect retrieval practice
Link: https://www.structural-learning.com/post/testing-effect-retrieval-practiceSource snippet
The Testing Effect: Why Retrieval Practice Works29 Dec 2025 — Testing helps learners recall facts, according to Bjork (1994)...
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC3983480/Source snippet
practice enhances new learning: the forward effect of...by B Pastötter · 2014 · Cited by 277 — This work was largely on the backward eff...
-
Source: education-ni.gov.uk
Link: https://www.education-ni.gov.uk/sites/default/files/2025-04/May%20Newsletter%20-%20Retrieval%20Practice%20%20What%20it%20is%2C%20Why%20it%20Works%20and%20How%20to%20Do%20It%20Better.PDFSource snippet
psychology research.Read more...
-
Source: youtube.com
Title: Making [Low-Stakes]({{ ‘low-stakes/’ | relative_url }}) Practice Tests More Effective
Link: https://www.youtube.com/watch?v=XA-EKuD3hg4Source snippet
How to Do Practice Questions Properly (Study Strategy That Works)...
-
Source: youtube.com
Title: The Ultimate Active Recall Tutorial
Link: https://www.youtube.com/watch?v=bbd5wfsLo9sSource snippet
How to Master Active Recall (Step-by-Step for Students)...
-
Source: youtube.com
Title: Study Strategies: Retrieval Practice
Link: https://www.youtube.com/watch?v=Pjrqc6UMDKMSource snippet
Making Low-Stakes Practice Tests More Effective...
-
Source: youtube.com
Title: How to Master Active Recall (Step-by-Step for Students)
Link: https://www.youtube.com/watch?v=f-XceGoIdNgSource snippet
Study Strategies: Retrieval Practice...
Topic Tree