An item distractor, also known as a foil or a trap, is an incorrect option for a selected-response item on an assessment.
What makes a good item distractor?
One word: plausibility. We need the item distractor to attract examinees. If it is so irrelevant that no one considers it, then it does not do any good to include it in the item. Consider the following item.
What is the capital of the United States of America?
A. Los Angeles
B. New York
C. Washington, D.C.
D. Mexico City
The last option is quite implausible – not only is it outside the USA, but it mentions another country in the name, so no student is likely to select this. This then becomes a three-horse race, and students have a 1 in 3 chance of guessing. This certainly makes the item easier.
How much do distractors matter? Well, how much is the difficulty affected by this new set?
What is the capital of the United States of America?
A. Paris
B. Rome
C. Washington, D.C.
D. Mexico City
In addition, the distractor needs to have negative discrimination. That is, while we want the correct answer to attract the more capable examinees, we want the distractors to attract the lower examinees. If you have a distractor that you thought was incorrect, and it turns out to attract all the top students, you need to take a long, hard look at that question! To calculate discrimination statistics on distractors, you will need software such as Iteman.
What makes a bad item distractor?
Obviously, implausibility and negative discrimination are frequent offenders. But if you think more deeply about plausibility, the key is actually plausibility without being arguably correct. This can be a fine line to walk, and is a common source of problems for items. You might have a medical item that presents a scenario and asks for a likely diagnosis; perhaps one of the distractors is very unlikely so as to be essentially implausible, but it might actually be possible for a small subset of patients under certain conditions. If the author and item reviewers did not catch this, the examinees probably will, and this will be evident in the statistics. This is one of the reasons it is important to do psychometric analysis of test results; in fact, accreditation standards often require you to go through this process at least once a year.
Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world.
Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He’s published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.
