With so many things to consider, it’s no wonder psychometricians often recommend the retirement of poor performing items. Here are some of the most common issues we see, along with our tried and true methods for designing good, psychometrically sound items. We could all use some reminders on good item writing tips. These are made easier if you use a strong item banking system like FastTest.
Issue |
Recommendation |
| Key is invalid due to multiple correct answers. | Consider each answer option individually; the key should be fully correct with each distractor being fully incorrect. |
| Item was written in a hard to comprehend way, examinees were unable to apply their knowledge because of poor wording.
|
Ensure that the item can be understood after just one read through. If you have to read the stem multiple times, it needs to be rewritten. |
| Grammar, spelling, or syntax errors direct savvy test takers toward the correct answer (or away from incorrect answers). | Read the stem, followed by each answer option, aloud. Each answer option should fit with the stem. |
| Information was introduced in the stem text that was not relevant to the question. | After writing each question, evaluate the content of the stem. It should be clear and concise without introducing irrelevant information. |
| Item emphasizes trivial facts. | Work off of a test blue print to ensure that each of your items map to a relevant construct. If you are using Bloom’s taxonomy or a similar approach, items should be from higher order levels. |
| Numerical answer options overlap. | Carefully evaluate numerical ranges to ensure there is no overlap among options. |
| Examinees noticed answer was most often A. | Distribute the key evenly among the answer options. This can be avoided with FastTest’s randomized delivery functionality. |
| Key was overly specific compared to distractors. | Answer options should all be about the same length and contain the same amount of information. |
| Key was only option to include key word from item stem. | Avoid re-using key words from the stem text in your answer options. If you do use such words, evenly distribute them among all of the answer options so as to not call out individual options. |
| Rare exception can be argued to invalidate true/false always/never question. | Avoid using “always” or “never” as there can be unanticipated or rare scenarios. Opt for less absolute terms like “most often” or “rarely”. |
| Distractors were not plausible, key was obvious. | Review each answer option and ensure that it has some bearing in reality. Distractors should be plausible. |
| Idiom or jargon was used; non-native English speakers did not understand. | It is best to avoid figures of speech, keep the stem text and answer options literal to avoid introducing undue discrimination against certain groups. |
| Key was significantly longer than distractors. | There is a strong tendency to write a key that is very descriptive. Be wary of this and evaluate distractors to ensure that they are approximately the same length. |
