student-testlet

What is a testlet?

Nathan Thompson, PhDJanuary 17, 2024

Testlet is a term in educational assessment that refers to a set of test items or questions grouped together on a test, often with a common theme or scenario. This approach aims to provide a more comprehensive and nuanced assessment of an individual’s abilities compared to traditional testing methods.

What is a testlet?

As mentioned above, a testlet is a group of items delivered together.  There are two ways of doing this.

  1. Items that share a common stimulus or otherwise MUST be together.  An example of this is a reading passage with 4 questions about it.  You can’t have the passage and the 4 questions scattered about a 100 item test as 5 screens in random places!  It all has to be together to make sense.
  2. Items that do not have to be together, but it improves the purpose of the assessment.  In this case, you might have 10 items that are standalone (no reading passage or anything relating them), but your test might be multistage testing and all items are delivered in blocks of 10.  Test designers can tailor the difficulty level based on the test-taker’s performance. As a test-taker progresses through a testlet, the system dynamically adjusts the complexity of subsequent questions, ensuring a personalized and accurate assessment of their proficiency.

Example item - testlet

 

Why use testlets?

The answer is obvious in the first case: you have to.  But it does get deeper than that.

One key feature of testlets is their ability to mimic real-world scenarios. Unlike standalone questions, testlets present a series of interconnected problems or tasks that require the test-taker to apply their knowledge in a cohesive manner. This not only assesses their understanding of isolated concepts but also evaluates their ability to integrate information and solve complex problems.  Testlets are can be particularly effective in assessing critical thinking, problem-solving skills, and practical application of knowledge. By presenting questions in a contextually linked manner, testlets offer a more authentic representation of a person’s ability to handle real-world challenges.

Testlets promote efficiency in testing. With a focused set of questions, they save time and reduce the fatigue associated with extensive testing sessions. This makes them an attractive option for educators and testing organizations seeking to streamline assessment processes while maintaining accuracy.  That is, if you want 20 items on reading comprehension, you could have 20 reading passages each with 1 question, or 4 reading passages each with 5 questions.  The fatigue would be far less in the latter test!

The second case, of standalone items, is a bit more nuanced.  It often has to do with managing the blueprints of the test, making best use of the item bank, and other operational considerations.  For example, perhaps the test has a blueprint to have 50% algebra items, 30% geometry, and 20% trigonometry.  You might build packets of 10 items with 5, 3, 2 respectively, and use those packets.

 

How do you score testlets?

Testlets can be scored with traditional methods, or with a new technology that was developed for this unique situation.

First, you can score with classical test theory, which is the traditional method of number-correct or points.

Second, you can use item response theory.  However, if the items share a strong relation, this might violate the IRT assumption of local independence.

Third, testlet response theory (TRT; Wainer, Bradlow, & Wang, 2007) works to address some of the concerns with traditional IRT.

 

Summary

In conclusion, a testlet is a powerful and flexible tool in toolbox of assessment designers. Its ability to present interconnected questions, mimic real-world scenarios, and adapt to individual performance makes it a valuable asset in gauging a person’s knowledge and skills. As education and assessment methods continue to evolve, the role of testlets is likely to expand, contributing to more accurate and meaningful evaluations of individuals in various fields.

The following two tabs change content below.
Avatar for Nathan Thompson, PhD
My Twitter profileMy LinkedIn profileMy YouTube channel

Nathan Thompson, PhD

Nathan Thompson earned his PhD in Psychometrics from the University of Minnesota, with a focus on computerized adaptive testing. His undergraduate degree was from Luther College with a triple major of Mathematics, Psychology, and Latin. He is primarily interested in the use of AI and software automation to augment and replace the work done by psychometricians, which has provided extensive experience in software design and programming. Dr. Thompson has published over 100 journal articles and conference presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/ .
/by