ASC 2022 Logo no tagline 300

Contrasting Groups Method

contrasting groups method

The Contrasting Groups Method is a common approach to setting a cutscore.  It is very easy to do, but has the important drawback that some sort of “gold standard” is needed to assign examinees into categories such as Pass and Fail.  This “gold standard” should be unrelated to the test itself.

For example, suppose you wanted to set a cutscore on a practice test that is helping examinees determine if they are ready for a high-stakes certification test.  You might have past data for examinees who took both your practice exam and the actual certification test.  Their results from the certification test can be used to assign them to groups of Pass or Fail, and then you can evaluate the practice test score distributions for each group.  These distributions are typically smoothed, and their intersection represents an appropriate cutscore for the practice test.  In the example below, the two curves intersect near a score of 85, suggesting that this is an appropriate cutscore for the practice test that will closely predict the results of the official certification test.

I developed a simple tool in MS Excel that allows our psychometricians to easily produce both the smoothed and unsmoothed versions of this method, given nothing more than a list of practice test scores and “real” test classification for examinees.  Click here to download the tool.

contrasting groups method

Nathan Thompson, PhD

Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world.

Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He’s published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.

Share This Post

Facebook
Twitter
LinkedIn
Email

More To Explore

gamification in learning and assessment
Education

Gamification in Learning & Assessment

Gamification in assessment and psychometrics presents new opportunities for ways to improve the quality of exams. While the majority of adults perceive games with caution