Contrasting Groups Tool

A cutscore that is not empircally evidenced is not legally defensible.  There are several accepted methods for setting a valid cutscore, and one of these is the contrasting groups method.  It has the advantage over others in that it can be used in a purely data-driven way, in addition to being used with subject matter expert (SME) ratings.

The goal of the contrasting groups method is to evaluate how test scores predict some sort of “gold standard” in classifying examinees.  One example is a practice test. Suppose you are delivering a practice test for a national certification exam. You have 100 people take your test, and you also gather their Pass/Fail results from the national exam.  Naturally, people who score higher on your exam are more likely to pass. We want to find the cutscore on your exam that best predicts this.

This “gold standard” can be a range of other classifications too.  Here are some more plausible situations:

• You are developing a pre-employment test, and you have supervisor ratings of whether each person had acceptable job performance
• You are developing an admissions test, and have data on whether each person graduated university within the expected 4 years
• You are developing a clinical survey, and have clinical diagnoses of each person
• You are developing a certification, and have SME ratings on whether each person should be certified (for example, based on a body of work)

Need to set a cutscore but don’t have a “gold standard”?  We recommend the Bookmark and modified-Angoff methods.

We provide the Contrasting Groups Tool at no cost, to help practitioners better establish cutscores.  Our mission at Assessment Systems is to improve assessment. If you would like to leverage our expertise to help your organization develop higher-quality assessments, please contact us!