The Angoff Analysis Tool
A free spreadsheet to set cutscores that are legally defensible, using the modified-Angoff method with Beuk compromise and inter-rater reliability
Methods like the Angoff approach are necessary to be defensible or achieve NCCA/ANSI Accreditation and follow AERA/APA/NCME guidelines
Setting a better cutscore means that decisions made with the test will be more accurate
Leverage remote raters; interactively discuss Angoff results; improve reliability
The Angoff Analysis Tool supports best practices in implementing the modified-Angoff approach
Best of all, this is available for free: simply fill out the form on the right and you can download the spreadsheet.
Need a consultant to run the standard-setting study in a way that meets accreditation standards? Contact us and set up a time to talk to one of our experts.
Standard setting is the process of setting a cutscore or passing point on an exam, typically to indicate some sort of mastery. For example, by passing the test you might be considered certified to work in a profession or eligible to interview for a certain job. Obviously, there are important stakes associated with such exams, so it is not defensible to pick an arbitrary score as the cutscore. For example, you can’t just pick 70% because that’s what was considered passing on your 5th grade spelling tests. If you are making a criterion-referenced interpretation of the scores, you need a criterion-referenced method of setting the cutscore.
Here, it means some well-defined area of knowledge that is being assessed, such as knowledge of accounting practices or essential aspects of being a nurse. Unfortunately, criterion has another meaning in assessment too; it sometimes refers to an external variable used to provide corroborative evidence for test score interpretations, historically known as criterion-related validity.
There are several methods available, including modified-Angoff, Bookmark, Hofstee, Body Of Work, Borderline, and Contrasting Groups. This spreadsheet helps you implement the most common method, modified-Angoff, with or without the common add-on of the Beuk Compromise. It also supports the Hofstee method.
If you are a credentialing organization, using methods like these is a requirement of NCCA and ANSI accreditation.
A step-by-step overview is described in this blog post. You will need a pool of high-quality items, preferably ones that are already pretested. You will then need a panel of at least 6 subject matter experts (SMEs). Need an enterprise-class item banker platform to implement an Angoff study as part of the item authoring/review process? Contact us about FastTest.