The Modified-Angoff approach to standard-setting starts by defining the concept of a Minimally Competent Candidate, and then SMEs estimate the percentage of MCCs that will get each question right. Typically, the results are analyzed and SMEs are made to discuss items with disagreement, and then updated ratings provided which hopefully have more consensus. Taking the average of all raters and items gives you the score that the raters would expect for an MCC.
The Bookmark Method of standard-setting assumes that you have statistics on all your items – meaning they have all been used already, even if in an unscored manner. Of course, this isn’t always the case, but if it is, then the Bookmark Method is a compelling approach. Items are printed in a booklet (or online) in the order of their difficult, and your experts place a bookmark at the point where they think the passing score should be. You then average this across raters.
There are a number of other standard-setting methodologies supported by the scientific research. We can help you select one that makes sense for you. For example, the Contrasting Groups method is very useful if you are doing pass/fail exams that are a practice test and you have data on the “real” exam for your candidates.
