Borderline group method standard setting
Content
The borderline group method of standard setting is one of the most common approaches to establishing a cutscore for an exam. In comparison with the item-centered standard setting methods such as modified-Angoff, Nedelsky, and Ebel, there are two well-known examinee-centered methods (Jaeger, 1989), the contrasting groups method and the borderline group method (Livingston & Zieky, 1989). This post will focus on the latter one.
The concept of the borderline group method
Examinee-centered methods require participants to judge whether an individual examinee possesses adequate knowledge, skills, and abilities across specific content standards. The borderline group method is based on the idea to determine a common passing score for all examinees that would be expected from an examinee whose competencies are on the borderline between quite adequate and yet not inadequate.
How to perform the borderline group method
First of all, the judges are selected from those who are thoroughly familiar with the content examined and are knowledgeable about knowledge, skills, and abilities of individual examinees. Next, the judges engage in a discussion to develop a description of an examinee who is on the borderline between two extremes, mastery and non-mastery. Alternatively, the judges may be tasked to sort examinees into three categories: clearly competent, clearly incompetent, and those in-between.
After the description is agreed upon, borderline examinees need to be identified. The ultimate goal of the borderline group method is to distribute the borderline examinees’ scores and to find the median of that distribution (50th percentile), which would become the recommended cut score for the borderline group.
Why is the median used and not the mean, you might ask? The reason is that the median is much less affected by extremely high or extremely low values. This feature of the median is particularly important for the borderline group method, because an examinee with a very high or very low score is likely to not really belong in the group.
Analyzing the borderline group method
Advantages of this method:
- Time efficient
- Straightforward to implement
Disadvantages of this method:
- Difficult to achieve consensus on the nature borderline examinees
- The cut score could have low validity in case of a small number of borderline examinees
What could the borderline group method work poorly and how this can be tackled?
| Possible issue | Probable solution |
| The judges could identify some examinees as borderline by mistake (e.g. their skills were difficult to judge), so the borderline group might contain examinees who do not belong it. | Remind the judges not to include in the borderline group any examinees whose competencies they are not sure about. |
| The judges may base their judgements on something other than what the examine measures. | Give the judges appropriate instructions and get them agree with each other when defining a borderline examinee. |
| The judgements in terms of individual standards regarding the examinees’ skills and abilities may differ greatly. |
There is also a risk that judges would be sensitive to errors of central tendency and, therefore, might assign a disproportionately large number of examinees to the borderline group if they do not have sufficient knowledge about individual examinees’ performances. Thus, it is key for implementing the borderline group method to pick highly competent judges.
Conclusion
Let’s summarize which steps need to be made to implement the borderline group method:
- Select the competent judges
- Define the borderline level of examinees’ knowledge, skills, and abilities
- Evolve the borderline examinees
- Obtain the test scores of the borderline examinees
- Calculate the cut off score as the median of the distribution of the borderline examinees’ test scores
References
Jaeger, R. M. (1989). Certification of student competence.
Livingston, S. A., & Zieky, M. J. (1989). A comparative study of standard-setting methods. Applied Measurement in Education, 2(2), 121-141.
Laila Issayeva M.Sc.
Latest posts by Laila Issayeva M.Sc. (see all)
- What is a z-score? - November 28, 2024
- Confidence Intervals in Assessment and Psychometrics - November 25, 2024
- General Intelligence and Its Role in Assessment and Measurement - November 10, 2024
