Estimated reading time: 3 minutes
The Frary, Tideman, and Watts (1977) g2 index is a collusion (cheating) detection index, which is a standardization that evaluates a number of common responses between two examinees in the typical standardized format: observed common responses minus the expectation of common responses, divided by the expected standard deviation of common responses. It compares all pairs of examinees twice: evaluating examinee copying off b and vice versa.
The g2 collusion index starts by finding the probability, for each item, that the Copier would choose (based on their ability) the answer that the Source actually chose. The sum of these probabilities than the expected number of equivalent responses. We can then compare this to the actual observed number of equivalent responses and standardize that difference with the standard deviation. A very positive value could be possibly indicative of copying.
Cab = Observed number of common responses (e.g., both examinees selected answer D)
k = number of items i
Uia = Random variable for examinee a’s response to item i
Xia = Observed response of examinee b to item i.
Frary et al. estimated P using classical test theory, and the definitions are provided in the original paper, while a slightly more clear definitions are provided in Khalid, Mehmood, and Rehman (2011).
The g2 approach produces two half-matrices, which SIFT presents as a single matrix separated by a blank diagonal. That is, the lower half of the matrix evaluates whether examinee a copied off b, and the upper half whether b copied off a. More specifically, the row number is the copier and the column number is the source. So Row1/Column2 evaluates whether 1 copied off 2, while Row2/Column1 evaluates whether 2 copied off 1.
For g2 and Wollack’s (1997) ω, the flagging procedure counts all values in the matrix greater than the critical value, so it is possible – likely actually – that each pair will be flagged twice. So the numbers in those flag total columns will be greater than those in the unidirectional indices.
How to interpret? This collusion index is standardized onto a z-metric, and therefore can easily be converted to the probability you wish to use. A standardized value of 3.09 is default for g2, ω, and Zjk because this translates to a probability of 0.001. A value beyond 3.09 then represents an event that is expected to be very rare under the assumption of no collusion.
Want to implement this statistic? Download the SIFT software for free.
Nathan Thompson, PhD
Latest posts by Nathan Thompson, PhD (see all)
- Webinar Recording: A History of Computerized Adaptive Testing with Prof. David J. Weiss - June 14, 2022
- Incremental Validity - June 3, 2022
- Case Study: Escuela Superior de Administración Pública (ESAP), Colombia - June 2, 2022