Angoff Analysis Tool

Angoff Analysis Tool Does your testing program utilize the modified Angoff method to establish cutscores? ASC’s Angoff Analysis Tool (AAT) produces all the calculations you need to make a professional, defensible report of the results. Ultimately, the goal of many tests is to provide information to make decisions. For testing programs where this decision is to classify examinees into one of two groups, the directors of the program must establish a cutscore. Several methods for establishing the cutscore exist, but the most common is the modified Angoff method (Sireci & Biskin, 1992). A panel of subject matter experts (SMEs) is assembled, asked to define a reference group, and then rate each item in the test. The reference group can be definitions like minimally qualified job applicants, minimally competent professionals, or students with a minimal level of subject mastery. The value of the rating is the percentage of the reference group that will answer each item correctly. The average of all the ratings often then serves as the cutscore. However, to be defensible, the actual analysis of the ratings must involve much more than a simple average. Because the test's governing board, council, or committee has the final authority to decide on the cutscore (Mills, 1995), it is crucial to present adequate information for an informed decision. The AAT performs many additional calculations that are important in both the completion of the SME workshop and in the establishment of the final cutscore, including the estimation of raw score mean and SD given classical item statistics. AAT Output is critical – but easy to read! Output includes: Used during SME workshop Used in final report Item rating average Projected cutscore Item rating SD Projected pass rate Item rating minimum Projected score average Item rating maximum Projected score SD Projected cutscore Beuk and Hofstee adjustments Projected pass rate Projected pass rate with Beuk Projected cutscore for each SME Standard error of judgments Grouped frequencies of ratings Inter-rater reliability Min, max, and SD of SME cutscores AAT Input is simple – type or paste into sheet! AAT has two spreadsheets for input, for multiple rounds of ratings (Mills, 1995). Each spreadsheet handles up to 500 items and 50 raters. Unlike some psychometric software, the input and running of the program could not be easier. Simply type or paste your results into blue areas on the spreadsheet, and the results table will automatically populate! AAT Manual AAT comes with a pdf manual that provides detailed instructions as well as a background on modified Angoff studies. It is also useful in helping you learn more and decide if it will be a good tool for your organization to purchase, so it is available for free download. Download the AAT Manual (212 KB, zipped, PDF)   Order Options and Details   AAT - Academic/Government 99.00  Quantity: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 Add to cart AAT - Non-Academic/Government 199.00  Quantity: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 Add to cart

Send to friend

Your name: * Your e-mail: * Recipient's e-mail: *  Send to friend