Last week, I had the opportunity to attend the 2017 Conference on Test Security (COTS), hosted by the University of Wisconsin-Madison. If your organization has any concerns about test security (that is, you have any sort of real stakes tied to your test!), I recommend that you attend COTS. It has a great mix of psychometric research with practical discussions such as policies and procedures. While it was originally titled “Conference on Statistical Detection of Test Fraud” it has since expanded its scope and thankfully reduced the number of syllables in the name.
The venue was the Pyle Center on the shores of Lake Mendota, just one block from the famous State Street. Madison is a beautiful city, situated on an isthmus between two large lakes; great for visuals but not so great for traffic patterns. The location was incredibly convenient for me, as it is driving distance from my home in Minnesota, and allowed me to stay with my family in nearby Watertown, watch my brother coach a high school football game in Columbus, and stop at the CamRock mountain bike trails that I’ve always wanted to try (highly recommend!).
One highlight of the conference was the chance to present with my friend, former colleague, and graduate school office-mate Jennifer Davis from the National Association of Boards of Pharmacy. We compared three software programs for psychometric forensics: SIFT, CopyDetect, and Outlier Detection Tool. SIFT and CopyDetect both provide several collusion indices, but SIFT provides more and is incredibly faster (CopyDetect took 2 hours to run 134 examinees). The Outlier Detection Tool is an internal spreadsheet used by NABP that serves a slightly different purpose; for more information, contact them.
The best part of the Conference on Test Security, just like the IACAT conference I just attended, was the chance to spend time with old friends that I only see once every year or two, as well as make new friends such as a researcher from ASC’s partner Ascend Learning. In fact, I didn’t even get a chance to attend any sessions on the second day, I instead spent the time talking to colleagues.
Biggest disappointment? I didn’t hang around until Saturday to attend the Badger game and join the traditional “Jump Around!”
Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world.
Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He’s published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.