I just returned from the 2017 Innovations in Testing conference in Scottsdale, AZ, organized by the Association of Test Publishers. This is one of my favorite conferences because it contains a compelling blend of ingredients – quality psychometrics, innovative technology, and the business of assessment – and the 2017 edition definitely did not disappoint.
Takeaways of the conference
We attended presentations on a variety of topics, and it was evident that AI and Big Data are playing an increasingly impactful role in our field. We are excited to be on the forefront of that; we’ve been doing Machine Learning since the 1970s, long before that term became en vogue. Much of our current innovation effort is focused on automation of psychometrics, such as automated creation of NCCA Accrediation Reports and Classical Test Theory Reports.
Exhibit Hall
ATP organized an exhibit hall for the testing conference on Monday and Tuesday, and we had a great high-traffic location near the center of the room (between two food serving areas, which is always nice!). In addition to the normal handouts, we distributed scorpion suckers and gave away a Google Home device, a nod towards our love of AI and machine learning.
ASC’s Presentations
ASC was grateful for the opportunity to speak 4 times about the innovations we are making in the testing industry – certainly a great fit, given the title of the conference.
Tech-Enhanced Items: How Can They Provide Better Measurement?
Nathan Thompson and Andrew Lunstad from ASC, and Paul Jaquith from ACTVET, discussed tech-enhanced items from a psychometric and software design perspective, evaluating their impact on the ultimate goal: improving measurement.
Sift & Winnow: Options For Data Forensics
Nathan Thompson from ASC, and Joy Matthews-Lopez from Professional Testing Inc., discussed methodologies and software to help detect instances of test fraud, and what to do about it if you find anything worth investigating.
FastTest: A Configurable System for Cross-Sector Assessment
Andrew Lunstad demonstrated FastTest, a secure, cloud-based platform uniquely designed to improve any type of assessment, from certification to pre-employment to education to psychology.
SIFT: Innovating Data Forensics
Nathan Thompson demonstrated SIFT, an innovative program designed as a cost-effective option for detecting test fraud and other threats to validity, such as low motivation.
Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world.
Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He’s published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.
