A Center for Psychometric Research

Dedicated to defining the science of assessment

psychometric-tests-measure-mental-processes

We have been leading the application of technology to solving problems in the assessment industry since the 1970s. Here’s a portion of the psychometric research that has been accomplished here.

Interested leveraging this expertise in improving your assessments?

Click here for a free consultation with a psychometric expert

2020

Thompson, N.A. (2020). The Future of Assessment: How Will AI, Automation, and Machine Learning Change How We Develop and Deliver Assessments? In M. Khine (Ed.) Contemporary Perspectives on Research in Educational Assessment. Charlotte, NC: Information Age Publishing.

Thompson, N.A., & Curi, M. (2020). Adaptive Testing. In W. van der Linden (Ed.), Guidelines for Technology Based Assessment. Under review.

Ferris, M. & Thompson, N.A. (2020). Technology enhanced items. White paper published by the Institute for Credentialing Excellence. Under review.

Thompson, N.A. (2020). Ada: AI Driven Assessment. Product Demo at the Middle East & Africa Association of Test Publishers, Muscat, Oman

Thompson, N.A. (2020). So New Yet So Old: AI & Machine Learning in Educational Assessment. Presentation at the Middle East & Africa Association of Test Publishers, Muscat, Oman.

2019

Thompson, N.A. (2019). AI and Machine Learning in Psychometrics? Old news. Presentation at the 2019 Conference of the International Association for Computerized Adaptive Testing.

Austria, F., Thompson, N., Ortiz-Saavedra, B.D., Astudillo-García, C.I., Acosta-Castillo, I.G. & Sosa-Ortiz A.L. (2019). 10/66 Dementia Research Group Cognitive Test Battery: results from IRT and CAT-simulation studies. Presentation at the 2019 Conference of the International Association for Computerized Adaptive Testing.

Thompson, N.A., & Sahin, A. (2019). Feasibility of Using Small Samples to Train an Automated Essay Scoring Engine. Presentation at the 2019 Conference of the International Association for Computerized Adaptive Testing.

Thompson, N.A. (2019). So New Yet So Old: AI & Machine Learning in Educational Assessment. Presentation at the Conference on Artificial Intelligence and Big Data in Education, Beijing, China.

Thompson, N.A. (2019). Iteman/Xcalibre: Psychometrics as a service. Presentation at the Innovations in Testing Conference, Orlando, FL.

2018

Wang, Z. (2018). Does it matter which IRT Software you use? Yes. Technical Report.

2017

Thompson, N.A., Lunstad, A., Matthews-Lopez, J. (2017). SIFT & Winnow: Options for Data Forensics. Presentation at the Innovations In Testing conference, Scottsdale, AZ.

Thompson, N.A., Lunstad, A., & Jaquith, P. (2017). Tech-Enhanced Items: How can they provide better measurement? Presentation at the Innovations In Testing conference, Scottsdale, AZ.

Thompson, N.A., Stoeger, J.N. (2017). Evaluation of Parameter Recovery, Drift, and DIF with CAT Data. Presentation at the Sixth Annual Conference of the International Association for Computerized Adaptive Testing, Niigata, Japan.

Thompson, N.A. (2017). CAT: An Integrated Perspective. Presentation at the Sixth Annual Conference of the International Association for Computerized Adaptive Testing, Niigata, Japan.

Hurtado, H.G. & Thompson, N.A. (2017). Adaptive Testing in Higher Education Admissions Assessments. Presentation at the Second International Congress on Educational Innovation in Monterrey, Mexico.

Thompson, N.A. & Davis, J. (2017). The New SETI Project: Tools to help the search for examination treachery and iniquity. Presentation at the Conference of Test Security in Madison, WI.

McNeal, K.D., Stoeger, J.N., & Kreun, A.L. (2017). 100 Years of Titles in the Journal of Applied Psychology. The Industrial-Organizational Psychologist.

2016

Donath, C. and Crabtree, J. (2016). Is Automated Test Development Really an Option for My Organization? e-Poster presented at the Institute for Credentialing Excellence Exchange, Colorado Springs, CO.

Thompson, N.A. (2016). Statistical Methods of Detecting Test Fraud: Can We Get More Practitioners on Board? Paper presented at the Conference on Test Security, Cedar Rapids, IA.

Thompson, N.A., & Ausman, T. (2016). SIFT: Software for Investigating Fraud in Testing. Paper presented at the Conference on Test Security, Cedar Rapids, IA.

Thompson, N.A., & Ausman, T. (2016). Statistical Detection: Where Do I Start? Paper presented at the Conference on Test Security, Cedar Rapids, IA.

Thompson, N.A., & Ausman, T. (2016). Let’s Find Those Needles in the Haystack: Options for Data Forensics. Paper presented at the Conference of the International Testing Council, Vancouver, Canada.

Thompson, N.A., & Ausman, T. (2016). Adaptive Testing: How Can We Better Leverage? Paper presented at the Conference of the International Testing Council, Vancouver, Canada.

Thompson, N., & Kinart, C. (2016). One size does not fit all: Making test security configurable and scalable. ICE Digest, Q2 2016.

Thompson, N.A., Weiss, D.J., & Peters, K. (2016). Let the CAT out of the Bag: Making Adaptive Testing More Accessible and Feasible. Presentation at the Innovations In Testing conference, Orlando, FL.

Ausman, T., Thompson, N.A., & Kassner, D. (2016). Is Remote Proctoring Really Less Secure than In-Person Proctoring? Presentation at the Innovations In Testing conference, Orlando, FL.

Thompson, N.A., Ausman, T., & Peters, K. (2016). The Best of Both Worlds: Leveraging TEIs Without Sacrificing Psychometrics. Presentation at the Innovations In Testing conference, Orlando, FL.

Thompson, N.A. (2016). FastTest: A Comprehensive System for Assessment. Presentation at the Innovations In Testing conference, Orlando, FL.

2015

Sahin, A., Weiss, D.J. (2015). Effects of Calibration Sample Size and Item Bank Size on Ability Estimation in Computerized Adaptive Testing. Educational Sciences: Theory & Practice 15(6).

Sie, H., Finkelman, M. Bartroff, J., & Thompson, N. (2015). Stochastic Curtailment in Adaptive Mastery Testing: Improving the Efficiency of Confidence-Interval-Based Stopping Rules. Applied Psychological Measurement, available online Dec. 16, 2014.

Thompson, N.A. (2015). Introduction to Computerized Adaptive Testing. Workshop presented at the Fifth Annual Conference of the International Association for Computerized Adaptive Testing, Cambridge, England.

Thompson, N.A. (2015). Practical issues in developing a computerized adaptive test. Paper presented at the Fifth Annual Conference of the International Association for Computerized Adaptive Testing, Cambridge, England.

Padhke, C., Weiss, D., & Crist, T. (2015). Application of the Adaptive Measurement of Change to Measuring Achievement Growth in Reading and Mathematics. Paper presented at the Fifth Annual Conference of the International Association for Computerized Adaptive Testing, Cambridge, England.

Sahin, A., & Weiss, D. (2015). Effects of errors in item parameter estimates on recovery of theta estimates in CAT. Paper presented at the Fifth Annual Conference of the International Association for Computerized Adaptive Testing, Cambridge, England.

2014

Donath, C., & Thompson, N.A. (2014). Best Practices in Utilizing SMEs for Exam Development. Poster presented at the 2014 Institute for Credentialing Excellence Exchange, San Antonio, Texas.

Thompson, N.A. (2014). Introduction to Computerized Adaptive Testing. Workshop presented at the Fourth Annual Conference of the International Association for Computerized Adaptive Testing, Princeton, New Jersey.

Levy, A., Donoghue, A., Bailey, B. Thompson, N., Jamoulle, O., Gravel, J. (2014). External Validation of Scoring Instruments for Evaluating Pediatric Resuscitation. Simulation in Healthcare, 9(6), 360-369.

2013

Thompson, N.A., & Donath, C. (2013). Can I do CAT?. Poster presented at the Institute for Credentialing Excellence 2013 Exchange, Amelia Island, FL.

2012

Ahmadi, A., & Thompson, N.A. (2012). Issues Affecting Item Response Theory Fit in Language Assessment: A Study of Differential Item Functioning in the Iranian National University Entrance Exam. Journal of Language Teaching and Research, 3(3), 401-412.

Thompson, N.A. (2012). Developing an Accredited Certification Program. Saint Paul, MN: Assessment Systems Corporation.

Thompson, N.A. (2012). CAT, IACAT, and the future. Keynote presentation at the Third Annual Conference of the International Association for Computerized Adaptive Testing, Sydney, Australia.

Thompson, N.A. (2012). Developing an Online CAT. Paper presented at the Third Annual Conference of the International Association for Computerized Adaptive Testing, Sydney, Australia.

Thompson, N.A. (2012). Multiple cutscore computerized classification testing with the generalized likelihood ratio test. Paper presented at the Third Annual Conference of the International Association for Computerized Adaptive Testing, Sydney, Australia.

Weiss, D.J. (in press). Item Banking, Test Development, and Test Delivery. 2011 APA Handbook on Testing and Assessment, American Psychological Association.

2011

Assessment Systems Corporation (2011). A Comparison of Item Parameter Estimates from Xcalibre 4.1 and BILOG-MG. Saint Paul, MN: Assessment Systems Corporation.

Guyer, R., & Thompson, N.A. (2011). Item Response Theory Parameter Recovery Using Xcalibre 4.1. Saint Paul, MN: Assessment Systems Corporation.

Weiss, D.J. (2011). Better Data From Better Measurements Using Computerized Adaptive TestingJournal of Methods and Measurement in the Social Sciences, 2(1), 1-23.

Weiss, D.J., & von Minden, S. (2011). Measuring Individual Growth With Conventional and Adaptive Tests. Journal of Methods and Measurement in the Social Sciences, 2(1), 80-101.

Thompson, N.A., & Weiss, D.J. (2011). A Framework for the Development of Computerized Adaptive Tests Practical Assessment, Research, and Evaluation, 16(1). Also presented at the 2011 Innovations in Testing Conference, Phoenix, AZ. Also available translated into Arabic here.

Thompson, N.A. (2011). Termination Criteria for Computerized Classification Testing. Practical Assessment, Research, and Evaluation, 16(4).

Thompson, N.A. (2011). Likelihood Ratio Based Computerized Classification Testing. Paper presented at the 2011 conference of the National Council for Measurement in Education, New Orleans, LA. Previous versions presented at CLEAR 2010 and AEA-Europe 2010.

Schultz, Matthew T., Thompson, Nathan A., & Vestal, David. (October, 2011). Theta Stability as a Function of Items Attempted. Paper presented at the International Association of Computerized Adaptive Testing (IACAT). Pacific Grove, CA.

2010

Thompson, N.A. (2010). Nominal Error Rates in Computerized Classification Testing. Paper presented at the First Annual Conference of the International Association for Computerized Adaptive Testing, Arnhem, The Netherlands.

Guyer, R. (2010). Effect of theta estimation method and starting value on the recovery of theta. Paper presented at the First Annual Conference of the International Association for Computerized Adaptive Testing, Arnhem, The Netherlands.

Thompson, N.A. (2010). Adaptive Testing – Is It Right for Me?. Saint Paul, MN: Assessment Systems Corporation.

Thompson, N.A. (2010). Exporting Test Data from Spreadsheets. Saint Paul, MN: Assessment Systems Corporation.

Thompson, N.A. (2010). Why use an item banker? Saint Paul, MN: Assessment Systems Corporation.

Finkelman, M., Weiss, D.J., & Kim-Kang, G. (2010). Item Selection and Hypothesis Testing for the Adaptive Measurement of Change. In press, Applied Psychological Measurement.

2009

Thompson, N.A. (2009). Item Selection in Computerized Classification Testing. Educational and Psychological Measurement, 69, 778-793.

Thompson, N.A., Ro, S., Smith, L., & Santos, J. (2009). Innovative item types require innovative analysis. Paper presented at the National Organization for Competency Assurance Conference, Phoenix, AZ.

Thompson, N.A. (2009). Utilizing the generalized likelihood ratio as a termination criterion. Presentation at the GMAC Conference on Computerized Adaptive Testing, Minneapolis, MN.

Thompson, N.A. (2009). The MEDPRO Project: An SBIR Project for a Comprehensive IRT and CAT Software System CAT Software. Presentation at the GMAC Conference on Computerized Adaptive Testing, Minneapolis, MN.

Thompson, N.A. (2009). Ability estimation with item response theory. Saint Paul, MN: Assessment Systems Corporation.

Thompson, N.A. (2009). Classical item and test analysis with CITAS. Saint Paul, MN: Assessment Systems Corporation.

Kim-Kang, G., & Weiss, D.J. (2009). Comparison of Computerized Adaptive Testing and Classical Methods for Measuring Individual Change . Presentation at the GMAC Conference on Computerized Adaptive Testing, Minneapolis, MN.

2008

Thompson, N.A. (2008). A proposed framework of test administration methods. Journal of Applied Testing Technology, 9(5).

Thompson, N.A. (2008). Adaptive and interactive assessments with eLearning. Learning Circuits, November 2008.

Thompson, N.A., & Weiss, D.J. (2008). Computerized and Adaptive Testing in Educational Assessment. In Scheurmann, F. (Ed.), Transition to Computer-Based Assessment: New Approaches to Skills Assessment and Implications for Large-scale Testing. European Commission Joint Research Centre, Centre for Research on Lifelong Learning.

Thompson, N.A., Ro, S., & Eatchel, N.S. (2008). An economic model of certification. Paper presented at the 2008 Annual Meeting of the National Organization for Competency Assurance, San Francisco, CA.

Weiss, D.J., & Kim-Kang, G. (2008). Adaptive Measurement of Individual Change. Zeitschrift for Psychologie / Journal of Psychology, 216, 49-58.

2007 and earlier

Thompson, N.A. (2007). A practitioner’s guide for variable-length computerized classification testing. Practical Assessment, Research, and Evaluation, 12(1).

Yoes, M. (1995). An updated comparison of micro-computer based item parameter estimation procedures used with the 3-parameter IRT model. Saint Paul, MN: Assessment Systems Corporation.

Gierl, M.J., Ackerman, T. (1996). Software Review: XCALIBRE 1.10. Applied Psychological Measurement, 12(1), 53-67.

Vale, C.D., & Gialluca, K.A. (1988). Evaluation of the efficiency of item calibration. Applied Psychological Measurement, 12(1), 53-67.

Assessment Systems Corporation. (1987). A brief introduction to computerized testing. Chapter 2 of the MicroCAT User’s Manual (Second Edition). Saint Paul, MN: Author.

Vale, C.D. (1984). Measurement at the extremes of ability: A design for a computerized adaptive intelligence test. Paper presented at the 92nd Annual Convention of the American Psychological Association, Montreal, Canada.

Vale, C.D. (1981). Design and implementation of a microcomputer-based adaptive testing system. Behavior Research Methods and Instrumentation, 13(4), 399-406.