Entries by Nathan Thompson, PhD

What is a Psychometrician? Definition, roles, & careers.

A psychometrician is a data scientist who studies how to develop and analyze exams so that they are reliable, valid, and fair. Using psychometrics, Psychometricians implement aspects of engineering, data science, and machine learning to ensure that tests provide accurate information about people, so we can be confident about decisions based on test scores.  They […]

What can Machine Learning tell us about our item banks?

Artificial intelligence (AI) and machine learning (ML) have become buzzwords over the past few years.  As I already wrote about, they are actually old news in the field of psychometrics.   Factor analysis is a classical example of ML, and item response theory (IRT) also qualifies as ML.  Computerized adaptive testing (CAT) is actually an application […]

The Generalized Partial Credit Model (GPCM)

The generalized partial credit model (GPCM, Muraki 1992) is an item response theory (IRT) model designed to work with items that are partial credit.  That is, instead of just right/wrong as possible, scoring an examinee can receive partial points for completing some aspects of the item correctly.  For example, a typical multiple-choice item is scored […]

R for psychometrics

If you are dealing with data science, which psychometrics most definitely is, you’ve probably come across  R. It is an environment that allows you to implement packages for many different types of analysis, which are built by a massive community of data scientists around the world. R has become one of the two main languages […]

,

Subject Matter Experts (SME) in Exam Development

Subject matter experts are an important part of the process in developing a defensible exam.  There are several ways that their input is required.  Here is a list from highest involvement/responsibility to lowest: Serving on the Certification Committee (if relevant) to decide important things like eligibility pathways Serving on panels for psychometric steps like Job […]

MICROCAT: What was assessment like in the 1980s?

ASC has been empowering organizations to develop better assessments since 1979.  Curious as to how things were back then?  Below is a copy of our newsletter from 1988, long before the days of sharing news via email and social media!  Our platform at the time was named MICROCAT.  This later became modernized to FastTest PC […]

What is the two parameter IRT model (2PL)?

Item response theory is the predominant psychometric paradigm for mid or large scale assessment.  As noted in my introductory blog post, it is actually a family of models.  In this post, we discuss the two parameter IRT model (IRT 2PL). Consider the following 3PL equation (simplified from Hambleton & Swaminathan, 1985, Eq. 3.3).  The IRT […]

What is the three parameter IRT model (3PL)?

Item response theory (IRT) is an extremely powerful psychometric paradigm that addresses many of the inadequacies of classical test theory (CTT).  If you are new to the topic, there is a broad intro here, where you will learn that IRT is actually a family of mathematical models rather than one specific one.  Today, I’m talking […]