If you are involved with certification testing and are accredited by the National Commission of Certifying Agencies (NCCA), you have come across the term decision consistency. NCCA requires you to submit a report of 11 important statistics each year, each for all active test forms. These 11 provide a high level summary of the psychometric […]
Sympson-Hetter is a method of item exposure control within the algorithm of Computerized adaptive testing (CAT). It prevents the algorithm from over-using the best items in the pool. CAT is a powerful paradigm for delivering tests that are smarter, faster, and fairer than the traditional linear approach. However, CAT is not without its challenges. One is […]
The standard error of the mean is one of the three main standard errors in psychometrics and psychology. Its purpose is to help conceptualize the error in estimating the mean of some population based on a sample. The SEM is a well-known concept from the general field of statistics, used in an untold number of […]
One of my graduate school mentors once said in class that there are three standard errors that everyone in the assessment or I/O Psych field needs to know: mean, error, and estimate. They are quite distinct in concept and application but easily confused by someone with minimal training. I’ve personally seen the standard error of […]
In the past decade, terms like machine learning, artificial intelligence, and data mining are becoming greater buzzwords as computing power, APIs, and the massively increased availability of data enable new technologies like self-driving cars. However, we’ve been using methodologies like machine learning in psychometrics for decades. So much of the hype is just hype. So, what […]
Last week, I had the opportunity to attend the 2017 Conference on Test Security (COTS), hosted by the University of Wisconsin-Madison. If your organization has any concerns about test security (that is, you have any sort of real stakes tied to your test!), I recommend that you attend COTS. It has a great mix of […]
Cognitive diagnostic models are a psychometric paradigm for designing and scoring tests with the goal of providing a profile of examinee skill mastery rather than just an overall test score. CDMS are an area of psychometric research that has seen substantial growth in the past decade, though the mathematics behind them, dating back to MacReady […]
What is a rubric? It’s a rule for converting unstructured responses on an assessment, such as essays that students write, into structured data that we can use psychometrically. Why do we need rubrics? Measurement is a quantitative endeavor. In psychometrics, we are trying to measure things like knowledge, achievement, aptitude, or skills. So we need […]
Today I read an article in The Industrial-Organizational Psychologist (the colloquial journal published by the Society for Industrial Organizational Psychology) that really resonated with me. Has Industrial-Organizational Psychology Lost Its Way? -Deniz S. Ones, Robert B. Kaiser, Tomas Chamorro-Premuzic, Cicek Svensson Why? Because I think a lot of the points they are making are also true […]
ASC attended the 2016 Conference on Test Security (COTS), held October 18-20 in Cedar Rapids IA, graciously hosted by Pearson. The conference brings together thought leaders on all aspects of test security, including statistical detection of test fraud, management of test centers, candidate agreements, investigations, and legal implications. ASC was lucky enough to win three […]
