Nathan Thompson articles
Assessment Systems presents at the Conference on Test Security (COTS)
The Conference on Test Security is a scholarly conference devoted to the topic of security in educational, credentialing, and employment assessment. Test fraud/cheating is one of the most salient threats to score validity, and can
Four-Fifths Rule: Fair Employment Selection
The Four-Fifths Rule is a term that refers to a guideline for fairness in hiring practices in the USA. Because tests are often used in making hiring decisions, the Four-Fifths Rule applies to them so
Classical Test Theory: Item Statistics
Classical Test Theory (CTT) is a psychometric approach to analyzing, improving, scoring, and validating assessments. It is based on relatively simple concepts, such as averages, proportions, and correlations. One of the most frequently used aspects
Content Validity in Assessment
Content validity is an aspect of validity, a term that psychometricians use to refer to evidence that interpretations of test scores are supported. For example, predictive validity provides evidence that a pre-employment test will predict
Predictive Validity of Test Scores
Predictive Validity is a type of test score validity which evaluates how well a test predicts something in the future, usually with a goal of making more effective decisions about people. For instance, it is
Classical Test Theory vs. Item Response Theory
Classical Test Theory and Item Response Theory (CTT & IRT) are the two primary psychometric paradigms. That is, they are mathematical approaches to how tests are analyzed and scored. They differ quite substantially in substance
All Psychometric Models Are Wrong
The British statistician George Box is credited with the quote, “All models are wrong but some are useful.” As psychometricians, it is important that we never forget this perspective. We cannot be so haughty as
Best Online Proctoring Providers: The Alternative List You Need
There are many online proctoring providers on the market today, so how can you strategically shop for one? This post provides some considerations, and a list of most vendors, to help you with the process.
Certification Management System
A Certification Management System (CMS) or Credential Management System plays a pivotal role in streamlining the key processes surrounding the certification or credentialing of people, namely that they have certain knowledge or skills in a
ANSI ISO/IEC 17024 Accreditation
ANSI ISO/IEC 17024 accreditation is an internationally recognized standard for the accreditation of personnel certification bodies. That is, it is a stamp of approval from an independent audit which says your certification is good quality.
What is a Microcredential?
Microcredentials are short, focused, and targeted educational or assessment-based certificate programs that offer learners a way to acquire specific skills or knowledge in a particular field. In today’s fast-paced and rapidly evolving job market, traditional
What is Automated Essay Scoring?
Automated essay scoring (AES) is an important application of machine learning and artificial intelligence to the field of psychometrics and assessment. In fact, it’s been around far longer than “machine learning” and “artificial intelligence” have
Coefficient Alpha Reliability Index
Coefficient alpha reliability, sometimes called Cronbach’s alpha, is a statistical index that is used to evaluate the internal consistency or reliability of an assessment. That is, it quantifies how consistent we can expect scores to
Differential Item Functioning (DIF)
Differential item functioning (DIF) is a term in psychometrics for the statistical analysis of assessment data to determine if items are performing in a biased manner against some group of examinees. This analysis is often
“Dichotomous” Vs “Polytomous” in IRT?
What is the difference between the terms dichotomous and polytomous in psychometrics? Well, these terms represent two subcategories within item response theory (IRT) which is the dominant psychometric paradigm for constructing, scoring and analyzing assessments.
How do I develop a test security plan?
A test security plan (TSP) is a document that lays out how an assessment organization address security of its intellectual property, to protect the validity of the exam scores. If a test is compromised, the
Multistage Testing
Multistage testing (MST) is a type of computerized adaptive testing (CAT). This means it is an exam delivered on computers which dynamically personalize it for each examinee or student. Typically, this is done with respect
Automated Item Generation
Automated item generation (AIG) is a paradigm for developing assessment items (test questions), utilizing principles of artificial intelligence and automation. As the name suggests, it tries to automate some or all of the effort involved