Nathan Thompson articles
What are technology enhanced items?
Technology-enhanced items are assessment items (questions) that utilize technology to improve the interaction of a test question in digital assessment, over and above what is possible with paper. Tech-enhanced items can improve examinee engagement (important
Modified-Angoff Method Study
A modified-Angoff method study is one of the most common ways to set a defensible cutscore on an exam. It therefore means that the pass/fail decisions made by the test are more trustworthy than if
What is Item Response Theory (IRT)?
Item response theory (IRT) is a family of machine learning models in the field of psychometrics, which are used to design, analyze, validate, and score assessments. It is a very powerful psychometric paradigm that allows
Criterion-related validity
Criterion-related validity is evidence that test scores are related to other data which we expect them to be. This is an essential part of the larger issue of test score validity, which is providing evidence
Why PARCC EBSR Items Provide Bad Data
The Partnership for Assessment of Readiness for College and Careers (PARCC) is a consortium of US States working together to develop educational assessments aligned with the Common Core State Standards. This is a daunting task,
What is Test Scaling?
Scaling is a psychometric term regarding the establishment of a score metric for a test, and it often has two meanings. First, it involves defining the method to operationally scoring the test, establishing an underlying
Certification Exam Development and Delivery
Certification exams are a critical component of workforce development for many professions and play a significant role in the global Testing, Inspection, and Certification (TIC) market, which was valued at approximately $359.35 billion in 2022
What is a T score?
A T Score is a conversion of scores on a test to a standardized scale with a mean of 50 and standard deviation of 10. This is a common example of a scaled score in
What is the Spearman-Brown formula?
The Spearman-Brown formula, also known as the Spearman-Brown Prophecy Formula or Correction, is a method used in evaluating test reliability. It is based on the idea that split-half reliability has better assumptions than coefficient alpha
Item Writing: Tips for Authoring Test Questions
Item writing (aka item authoring) is a science as well as an art, and if you have done it, you know just how challenging it can be! You are experts at what you do, and
Validity Threats and Psychometric Forensics
Validity threats are issues with a test or assessment that hinder the interpretations and use of scores, such as cheating, inappropriate use of scores, unfair preparation, or non-standardized delivery. It is important to establish a
Item Review Workflow for Exam Development
Item review is the process of ensuring that newly-written test questions go through a rigorous peer review, to ensure that they are high quality and meet industry standards. What is an item review workflow? Developing
Likert Scale Items
Likert scales (items) are a type of item used in human psychoeducational assessment, primarily to assess noncognitive constructs. That is, while item types like multiple choice or short answer are used to measure knowledge or
What are Stackable Credentials?
Stackable credentials refer to the practice of accumulating a series of credentials such as certifications or microcredentials over time. In today’s rapidly evolving job market, staying ahead of the curve is crucial for career success. Employers
Test Blueprints & Specifications
Test blueprints, aka test specifications (shortened to “test specs”), are the formalized design of an assessment, test, or exam. This can be in the context of educational assessment, pre-employment, certification, licensure, or any other type.
What is a testlet?
Testlet is a term in educational assessment that refers to a set of test items or questions grouped together on a test, often with a common theme or scenario. This approach aims to provide a
What is Job Task Analysis for Certification?
Job Task Analysis (JTA) is an essential step in designing a test to be used in the workforce, such as pre-employment or certification/licensure, by analyzing data on what is actually being done in the job.
Question and Test Interoperability (QTI)
Question and Test Interoperability® (QTI®) is a set of standards around the format of import/export files for test questions in educational assessment and HR/credentialing exams. This facilitates the movement of questions from one software platform