Psychometrics is the science of educational and psychological assessment, using data to ensure that tests are fair and accurate. Ever felt like you took a test which was unfair, too hard, didn’t cover the right topics, or was full of questions that were simply confusing or poorly written? Psychometricians are the people who help organizations […]
Setting a cutscore on a test scored with item response theory (IRT) requires some psychometric knowledge. This post will get you started. How do I set a cutscore with item response theory? There are two approaches: directly with IRT, or using CTT then converting to IRT. Some standard setting methods work directly with IRT, such […]
Technology-enhanced items are assessment items (questions) that utilize technology to improve the interaction of a test question in digital assessment, over and above what is possible with paper. Tech-enhanced items can improve examinee engagement (important with K12 assessment), assess complex concepts with higher fidelity, improve precision/reliability, and enhance face validity/sellability. To some extent, the last […]
A modified-Angoff method study is one of the most common ways to set a defensible cutscore on an exam. It therefore means that the pass/fail decisions made by the test are more trustworthy than if you picked an arbitrary round number like 70%. If your doctor, lawyer, accountant, or other professional has passed an exam […]
Item response theory (IRT) is a family of machine learning models in the field of psychometrics, which are used to design, analyze, validate, and score assessments. It is a very powerful psychometric paradigm that allows researchers to build stronger assessments, whether they work in Education, Psychology, Human Resources, or other fields. It also solves critical […]
Criterion-related validity is evidence that test scores are related to other data which we expect them to be. This is an essential part of the larger issue of test score validity, which is providing evidence that test scores have the meaning we intend them to have. If you’ve ever felt that a test doesn’t cover […]
The Partnership for Assessment of Readiness for College and Careers (PARCC) is a consortium of US States working together to develop educational assessments aligned with the Common Core State Standards. This is a daunting task, and PARCC is doing an admirable job, especially with their focus on utilizing technology. However, one of the new item […]
Scaling is a psychometric term regarding the establishment of a score metric for a test, and it often has two meanings. First, it involves defining the method to operationally scoring the test, establishing an underlying scale on which people are being measured. A common example is the T-score, which transforms raw scores into a standardized […]
Certification exam development, is a well-defined process governed by accreditation guidelines such as NCCA, requiring steps such as job task analysis and standard setting studies. For certification, and other credentialing like licensure or certificates, this process is incredibly important to establishing validity. Such exams serve as gatekeepers into many professions, often after people have invested […]
A T Score (sometimes hyphenated T-Score) is a common example of a scaled score in psychometrics and assessment. A scaled score is simply a way to present scores in a more meaningful and easier-to-digest context, with the benefit of hiding the sometimes obtuse technicalities of psychometrics. Therefore, a T Score is a standardized way that […]
