Predictive Validity is a type of test score validity. Validity, in a general sense, is evidence that we have to support intended interpretations of test scores. There are different types of evidence that we can gather to do so. Predictive validity refers to evidence that the test predicts things that it should predict. If we have quantitative data to support such conclusions, it makes the test more defensible and can improve the efficiency of its use. For example, if a university admissions test does a great job of predicting success at university, then universities will want to use it to select students that are more likely to succeed.
Examples of Predictive Validity
Predictive validity evidence can be gathered for a variety of assessment types.
- Pre-employment: Since the entire purpose of a pre-employment test is to positively predict good things like job performance or negatively predict bad things like employee theft or short tenure, a ton of effort goes into developing tests to function in this way, and then documenting that they do.
- University Admissions: Like pre-employment testing, the entire purpose of university admissions exams is predictive. They should positively correlate with good things (first year GPA, four year graduation rate) and negatively predict the negative outcomes like academic probation or dropping out.
- Prep Exams: Preparatory or practice tests are designed to predict performance on their target test. For example, if a prep test is designed to mimic the Scholastic Aptitude Test (SAT), then one way to validate it is to gather the SAT scores later, after the examinees take it, and correlate with the prep test.
- Certification & Licensure: The primary purpose of credentialing exams is not to predict job performance, but to ensure that the candidate has mastered the material necessary to practice their profession. Therefore, predictive validity is not important, compared to content-related validity such as blueprints based on a job analysis. However, some credentialing organizations do research on the “value of certification” linking it to improved job performance, reduced clinical errors, and often external third variables such as greater salary.
- Medical/Psychological: There are some assessments that are used in a clinical situation, and the predictive validity is necessary in that sense. For instance, there might be an assessment of knee pain used during initial treatment (physical therapy, injections) that can be predictively correlated with later surgery. The same assessment might then be used after the surgery to track rehabilitation.
The case of pre-employment testing is perhaps the most common use of this type of validity evidence. A new study (Sacket, Zhang, Berry, & Lievens, 2022) was recently released that was a meta-analysis of the various types of pre-employment tests and other selection procedures (e.g., structured interview), comparing their predictive validity power. This was a modern update to the classic article by Schmidt & Hunter (1998). While in the past the consensus has been that cognitive ability tests provide the best predictive power in the widest range of situations, the new article suggests otherwise. It recommends the use of structured interview and job knowledge tests, which are more targeted towards the role in question, and therefore not surprising that they are well-performing. This in turn suggests that you should not buy pre-fab ability tests and use them in a shotgun approach with the assumption of validity generalization, but instead leverage an online testing platform like FastTest that allows you to build high-quality exams that are more specific to your organization.
Summary
Predictive validity is one type of test score validity, referring to evidence that scores from a certain test can predict their intended target variables. The most common application of it is to pre-employment testing, but it is useful in other situations as well. But validity is an extremely important and wide-ranging topic, so it is not the only type of validity evidence that you should gather.
Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world.
Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He’s published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.
