assessment systems logo

What is a psychometrician?

psychometrician laptop

What is a psychometrician?  A psychometrician is someone who practices psychometrics – the Science of Assessment.  That is, they study the process of assessment development and validation itself, regardless of the type of assessment (certification, K-12, etc.).  They are familiar with the scientific literature devoted to the development of fair, high-quality assessments, and they use this knowledge to improve assessments.  Psychometricians learn about many different topics, and can take a number of slants, such as applied vs. academic, or quantitative vs. test development.

Also, in some parts of the world, the term psychometrician refers to someone who administers tests, typically in an employment or counseling setting, and does not actually know anything about the development or validation of tests.  That usage is incorrect; such a person is a psychometrist, as you can see at the website for the National Association of Psychometrists.

What does a psychometrician do?

There are many steps that go into developing a high quality, defensible assessment.  These differ by the purpose of the test.  When working on professional certifications or employment tests, a job analysis is typically necessary.  It’s totally irrelevant for K-12 formative assessments; the test is based on a curriculum.

Some topics include:multistage testing

This is a highly quantitative profession.  Psychometricians spend most of their time working with datasets, using specially designed software or writing code in languages like R and Python.

Where does a psychometrician work?

They work any place that develops high-quality tests.  Some examples:

 

How do I get a job as a psychometrician?

First, you need a graduate degree.  In this field, a Master’s degree is considered entry-level, and a PhD is considered a standard level of education.  It can often be in a related area like I/O psychology.

Wondering what kind of opportunities are out there?  Check out the NCME Job Board and Horizon Search, a headhunter for assessment professionals.

 

Are all they created equal?

Absolutely not!  Like any other profession, there are levels of expertise and skill.  I liken it to top-level athletes: there are huge differences between what constitutes a good football/basketball/whatever player in high school, college, and the professional level.  And the top levels are quite elite; many people who study psychometrics will never achieve them.

Personally, I group psychometricians into three levels:

Level 1: Practitioners at this level are perfectly comfortable with basic concepts and the use of classical test theory, evaluating items and distractors with P and Rpbis.  They also do client-facing work like Angoff studies; many Level 2 and Level 3 psychometricians do not enjoy this work.

Level 2: Practitioners at this level are familiar with advanced topics like item response theory, differential item functioning, and adaptive testing.  They routinely perform complex analyses with software such as Xcalibre.

Level 3: Practitioners at this level contribute to the field of psychometrics.  They invent new statistics/algorithms, develop new software, publish books, start successful companies, or otherwise impact the testing industry and science of psychometrics in some way.

Note that practitioners can certainly be extreme experts in other areas: someone can be an internationally recognized expert in Certification Accreditation or Pre-Employment Selection but only be a Level 1 psychometrician because that’s all that’s relevant for them.  They are a Level 3 in their home field.

Do these levels matter?  To some extent, they are just my musings.  But if you are hiring a psychometrician, either as a consultant or an employee, this differentiation is worth considering!

 

The following two tabs change content below.

Nathan Thompson, PhD

CEO at Assessment Systems
Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world. Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He's published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.

Share This Post

Facebook
Twitter
LinkedIn
Email

More To Explore

incremental validity
Psychometrics

Incremental Validity

Incremental validity is an aspect of validity that refers to what an additional assessment or predictive variable can add to the information provided by existing