psychometrician professor

What is a psychometrician?  A psychometrician is someone who practices psychometrics – the Science of Assessment.  That is, they are familiar with the scientific literature devoted to the development of fair, high-quality assessments, and they use this knowledge to improve assessments.  Psychometricians learn about many different topics, and can take a number of slants, such as applied vs. academic, or quantitative vs. test development.  Also, in some parts of the world, psychometrician refers to someone who administers tests, typically in an employment or counseling setting, and does not actually know anything about psychometrics.  We will not consider that here since I consider that usage to be bogus.

What does a psychometrician do?

There are many steps that go into developing a high quality, defensible assessment.  These differ by the purpose of the test.  When working on professional certifications or employment tests, a job analysis is typically necessary.  It’s totally irrelevant for K-12 formative assessments; the test is based on a curriculum.

Some topics include:

  • Analyzing test data with item response theory or classical test theory to evaluate item performance
  • Linking and equating
  • Job analysis and test blueprint design
  • Standard setting cutscore studies like modified-Angoff studies
  • Item writing workshops
  • Assessment program design and management
  • Software design and development

Where does a psychometrician work?

They work any place that develops high-quality tests.  Some examples:

 

Are all psychometricians created equal?

Absolutely not!  Like any other profession, there are levels of expertise and skill.  I liken it to top-level athletes: there are huge differences between what constitutes a good football/basketball/whatever player in high school, college, and the professional level.  And the top levels are quite elite; many people who study psychometrics will never achieve them.

Personally, I group psychometricians into three levels:

Level 1: Practitioners at this level are perfectly comfortable with basic concepts and the use of classical test theory, evaluating items with P and Rpbis.  They also do client-facing work like Angoff studies; many Level 2 and Level 3 psychometricians do not enjoy this work.

Level 2: Practitioners at this level are familiar with advanced topics like item response theory, differential item functioning, and adaptive testing.  They routinely perform complex analyses with software such as Xcalibre.

Level 3: Practitioners at this level contribute to the field of psychometrics.  They invent new statistics/algorithms, develop new software, publish books, start successful companies, or otherwise impact the testing industry and science of psychometrics in some way.

Note that practitioners can certainly be extreme experts in other areas: someone can be an internationally recognized expert in Certification Accreditation or Pre-Employment Selection but only be a Level 1 psychometrician because that’s all that’s relevant for them.  They are a Level 3 in their home field.

Do these levels matter?  To some extent, they are just my musings.  But if you are hiring a psychometrician, either as a consultant or an employee, this differentiation is worth considering!

Want to improve the quality of your assessments?

Sign up for our newsletter and hear about our free tools, product updates, and blog posts first! Don’t worry, we would never sell your email address, and we promise not to spam you with too many emails.

Newsletter Sign Up
First Name*
Last Name*
Email*
Company*
Market Sector*
Lead Source

The following two tabs change content below.

nthompson

Nathan Thompson earned his PhD in Psychometrics from the University of Minnesota, with a focus on computerized adaptive testing. His undergraduate degree was from Luther College with a triple major of Mathematics, Psychology, and Latin. He is primarily interested in the use of AI and software automation to augment and replace the work done by psychometricians, which has provided extensive experience in software design and programming. Dr. Thompson has published over 100 journal articles and conference presentations, but his favorite remains https://pareonline.net/getvn.asp?v=16&n=1.

Latest posts by nthompson (see all)