Linking & Equating
If your test has multiple forms out at one time, or switches to a new form every year, how do you know that results are comparable? Does a 67 on Form A mean the same as a 67 on Form B, or is Form B a little harder/easier? The answer to this question is nowhere near as simple as you might think.
Are you a government agency that is required to hire external contractors for QA/Replication efforts? Or other psychometric work that doesn’t fit into the large-scale multi-year contracts? We love that stuff.
We use machine learning and other quantitative methods to search for evidence of unusual data from students. This can range from simply low motivation to a classroom of copiers to widespread cheating efforts.
How do you define what scores mean? How should they be reported? How can you compare scores across grade levels? These are some thorny problems.
Item response theory
IRT is an application of machine learning (logistic regression on a latent trait) to solving some major issues in the world of assessment. We were one of the first organizations to adopt it, and remain one of the leaders.