Psychometric Analysis of Item and Test Performance
How do you know if your test is high quality, or how to improve it?
Our PhD psychometricians excel not only in utilizing best practices and leading-edge software to perform these complex quantitative analyses but providing the results with understandable explanations that allow you and your subject matter experts to truly increase the quality of your test by modifying the content appropriately.
From simple reports of classical test theory to complex equating & scaling studies with item response theory (IRT), trust our team to provide the psychometric analysis you need, and cost-effectively.
Why do I need psychometric analysis of my assessments?
An absolutely essential step in developing a defensible test is psychometric analysis of response data. This step provides detailed feedback on what is working well in the test, what is not, and why. This drives continual improvement, but also provides documentation of validity evidence.
What sort of work is involved in this process?
Our psychometricians will need response data from examinees as well as metadata on each item. Once you provide these, we clean the data then run it through our specially designed software. This software drafts the report for you in seconds, which means we can provide this service at a fraction of the cost that large vendors will charge you. Though of course, we first go through the results with a fine-toothed comb and write up our recommendations.
What sort of result do I get from this process?
Well, suppose that we analyze a test question whose correct answer is A, but find that only 40% of your candidates got it correct (item difficulty). Meanwhile, just as many examinees answered B – and it happened to be smarter examinees. This item would be flagged and discussed with your experts, who might point out something like “Only candidates who have completed a special training course would know how to answer this question.” Even though the answer is technically correct, the statistics have helped flag an issue with the item, and it might be retired or rewritten.
I know this stuff! Can we just do it ourselves?
Have psychometric talent in-house but want to make them more effective in conducting these analyses? Iteman and Xcalibre provide user-friendly interfaces and automatically create Microsoft Word reports with all the tables and graphs already embedded for you. Or, start using our best-in-class platform Assess.ai and have the reports ready-made for you!