Adaptive testing simulations with CATSim

Designed specifically to help you convert from traditional linear tests to adaptive tests

Improve your assessments by making them adaptive!

Computerized adaptive testing (CAT) is a sophisticated approach for delivering tests, surveys, and other measurements. It is based on complex computer algorithms that adapt the test to each examinee while also controlling for practical issues such as content distribution, item exposure, and test length. CATs have been shown to reduce test length by up to 90% without a loss of precision. However, to achieve such a reduction and make use of the advantages of CAT, it is absolutely essential that the CAT developer perform research studies to simulate the performance of CATs (see Thompson & Weiss, 2011).

Establish validity with adaptive testing simulations

CATSim implements three types of adaptive testing simulations: post-hoc (real-data) simulations, hybrid simulations, and monte-carlo simulations. These should be used in various stages of developing a CAT by documenting validity evidence for CAT options such as starting theta, theta estimation method, item selection method, item exposure control, and termination criterion. CATSim implements simulations for all three dichotomous item response theory (IRT) models and five polytomous IRT models. CATSim includes all of the CAT options in FastTest, our secure platform for online testing, so that the results of using CATSim can easily be implemented in your testing program.

Using CATSim

CATSim is designed to present options that directly represent specifications that must be made for CAT delivery. CAT requires the specification of 5 components:

Item bank: import your own, or generate feasible data directly in CATSim.
Initial theta: fixed, randomized, or read from existing data.
Three primary item selection algorithms, with subalgorithms representing practical concerns such as content distribution, item exposure, and enemy items.
Scoring: maximum likelihood, weighted maximum likelihood, or Bayesian (EAP or MAP) estimation.
Termination criteria, such as target SEM or information.

CATSim Output

CATSim records the adaptive testing simulations with item-by-item detail, but also provides a summary report describing the dependent variables necessary to adequately evaluate a CAT, such as average test length, accuracy, and item exposure.

Adaptive testing simulations with CATSim

Improve your assessments by making them adaptive!

Establish validity with adaptive testing simulations

Using CATSim

CATSim Output

ONLINE ASSESSMENT

PSYCHOMETRIC SOFTWARE

PSYCHOMETRIC SERVICES