Item Banks: 6 Ways To Improve

certification licensure exam laptop

The foundation of a decent assessment program is the ability to develop and manage strong item banks. Item banks are a central repository of test questions, each stored with important metadata such as Author or Difficulty. They are designed to treat items are reusable objects, which makes it easier to publish new exam forms.

Of course, the storage of metadata is very useful as well and provides validity documentation evidence. Most importantly, a true item banking system will make the process of developing new items more efficient (lower cost) and effective (higher quality).

1. Item writers are screened for expertise

Make sure the item writers (authors) that are recruited for the program will meet minimum levels of expertise. Often this involves a lot of years of experience in the field. You also might want to make sure their demographics are sufficiently distributed, such as specialty area or geographic region.

2. Item writers are trained on best practices

Item writers must be trained on best practices in item writing, as well as any guidelines provided by the organization. A great example is this book from TIMSS. ASC has provided their guidelines for download here. This facilitates higher quality item banks.

3. Items go through review workflow to check best practices

After items are written, they should proceed through a standardized workflow and quality assurance. This is the best practice in developing any products. The field of software development uses a concept called the Kanban Board, which ASC has implemented in its item banking platform.

Review steps can include psychometrician, bias, language editing, and course content.

4. Items are all linked to blueprint/standards

All items in the item banks should be appropriately categorized. This guarantees that no items are measuring an unknown or unneeded concept. Items should be written to meet blueprints or standards.

5. Item banks pilotingitem writing laptop paper

Items are all written with good intent. However, we all know that some items are better than others. Items need to be given to some actual examinees so we can obtain feedback, and also obtain data for psychometric analysis.

Often, they are piloted as unscored items before eventual use as “live” scored items. But this isn’t always possible.

6. Psychometric analysis of items

After items are piloted, you need to analyze them with classical test theory and/or item response theory to evaluate their performance. I like to say there are three possible choices after this evaluation: hold, revise, and retire. Items that perform well are preserved as-is.

Those of moderate quality might be modified and re-piloted. Those that are unsalvageable are slated for early retirement.

How to accomplish all this?

This process can be extremely long, involved, and expensive. Many organizations hire in-house test development managers or psychometricians; those without that option will hire organizations such as ASC to serve as consultants.

Regardless, it is important to have a software platform in place that can effectively manage this process. Such platforms have been around since the 1980s, but many organizations still struggle by managing their item banks with Word, PowerPoint, and Email!

ASC provides an item banking platform for free, which is used by hundreds of organizations. Click below to sign up for your own account.

Sign Up For Free Account

The following two tabs change content below.
Avatar for Nathan Thompson, PhD

Nathan Thompson, PhD

Nathan Thompson earned his PhD in Psychometrics from the University of Minnesota, with a focus on computerized adaptive testing. His undergraduate degree was from Luther College with a triple major of Mathematics, Psychology, and Latin. He is primarily interested in the use of AI and software automation to augment and replace the work done by psychometricians, which has provided extensive experience in software design and programming. Dr. Thompson has published over 100 journal articles and conference presentations, but his favorite remains .
Avatar for Nathan Thompson, PhD

Latest posts by Nathan Thompson, PhD (see all)