Online proctoring has been around for over a decade by now. But given the recent outbreak of COVID-19, educational and workforce/certification institutions are scrambling to change their operations, and a huge part of this is an incredible surge in online proctoring. This blog post is intended to provide an overview of the online proctoring industry […]
An item distractor, also known as a foil or a trap, is an incorrect option for a selected-response item on an assessment. What makes a good item distractor? One word: plausibility. We need the item distractor to attract examinees. If it is so irrelevant that no one considers it, then it does not do any […]
What is item review? It is the process of performing quality control on items before they are ever delivered to examinees. This is an absolutely essential step in the development of items for medium and high stakes exams; while a teacher might not have other teachers review questions on a 4th grade math quiz, items […]
Technology enhanced items are assessment items (questions) that utilize technology to improve the interaction of the item, over and above what is possible with paper. Technology enhanced items can improve examinee engagement (important with K12 assessment), assess complex concepts with higher fidelity, improve precision/reliability, and enhance face validity/sellability. To some extent, the last word is […]
Automated item generation (AIG) is a paradigm for developing assessment items, aka test questions, utilizing principles of artificial intelligence and automation. As the name suggests, it tries to automate some or all of the effort involved with item authoring, as that is one of the most time-intensive aspects of assessment development – which is no […]
If you have worked in the field of assessment and psychometrics, you have undoubtedly encountered the word “standard.” While a relatively simple word, it has the potential to be confusing because it is used in three (and more!) completely different but very important ways. Here’s a brief discussion. Standard = Cutscore As noted by the […]
The generalized partial credit model (GPCM; Muraki, 1992) is one of the family of models from item response theory. It is designed to work, as you might have guess, with items that are partial credit. That is, instead of just right/wrong as possible scoring, an examinee can receive partial points for completing some aspects of […]
I was recently asked about scaled scoring and if the transformation must be based on the normal curve. This is an important question, especially since most human traits fall in a fairly normal distribution, and item response theory places items and people on that latent scale. The short answer is “no” – there are other […]
Computerized adaptive tests (CATs) are a sophisticated method of test delivery based on item response theory (IRT). They operate by adapting both the difficulty and quantity of items seen by each examinee. Difficulty Most characterizations of adaptive testing focus on how item difficulty is matched to examinee ability. High ability examinees receive more difficult items, […]
A Standard Setting Study is a formal process fo establishing an performance standard. In the assessment world, there are actually two uses of the word standard – the other one refers to a formal definition of the content that is being tested, such as the Common Core State Standards in the USA. For this reason, […]