Online proctoring has been around for over a decade. But given the recent outbreak of COVID-19, educational and workforce/certification institutions are scrambling to change their operations, and a huge part of this is an incredible surge in online proctoring. This blog post is intended to provide an overview of the online proctoring industry for someone that is new to the topic is just overwhelmed by all the many options out there!
Online Proctoring: Two Distinct Markets
First, I would describe the online proctoring industry as actually falling into two distinct markets, so the first step is to determine which of these fits your organization
- Large scale, lower cost (when large scale), lower security systems designed to be used only as a plugin to major LMS platforms like Blackboard or Canvas. These online proctoring systems are therefore designed for medium-stakes exams like an Intro to Psychology midterm at a university.
- Lower scale, higher cost, higher security systems designed to be used with standalone assessment platforms. These are generally for higher-stakes exams like certification or workforce, or perhaps special use at universities like Admissions and Placement exams.
How to tell the difference? The first type will advertise about easy integration with systems like Blackboard or Canvas as a key feature. They will also often focus on AI review of videos, rather than using real humans. Another key consideration is to look at the existing client base, which is often advertised.
Other ways that Online Proctoring systems can differ
AI vs humans:
Some systems rely purely on artificial intelligence algorithms to flag video recordings of examinees. Other systems utilize real humans.
Record & Review Humans vs. Real-Time Humans:
If live humans are used, there are two ways. First, it can be live and real-time, meaning that there is a human on the other end of the video that can confirm identity before allowing the test to start, and stop the test if there is obviously illicit activity. Record & Review will record the audio and a human will check it within 24-48 hours. This is more scalable, but you can’t stop the test if someone is stealing the content – you probably won’t know until the next day.
Screen capture:
Some online proctoring providers have an option to record/stream the screen as well as the webcam. Some also provide the option to only do this (no webcam) for lower stakes exams.
Mobile phone as the third camera:
Some newer platforms provide the option to easily integrate the examinee’s mobile phone as a third camera, which effectively operates as a human proctor. Examinees will be instructed to use the video to show under the table, behind the monitor, etc., before starting the exam. They then might be instructed to stand up the phone 2 meters away with a clear view of the entire room while the test is being delivered.
Using your own proctors:
Some online proctoring systems allow you to utilize your own staff as proctors, which is especially useful if the test is delivered in a small time window. If continuously delivered 24×7 all year, you probably want to use the vendor’s highly trained staff.
API integrations:
Some systems require software developers to set up an API integration with your LMS or assessment platform. Others are more flexible, and you can just log in yourself, upload a list of examinees, and you are all set.
On-Demand vs. Scheduled:
Some platforms involve the examinee scheduling a time slot. Others are purely on-demand, and the examinee can show up whenever they are ready. MonitorEDU is a prime example of this: examinees show up at any time, present their ID to a live human, and are then started on the test immediately – no downloads/installs, no system checks, no API integrations, nothing.
More security: A better test delivery system
A good testing delivery platform will also come with its own functionality to enhance test security: randomization, automated item generation, computerized adaptive testing, linear-on-the-fly testing, professional item banking, item response theory scoring, scaled scoring, psychometric analytics, equating, lockdown delivery, and more. In the context of online proctoring, perhaps the most salient is the lockdown delivery. In this case, the test will completely take over the examinee’s computer and they can’t use it for anything else until the test is done.
LMS systems rarely include any of this functionality, because they are not needed for a midterm exam of Intro to Psychology. However, most assessments in the world that have real stakes – university admissions, certifications, workforce hiring, etc. – depend heavily on such functionality. It’s not just out of habit or tradition, either. Such methods are considered essential by international standards including AERA/APA/NCMA, ITC, and NCCA.
ASC’s online proctoring Partners
ASC partners with some of the leaders in the space to give an out-of-the-box solution to our clients. These include: MonitorEDU, ProctorExam, Examity, Sumadi, Alemira, and ProctorFree. Learn more at our webpage regarding that functionality and another that explains the concept of scalable test security.
Nathan Thompson, PhD, is CEO and Co-Founder of Assessment Systems Corporation (ASC). He is a psychometrician, software developer, author, and researcher, and evangelist for AI and automation. His mission is to elevate the profession of psychometrics by using software to automate psychometric work like item review, job analysis, and Angoff studies, so we can focus on more innovative work. His core goal is to improve assessment throughout the world.
Nate was originally trained as a psychometrician, with an honors degree at Luther College with a triple major of Math/Psych/Latin, and then a PhD in Psychometrics at the University of Minnesota. He then worked multiple roles in the testing industry, including item writer, test development manager, essay test marker, consulting psychometrician, software developer, project manager, and business leader. He is also cofounder and Membership Director at the International Association for Computerized Adaptive Testing (iacat.org). He’s published 100+ papers and presentations, but his favorite remains https://scholarworks.umass.edu/pare/vol16/iss1/1/.
