Posts on psychometrics: The Science of Assessment

Do you conduct scholarly research on adaptive testing? Perhaps a thesis or dissertation? Or maybe you have developed adaptive tests and have a technical report or validity study? I encourage you to check out the Journal of Computerized Adaptive Testing as a publication outlet for your adaptive testing research. JCAT is the official journal of the International Association for Computerized Adaptive Testing, a nonprofit org dedicated to improving the science and technology of assessment.

JCAT has an absolutely stellar board of editors, and was founded to focus on improving the dissemination of adaptive testing research. The IACAT website also contains a comprehensive bibliography of adaptive testing research, across all journals and tech reports, for the past 50 years!

La pandemia COVID-19 está cambiando drásticamente todos los aspectos de nuestro mundo, y una de las áreas más afectadas es la evaluación educativa y otros tipos de evaluación. Muchas organizaciones aún realizaban pruebas con metodologías de hace 50 años, como colocar a 200 examinados en una sala grande con escritorios, exámenes en papel y un lápiz. COVID-19 está obligando a muchas organizaciones a dar un giro, lo que brinda la oportunidad de modernizar las evaluaciones. Pero, ¿cómo podemos mantener la seguridad en la evaluación, y por lo tanto la validez, a través de estos cambios? A continuación, presentamos algunas sugerencias, las cuales se pueden implementar fácilmente en las plataformas de evaluación de ASC, líderes en la industria. Comience registrándose para obtener una cuenta gratuita en http://assess.ai.

Verdadera banca de ítems con acceso a contenido

Una buena evaluación comienza con buenos ítems. Si bien los Sistemas de Gestión del Aprendizaje (LMS) y otras plataformas que no son realmente de evaluación incluyen algunas funciones de creación de ítems, por lo general no cumplen con los requisitos básicos para una verdadera banca de ítems. Existen prácticas recomendadas con respecto a la banca de ítems que son estándar en las organizaciones de evaluación a gran escala (p. Ej., Los Departamentos de Educación de Estado en EE. UU.), pero son sorprendentemente raras para los exámenes de certificación / licencia profesional, universidades y otras organizaciones. A continuación, se muestran algunos ejemplos.

• Los ítems son reutilizables (no es necesario cargarlos para cada prueba en la que se utilicen)

• Seguimiento de la versión del ítem

• Seguimiento y auditorías de edición hecha por usuarios

• Controles de contenido de autor (los profesores de matemáticas solo pueden ver elementos de matemáticas)

• Almacenar metadatos como parámetros de la Teoría de Respuesta al Ítem (TRI) y estadísticas clásicas

• Seguimiento del uso de ítems en las pruebas

Flujo de trabajo de revisión de ítems

Acceso basado en roles

Todos los usuarios deben estar limitados por roles, como Autor del ítem, Revisor del Ítem, Editor de Pruebas y Administrador de Examinados. Entonces, por ejemplo, es posible que alguien a cargo de administrar la lista de examinados / estudiantes nunca vea ninguna pregunta del examen.

Análisis forense de datos

Hay muchas formas de analizar los resultados de tu prueba para buscar posibles amenazas de seguridad / validez. Nuestro software SIFT proporciona una plataforma de software gratuita para ayudarte a implementar esta metodología moderna. Puedes evaluar los índices de colusión, que cuantifican qué tan similares son las respuestas para cualquier par de examinados. También puedes evaluar los tiempos de respuesta, el rendimiento del grupo y las estadísticas acumuladas.

Aleatorización

Cuando las pruebas se entregan en línea, debe tener la opción de aleatorizar el orden de los ítems y también el orden de las respuestas. Al imprimir en papel, debe haber una opción para aleatorizar el orden. Pero, por supuesto, está mucho más limitado respecto a esto cuando se usa papel.

Prueba lineal sobre la marcha (LOFT)

LOFT creará una prueba aleatoria única para cada examinado. Por ejemplo, puedes tener un grupo de 300 ítems distribuidos en 4 dominios, y cada examinado recibirá 100 ítems con 25 de cada dominio. Esto aumenta enormemente la seguridad.

Pruebas adaptativas computarizadas (CAT)

CAT lleva la personalización aún más lejos y adapta la dificultad del examen y el número de ítems que ve cada alumno, en base a ciertos algoritmos y objetivos psicométricos. Esto hace que la prueba sea extremadamente segura.

Navegador bloqueado

¿Quieres asegurarte de que el alumno no pueda navegar en busca de respuestas o tomar capturas de pantalla de ítems? Necesitas un navegador bloqueado. Las plataformas de evaluación de ASC, Assess.ai y FastTest, vienen con esto listo para usar y sin costo adicional.

Códigos de prueba para examinados

¿Quieres asegurarte de que la persona adecuada realice el examen adecuado? Genera contraseñas únicas de un solo uso para que las entregue un supervisor después de la verificación de identidad. Esto es especialmente útil en la supervisión remota; el estudiante nunca recibe ninguna información  antes del examen sobre cómo ingresar, excepto para iniciar la sesión de supervisión virtual. Una vez que el supervisor verifica la identidad del examinado le proporciona la contraseña única de un solo uso.

Códigos de supervisor

¿Quieres un paso adicional en el procedimiento de inicio de la prueba? Una vez que se verifica la identidad de un estudiante e ingresa su código, el supervisor también debe ingresar una contraseña diferente que sea exclusiva para él ese día.

Ventanas de fecha / hora

¿Quieres evitar que los examinados ingresen temprano o tarde? Configura una ventana de tiempo específica, como el viernes de 9 a 12 am.

Supervisión basada en IA (Inteligencia Artificial)

Este nivel de supervisión es relativamente económico, y hace un gran trabajo validando los resultados de un examinado individual. Sin embargo, no protege la propiedad intelectual de las preguntas de tu examen. Si un examinado roba todas las preguntas, no lo sabrás de inmediato. Por lo tanto, es muy útil para exámenes de nivel bajo o medio, pero no tan útil para exámenes de alto riesgo como certificaciones o licenciaturas. Obtenga más información sobre nuestras opciones de supervisión remota. También te recomiendo esta publicación de blog para obtener una descripción general de la industria de supervisión remota.

Supervisión en línea en tiempo real

Si no puedes asistir a los centros de pruebas en persona debido a COVID, esta es la siguiente mejor opción. Los supervisores en vivo pueden registrar al candidato, verificar la identidad e implementar todas las demás cosas anteriores. Además, pueden verificar el entorno del examinado y detener el examen si ven que el examinado roba preguntas u otros problemas importantes. MonitorEDU es un gran ejemplo de esto.

¿Cómo puedo empezar?

¿Necesitas ayuda para implementar algunas de estas medidas? ¿O simplemente quieres hablar sobre las posibilidades? Envía un correo electrónico a ASC a solutions@assess.com.

Traducido de la entrada de blog escrita por el Dr. Nathan Thompson.

Nathan Thompson obtuvo su doctorado en psicometría de la Universidad de Minnesota, con un enfoque en pruebas adaptativas computarizadas. Su licenciatura fue de Luther College con una triple especialización en Matemáticas, Psicología y Latín. Está interesado principalmente en el uso de la IA y la automatización de software para aumentar y reemplazar el trabajo realizado por psicometristas, lo que le ha proporcionado una amplia experiencia en el diseño y programación de software. El Dr. Thompson ha publicado más de 100 artículos de revistas y presentaciones de conferencias, pero su favorito sigue siendo https://pareonline.net/getvn.asp?v=16&n=1.

La vigilancia en línea existe desde hace más de una década. Pero dado el reciente brote de COVID-19, las instituciones educativas y de fuerza laboral / certificación están luchando por cambiar sus operaciones, y una gran parte de esto es un aumento increíble en la vigilancia en línea. Esta publicación de blog está destinada a proporcionar una descripción general de la industria de vigilancia en línea para alguien que es nuevo en el tema o está comenzando a comprar y está abrumado por todas las opciones que existen.

Vigilancia en Línea: Dos Mercados Distintos

En primer lugar, describiría la industria de vigilancia en línea como perteneciente a dos mercados distintos, por lo que el primer paso es determinar cuál de ellos se adapta a tu organización.

1. Sistemas a mayor escala, de menor costo (cuando son a gran escala) y con menos seguridad, diseñados para ser utilizados solo como un complemento para las principales plataformas LMS como Blackboard o Canvas. Por lo tanto, estos sistemas de vigilancia en línea están diseñados para exámenes de nivel medio, como un examen de mitad de período de Introducción a la psicología en una universidad.

2. Sistemas de menor escala, mayor costo y mayor seguridad diseñados para ser utilizados con plataformas de evaluación independientes. Estos son generalmente para exámenes de mayor importancia como certificación o fuerza laboral, o quizás para uso especial en universidades como exámenes de Admisión y Colocación.

¿Cómo reconocer la diferencia? El primer tipo anunciará la fácil integración con sistemas como Blackboard o Canvas como característica clave. También se centrarán a menudo en la revisión de videos por IA, en lugar de usar humanos en tiempo real. Otra consideración clave es observar la base de clientes existente, que usualmente es anunciada.

Otras formas en que los sistemas de vigilancia en línea pueden diferir

IA vs humanos: Algunos sistemas se basan exclusivamente en algoritmos de inteligencia artificial para marcar las grabaciones de video de los examinados. Otros sistemas utilizan humanos reales.

Grabar y Revisar vs Humanos en Tiempo Real: Existen dos formas si se utilizan humanos. Primero, puede ser en vivo y en tiempo real, lo que significa que hay un ser humano en el otro extremo del video que puede confirmar la identidad antes de permitir que comience la prueba, y detener la prueba si hay actividad ilícita. Grabar y Revisar grabará el audio y un humano lo comprobará en un plazo de 24 a 48 horas. Esto es más flexible, pero no puedes detener la prueba si alguien está robando el contenido; probablemente no lo sabrás hasta el día siguiente.

Captura de pantalla: Algunos proveedores de vigilancia en línea tienen la opción de grabar / transmitir la pantalla y también la cámara web. Algunos también brindan la opción de hacer únicamente esto (sin cámara web) para exámenes de menor importancia.

Teléfono móvil como tercera cámara: Algunas plataformas más nuevas ofrecen la opción de integrar fácilmente el teléfono móvil del examinado como una tercera cámara, que funciona efectivamente como un supervisor humano. Se les indicará a los examinados que utilicen el video para mostrar debajo de la mesa, detrás del monitor, etc., antes de comenzar el examen. Luego, se les puede indicar que coloquen el teléfono a 2 metros de distancia con una vista clara de toda la habitación mientras se realiza la prueba.

Uso de supervisores propios: Algunos sistemas de vigilancia en línea le permiten utilizar su propio personal como supervisores, lo que es especialmente útil si la prueba se realiza en un período de tiempo reducido. Si se entrega continuamente 24 × 7 durante todo el año, probablemente desee utilizar el personal altamente capacitado del proveedor.

Integraciones de API: Algunos sistemas requieren que los desarrolladores de software configuren una integración de API con su LMS o plataforma de evaluación. Otros son más flexibles y puedes iniciar sesión por ti mismo, cargar una lista de examinados y ya queda todo listo para la prueba.

Bajo pedido vs Programado: Algunas plataformas requieren que se programe un margen de tiempo para que los examinados realicen la prueba. Otros son puramente bajo demanda y el examinado puede presentarse cuando esté listo. MonitorEDU es un excelente ejemplo de esto: los examinados se presentan en cualquier momento, presentan su identificación a un humano en tiempo real y luego comienzan la prueba de inmediato: sin descargas / instalaciones, sin verificaciones del sistema, sin integraciones de API, nada.

Más seguridad: Un Mejor Sistema de Entrega de Pruebas

Una buena plataforma de entrega de pruebas también vendrá con su propia funcionalidad para mejorar la seguridad de las pruebas: aleatorización, generación automatizada de ítems, pruebas adaptativas computarizadas, pruebas lineales sobre la marcha, banca profesional de ítems, puntuación de la teoría de respuesta a los ítems, puntuación escalada, análisis psicométrico, equiparación, entrega de bloqueo y más. En el contexto de la vigilancia en línea, quizás lo más destacado sea la entrega de bloqueo. En este caso, la prueba se hará cargo por completo de la computadora del examinado y no podrá usarla para nada más hasta que termine la prueba.

Los sistemas LMS rara vez incluyen esta funcionalidad, porque no son necesarios para un examen de mitad de período de Introducción a la psicología. Sin embargo, hay muchas cosas en juego en la mayoría de las evaluaciones del mundo (admisiones universitarias, certificaciones, contratación de personal, etc.) y estas pruebas dependen en gran medida de dicha funcionalidad. Tampoco es solo una costumbre o una tradición. Dichos métodos se consideran esenciales según los estándares internacionales, incluidos AERA / APA / NCMA, ITC y NCCA.

Socios de ASC de Vigilancia en Línea

ASC les brinda a sus clientes una solución lista para ser usada, debido a que está asociado con algunos de los líderes en el ámbito. Estos incluyen: MonitorEDU, ProctorExam, Examity y Proctor360. Obtén más información en nuestra página web sobre esa funcionalidad y otra que explica el concepto de seguridad de prueba configurable.

Traducido de la entrada de blog escrita por el Dr. Nathan Thompson.

Nathan Thompson obtuvo su doctorado en psicometría de la Universidad de Minnesota, con un enfoque en pruebas adaptativas computarizadas. Su licenciatura fue de Luther College con una triple especialización en Matemáticas, Psicología y Latín. Está interesado principalmente en el uso de la IA y la automatización de software para aumentar y reemplazar el trabajo realizado por psicometristas, lo que le ha proporcionado una amplia experiencia en el diseño y programación de software. El Dr. Thompson ha publicado más de 100 artículos de revistas y presentaciones de conferencias, pero su favorito sigue siendo https://pareonline.net/getvn.asp?v=16&n=1.

The Test Information Function is a concept from item response theory (IRT) that is designed to evaluate how well an assessment differentiates examinees, and at what ranges of ability. For example, we might expect an exam composed of difficult items to do a great job in differentiating top examinees, but it is worthless for the lower half of examinees because they will be so confused and lost. The reverse is true of an easy test; it doesn’t do any good for top examinees. The test information function quantifies this, and has a number of other important applications and interpretations.

Test Information Function: How To Calculate It

The test information function is not something you can calculate by hand. First, you need to estimate item-level IRT parameters, which define the item response function. The only way to do this is with specialized software; there are a few options in the market, but we recommend Xcalibre. Next, the item response function is converted to an item information function for each item. The item information functions can then be summed into a test information function. Lastly, the test information function is often inverted into the conditional standard error of measurement function, which is extremely useful in test design and evaluation.

IRT Item Parameters

Software like Xcalibre will estimate a set of item parameters. What parameter you use depend on the item types and other aspects of your assessment. For example, let’s just use the 3-parameter model, which estimates a, b, and c. And we’ll use a small test of 5 items. These are ordered by difficulty: Item 1 is very easy and Item 5 is very hard.

Itemabc
11.00-2.000.20
20.70-1.000.40
30.400.000.30
40.801.000.00
51.202.000.25

Item Response Function

The item response function uses the IRT equation to convert the parameters into a curve. The purpose of the item parameters is to fit this curve for each item, like a regression model, to describe how it performs. Here are the response functions for those 5 items. Note the scale on the x-axis, similar to the bell curve, with the easy items to the left and hard ones to the right.

item response function

Item Information Function

The item information function evaluates the calculus derivative of the item response function. An item provides more information about examinees where it provides more slope. For example, consider Item 5: it is difficult, so it is not very useful for examinees in the bottom half of ability. The slope of the Item 5 IRF is then nearly 0 for that entire range. This then means that its information function is nearly 0.

item information functions

Test Information Function

The test information function then sums up the item information functions, to summarize where the test is providing information. If you imagine adding the graphs above, you can easily imagine some humps near the top and bottom of the range, where there are the prominent IIFs. This indeed happens.

Test information function

Conditional Standard Error of Measurement Function

The test information function can be inverted into an estimate of conditional standard error of measurement. What do we mean by conditional? Well, if you are familiar with classical test theory, you know that it estimates the same standard error of measurement for everyone that takes a test. But given the reasonable concepts above, it is incredibly unreasonable to expect this. If a test has only difficult items, then it measures top students well, and does not measure lower students well, so why should we say that their scores are just as accurate? The conditional standard error of measurement turns this into a function of ability. Also, note that it refers to the theta scale and not to the number-correct scale.

Conditional standard error of measurement function

How can I implement all this?

For starters, I recommend delving deeper into an item response theory book. My favorite is Item Response Theory for Psychologists by Embretson and Riese. Next, you need some item response theory software. Xcalibre can be downloaded as a free version for the purposes of learning, and is the easiest program to learn how to use (no 1980s-style command code… how is that still a thing?). But, if you are an R fan, there are plenty of resources in that community as well.

Tell me again: Why are we doing this?

The purpose of all this is to effectively model how items and tests work, namely how they interact with examinees. This then allows us to evaluate their performance so that we can improve them, thereby enhancing reliability and validity. Classical test theory had a number of shortcomings in this endeavor, which led to IRT being invented. IRT also facilitates some modern approaches to assessment, such as linear on the fly testing, adaptive testing, and multistage testing.

Professional certification programs that allow participants to validate their knowledge and skills abound in the U.S. and around the world. Examples range from long-standing, well-recognized teacher certification and CPR programs to more niche and cutting-edge offerings, such as the Project Management Professional and Amazon Web Services credentials. Candidates are often pursuing some combination of new career, promotion, higher salary, and self-fulfillment. They take varying risks with their time and finances for what could return great reward. Given the high stakes involved, an extensive effort goes into certification program management – that is, ensuring that the certifications are developed and run according to best practices. This includes psychometrics, but is most definitely not limited to that topic.

There are many aspects that go into certification program management, including legal status, board governance, accounting, test development, staffing and org charts, continuing education, recertification, prerequisites, and policies for candidates.

One important consideration in certification program management is the requirement of firewall between staff involved in Certification and those involved in Education. Basically, you don’t want the people who are teaching courses to have seen the items on the test – especially if there is an incentive for them to help the students pass! For example, if instructors are tracked by pass rate for their students or institution, they have a reason to want more people to pass, and could divulge more info about the exam than they should. If they never know such information, they instead concentrate on teaching.

Other stakes in certification program management include the protection of the public. This includes patients, students, customers, employees, employers, and all others affected by the performance of certified individuals. The program itself also wagers its reputation each time it confers a certification.

In this high-risk environment, savvy program managers are concerned with granting certification only to those likely to practice competently. They optimize their tests using tools from the science of psychometrics to ensure that candidates must demonstrate appropriate knowledge and skills in order to pass. Learn more about the process of test development in this blog post.

We get asked all the time at Assessment Systems Corporation: What are the steps involved in certification test development? What are the best practices that lead to a sound exam that can meet accreditation standards? There are a number of key steps:

Job Analysis

A best practice among certification programs is to understand the knowledge, skills, and attributes used in the performance of the job or role. A job task analysis, or role delineation, uses quantitative methods to gain insight into job competence. Subject matter expert (SME) involvement is critical to validity. The value of this analysis ranges from helping practitioners identify areas for development to benchmarking content for educational and training programs, field leadership for the certification organization, and promotion and elevation of the field.

Test Blueprint

Developing a blueprint, or content outline, for the test from job analysis results lends validity to test results. A best practice is to use the job analysis to derive which content areas should be on the test and how many items should be in each. Beyond test validity, the benefits of publishing the blueprint include transparency for all in the certification process and a guide for candidates to focus their test preparation.

Item Approval

A foundational principle of test development says that the effectiveness of the test is only as good as the items on it. Savvy certification programs have quality standards for their items covering type, content, format, and grammar. They ensure that each item meets these before inclusion on a test. Once items have been sampled, program managers consider whether their performance warrants use in future testing.

Standard Setting

The passing standard, or cut score, is usually where the certification program’s test intersects with their decision whether to confer certification. Test takers whose scores meet or exceed the passing standard receive a pass result, and those whose performance fails to meet the standard receive a fail. The stakes mentioned above rest on where and in what way this this standard is set. Common methods for setting the passing standard involve SME input and previous item performance. Publishing the passing standard and method used to derive it is valuable to candidates for understanding how they performed on the test and to certifying organizations for deciding where to grant certification.

Psychometric Reporting

Validity and reliability of the test are key concepts in psychological testing. Validity speaks to how accurately the test measures real-world job or role performance. Savvy programs document evidence of this in the points already discussed. Once candidates have answered items, analysis of test and item performance begins.

Traditionally, programs look at statistics such as reliability (especially at the passing standard), distribution of scores, difficulty, and discrimination of the test and each item for a particular group of test takers. Certifying organizations use this analysis to finalize test results and report on the test’s performance. Leading organizations with high candidate volumes are doing additional analysis of results and developing innovative ways to compile and administer tests, such as Linear on-the-fly (LOFT) testing and computer adaptive testing (CAT).

The Certification Test Development Cycle Continues…

Certification test development is not a one-and-done situation. If you leave one form of the test out in the field, the questions and answers will soon become common knowledge. It’s essential to periodically update the exam by developing new items and then rotating them in by publishing new forms. And every 5-10 years (depending on the profession) the test needs a complete overhaul, with a new job analysis and standard setting. As with any high performing machine, you need oil changes and periodic maintenance!

A question bank refers to a pool of test questions to be used on various assessments across time. For example, a Certified Widgetmaker Exam might have a pool of 500 questions that have been developed over the past 10 years. Suppose the exam is delivered in June and December of every year, and each time 150 questions are used. This strong pool of items allows the organization to easily select questions and publish a new form of the exam each time.

A question bank is more commonly called an item bank by those in the assessment industry. It is due to the fact that the term question is not often used, because many assessment items are not actually questions; they might be statements, vignettes, simulations, or many things other than the traditional question-and-4-answers.

However, a question bank is actually much more than the questions themselves. If you ran the Certified Widgetmaker Exam, you would want to keep track of some additional important information. This is all based on the concept of treating the question as a reusable object; if you use the item 4 times, you should never need to type/upload it 4 times. It should be in the system only once, with all its associated metadata!

What to track Examples
Which exam forms used each question Dec 2017, May 2018, May 2019, Dec 2020
Unique item ID Math.Algebra.078
Source/Reference Wilson (2016) p. 123
Status New, Under Review, Active, Retired
Statistics Classical difficulty and discrimination: Item response theory parameters
Reviewer comments Jake Smith 2020/11/22: “I think that D is arguably correct, and we need to provide greater detail in the stem.”
Content area, domain, blueprint Math / Algebra / Quadratic

As you can see, there’s actually quite a bit of functionality that goes into a true question bank system. And this is only regarding the questions themselves – it doesn’t get into additional topics such as Workflow Management, Automated Item Generation, or Test Assembly & Publishing. A real system for managing question bank will have much, much more than just a way to store the questions.

Looking for a deeper treatment of the topic? Check out the chapter Computerized Item Banking by ASC’s cofounder, C. David Vale, in the 2006 Handbook of Test Development.

Want to learn more about how a question banking system can help your organization? Click here, check out this other post, or fill out our contact form for a demonstration.

One misconception that I often see on the internet is the distinction, or lack thereof between the words psychometrician and psychometrist. While both work in the field of assessment, they are actually quite unrelated. This post describes how the two are different.

The most flagrant offender, curiously, is Google. Like most companies, we utilize AdWords. Google often treats the words psychometrician and psychometrist as interchangeable, even though the two are nowhere near each other. It also does the same with psychometrics and psychometric testing which are similarly quite distant, but that’s a blog post for another day.

Psychometrician

A psychometrician is someone involved with the engineering of assessments. As the subtitle to the book Modern Psychometrics notes, it is the science of psychological assessment, though I take major issue with the “psychological” word because the majority of assessment in the world happens in other areas, such as education, employment, and professional certification/licensure. It’s definitely not limited to psychology, though that is the historical root of the field.

Anyway, the key word here is science, which implies that a psychometrician is a scientist. Most people that work as psychometricians have indeed earned a PhD by doing extensive scientific research. And it’s not qualitative or simplistic, either; psychometricians delve into the fundamental process of assessment itself, researching things like data analytics and machine learning models that make assessments more efficient and accurate. The most common is item response theory. In the working world, they are typically involved in the building and validation of assessments, such as a large scale university admissions or professional certification test.

Consider the following definition for the US Bureau of Labor Statistics:

(A psychometrician) designs, scores, and analyzes data from exams. Psychometricians’ work ensures that each test is reliable and that all test results are valid.

Source: https://www.bls.gov/careeroutlook/2011/fall/yawhat.pdf

Psychometrist

A psychometrist is someone involved in the use and administration of assessments, and in most cases is working in the field of psychological testing. This is someone who uses tests every single day, and is familiar with how to administer such tests (especially complex ones like IQ) and interpret their results to provide feedback to individuals. They are also highly trained. Some have doctoral degrees as a clinical/counseling psychologist and have extensive expertise in that role; for example, use of an Autism-spectrum screening test to effectively diagnose patients and develop individualized plans.

Consider the following definition from the National Association of Psychometrists:

A psychometrist is responsible for the administration and scoring of psychological and neuropsychological tests under the supervision of a clinical psychologist or clinical neuropsychologist. 

Source: https://www.napnet.org/what

Comparison of Psychometrician and Psychometrist

Aspect Psychometrician Psychometrist
How are they involved with assessment? Engineering & validation Administration & interpretation
Education PhD in Psychometrics, Psychology, or Education Bachelor’s/Master’s in Psychology (often Counseling)
Quantitative skills Complex analyses like item response theory or factor analysis; complex designs such as adaptive testing Interpreting scores with summary statistics (mean, standard deviation, z-scores, correlations)
Soft skills Often a pure data analyst, but some work with expert panels for topics like job analysis or Angoff studies Works extensively with people, often in a counseling role, and can be highly trained on those aspects
Example Researcher involved in designing high-stakes exams such as medical certification or university admissions School or counseling psychologist that delivers IQ and other assessments

To draw a metaphorical comparison, if we were talking about buildings, the psychometrician would be the architect and the psychometrist would be the maintenance person that can manage/fix all the complex heating, electrical, and other issues. Or, if this was an airplane, the psychometrician would be the engineer that designed the plane and the psychometrist would be the pilot (or again, maintenance leader). Obviously both roles are important to effective functioning of the airplane, but the roles are quite different – even though the terms are almost the same in our case!

Want to learn more about how psychometricians can improve your organization’s assessments? Read up on what is involved with psychometric consulting.

The Association of Test Publishers is holding their first ever EdTech and Computational Psychometrics Summit, December 3-4 2020. Due to the coronavirus situation, this of course will be fully virtual. (The 2021 primary ATP conference will be as well, which is a shame!)

ASC has been invited to speak on a panel regarding artificial intelligence (AI) in education and psychometrics. The panelists will review trends and buzzwords, and help listeners make sense of where AI is actually helping and what is still fluff, as well as where the technology is headed in the next few years.

For more information: http://www.globalatpevents.com/ecps/

The American Chiropractic Board of Sports Physicians™ (ACBSP™: acbsp.com) has partnered with Assessment Systems Corporation (ASC: assess.com) to advance the written testing procedure to a digitalized assessment model with secure remote proctoring for the Certified Chiropractic Sports Physician® (CCSP®) and Diplomate American Chiropractic Board of Sports Physicians® (DACBSP®) certifications.  While this will provide the immediate benefit that ACBSP certification examinations will continue during the COVID-19 restrictions, future exams will also be held in this format. 

The certifications have been traditionally delivered via paper-and-pencil at specific locations; ASC will work with the ACBSP™ to digitalize the assessment and then deliver them with secure remote proctoring.  The remote proctoring provided by MonitorEDU will utilize two video streams for monitoring, one for the room and one for the candidate, as well as a lockdown browser to maintain security on the computer, providing support to the validity and integrity of the assessment.

Dr. Anne Sorrentino, the President of ACBSP™, states: “ACBSP™ is excited to announce the upcoming November 7, 2020 written exams will be remotely proctored computer-based testing (RP-CBT). This means travel to an exam site is not necessary. The exam may now be taken in the privacy of one’s own home! COVID temporarily closed down the opportunity for the advancement of the CCSP® and DACBSP® testing process this past April. The ACBSP™ moved to RP-CBT to enable everyone the opportunity to continue the certification process.”

Dr. Nathan Thompson, CPO & Vice President of ASC, notes: “ASC has helped many high-stakes assessment programs pivot during 2020, and we are enthusiastic about working with the ACBSP certification team to modernize the assessment process.  This will not only enable continuity during COVID but increase accessibility of the credentials by making them more widely available, with shorter turnaround times.”

For inquiries, contact Dr. Nathan Thompson at solutions@assess.com.

About ACBSP

Since 1980, the American Chiropractic Board of Sports Physicians™ (ACBSP™: acbsp.com) has led the development of sports medicine certification and has managed a world-class credentialing process that ensures certified sports chiropractors meet competency standards to effectively work with and treat athletes and those engaged in athletic activities. In addition, the ACBSP™ offers continuing education and research seminars to facilitate the dissemination of the latest scientific knowledge, treatment trends, and best practices for patient care.

The ACBSP™ is the governing board for the Certified Chiropractic Sports Physician® (CCSP®) and Diplomate American Chiropractic Board of Sports Physicians® (DACBSP®) certifications. Various accredited chiropractic colleges offer the curricula and training leading to qualification for taking the certification exams. The ACBSP™ governs the administration of the examinations and certifications. These certifications are designed for chiropractic doctors who want to specialize in chiropractic sports medicine.

About ASC

Assessment Systems Corporation (ASC: assess.com) has been a leader in the assessment industry since 1979, providing both world-class software for test development, secure delivery, and psychometric analytics, as well as extensive consulting services for testing organizations. ASC’s co-founder, Dr. David Weiss, is considered the father of computerized adaptive testing (CAT), and ASC was the first company to offer item banking and adaptive testing software to the public, in the early 1980s. ASC focuses on organizations that require high quality and high security from their exams, across industries such as certification, licensure, higher education, K-12 education, and pre-employment testing.  ASC drives innovation internationally, partnering with Ministries of Education in countries as diverse as Iceland, United Arab Emirates, Singapore, and Botswana. Follow us on LinkedIn, or sign up for a free account in our test development platform Assess.ai.