Time Limits in Assessment
Content
Time limits are an essential parameter to just about every type of assessment. A time limit is the length of time given to individuals to complete their assessment or a defined portion of it. Managing test timing effectively ensures fairness, accuracy, and a pleasant experience for all test-takers – and is therefore a component of validity, which means we need to thoughtfully research and establish the time limits. In this blog post, we will explore the concept of test timing, how time limits are determined, and how accommodations are provided for those who need extra time.
Power vs Speeded vs Timed
When it comes to the role of time, there are three types of assessment. This article focuses on timed tests. More on speeded vs power tests can be found in this article.
Power: This test is untimed, meaning that the examinee has as much time as they want to show just how much they can do and how far they can go. The goal of the test is to find the maximum performance level of the examinee. For example, we might give them a math test with items that are years ahead of what they have learned, but bright students might be able to figure them out if given enough time.
Speeded: A speeded test is one where the time limit is set low enough that it affects performance, and the goal of the test is more to evaluate the velocity of the examinee. For example, we might provide you with a list of 100 simple math problems and see how many you can finish in 30 seconds. Or, provide you with a list of 100 words to proofread in 30 seconds. In these cases, getting the correct answers is still the score, but it is driven by the time limit. There are, of course, assessments where the time is the score; many of us had to run a mile when we were in school, and of course there is only one “item” on the test, the score is entirely the time taken to finish.
Timed: A timed test is one where there is a time limit, but it is intentionally designed to not impact the majority of examinees, and is therefore more for practical purposes. Most assessments fall into this category. There might be 100 items with a 2-hour time limit, and most examinees finish within 1.5 hours. The time limit exists so that the student does not sit there all day, but it has virtually no impact on their performance. In some cases, the only students who hit the time limit are the high achievers who are so intent on getting 100% that they take every minute possible to keep checking their work!
Factors Considered When Determining Time Limits
Several factors are considered when deciding on time limits for assessments. One important factor to consider is the complexity of the content being assessed. As an example, a math or science test requiring intricate problem-solving will require more time to complete than a verbal reasoning or test of general cognitive ability.
Unsurprisingly, the time load of the questions is important as well. If there are reading passages, videos, complex images like X-rays, or other assets which require a lot of time to digest before even starting the question, then this obviously needs to be taken into account.
The intended purpose of the exam also influences time limits. For high-stakes exams, such as licensing or certification tests, the goal is to ensure the candidate demonstrates a high level of knowledge, and we need very high reliability and validity for the test. This usually means that the test has more questions, and we want to provide an opportunity for the candidate to show what they know. On the other hand, a quick screening test to see if a kid has learned the most recent unit of 4th grade math has much lower stakes, so a shorter time limit does not affect the purpose, and in this case actually aligns with the goal of “quick.” Screening tests for pre-employment purposes, or medical surveys, also align with this.
Finally, test security is another consideration. Some people take tests with the goal of stealing the intellectual property. If we give them extra time to sit there and memorize the items so they can put them on illegal websites, that does no good to anyone.
Determining Time Limits for Linear Tests
Historical data and statistical modeling are often used to estimate the optimal time for test-takers. Test developers rely on empirical evidence and past performance to predict how long an average test-taker might need to finish the test, and to adjust the time limit accordingly.
You might pilot a test and determine that the questions take 1 minute on average, and if the test is 100 items, then 120 minutes (2 hours) makes a plausible time limit. More complex modeling might also be useful, such as this article discussing lognormal distributions.
Determining Time Limits for Adaptive Tests
Unlike traditional fixed-form assessments, in which every test-taker answers the same questions, Computerized Adaptive Testing (CAT) adjusts the test in real time based on how an individual performs. While some adaptive tests provide the same number of items to each examinee, the difficulty will vary widely. Moreover, some tests will vary in the number of items used. For example, the NCLEX (nursing licensure exam) will range from 85 to 150 items.
The US Armed Services Vocational Aptitude Battery is adaptive, and configured the linear timing approach to this situation. Suppose again that items take 1 minute on average. If we know that a test is an average of 100 items with a standard deviation of 10, then 98% of examinees will take less than 120 items, that then means 98% of examinees will finish under 120 minutes with no speededness. Such research questions are discussed in technical reports like this one, and also the landmark book Computerized adaptive testing: From inquiry to operation. American Psychological Association.
Time Limit Extensions: Accommodations for Test-Takers
In a diverse world in which test-takers have varying needs, it is essential that accommodations are available for those who may require additional time to complete a test. Test-takers with learning disabilities, attention disorders, or physical impairments may find standard time limits difficult to manage, and they may benefit from time limit extensions.
For example, individuals with dyslexia or ADHD might take longer to process information or may require breaks to stay focused. Providing extra time, usually in the form of a varied or fixed additional number of minutes can help level the playing field for these candidates. Examinees with visual impairments might need special software or even a human to read the questions to them, requiring extra time.
To ensure fairness, accommodations are typically offered based on documentation of the individual’s specific needs. This can include medical assessments, educational records, or reports from licensed professionals that substantiate the need for extra time. The multiplier for extra time can differ from examinee to examinee, as seen below. These accommodations are not intended to provide unfair advantages, but to ensure every test-taker may demonstrate their knowledge.
Types of Time Limits
We’ve focused this discussion on time limits for a test, but there are actually several levels of time limits that can be applied.
Item: For some tests, you might set a hard limit of, say, 30 seconds per item. This is usually only when it is relevant to the measurement, such as a working memory test.
Section: Longer tests are broken into sections, so the time limit might be more relevant at this level. You might have an hour for section 1, a 10 minute break, and then an hour for section 2. Setting an overall time limit of 2:10 then seems unnecessary.
Test: As discussed above, this is what we typically think of, with a test that has one section and a time limit overall.
Session: Some assessments have multiple tests, which is known as a battery. While the individual tests might each have time limits, there might also be a session maximum.
Functionality regarding test limits also needs to mesh with other functionality regarding test security, such as re-entry. For example, you can see that our Assess.ai platform has an option to allow an examinee to leave, but continue the clock ticking while they are out, with automatic submission when the time limit is complete.
Conclusion
Time limits are essential aspects in assessment, having a direct effect on the validity of scores as well as the experience of examinees. Determining appropriate time limits requires careful consideration of multiple factors, including the complexity of the content, the number of questions, and test security concerns. At the same time, it is essential to provide accommodations for test-takers who require additional time to ensure that individuals can perform to the best of their abilities, regardless of any physical or cognitive challenges. By including these considerations, testing organizations can ensure that their assessments are both fair and accessible, promoting a more inclusive and equal testing experience for all.
Nathan Thompson, PhD
Latest posts by Nathan Thompson, PhD (see all)
- What is a rubric? - February 13, 2025
- Time Limits in Assessment - February 8, 2025
- SIFT: A new tool for detection of test fraud - February 4, 2025
