The State of Connecticut has enacted a new policy to require all 11th grade students to take the SAT (New York Times, 6 Aug 2015). As a lifelong advocate of quality assessment, this concerns me. I consider Instruction and Assessment to be the two primary components of Education; instruction without assessment is like a weight loss program without a scale or mirror. But the assessment has to be done right. How do we know what is right? To evaluate this, we need to couch the discussion in the most important concept in assessment: validity.
Validity refers to whether the score interpretations being made form a test are those for which the test was intended and designed, and are supported by evidence. One of my graduate school mentors used to say, “The right tool for the right job.” You could pound in a nail with a screwdriver, but that is not what it was designed for. The same goes for assessment. An actual need has to be identified that requires assessment to be done, and the assessment should be designed for that need. So let’s look at the situation in Connecticut.
What does Connecticut Need?
Connecticut needs a measure of student success, so that they can evaluate whether a student is learning the curriculum. A curriculum is designated by the State, and is currently moving towards the Common Core, which is a big improvement over the old days where each State did their own… leading States with poor education systems to simply dumb down their curriculum and lower their standards to make it appear that their students were smart. A test of what a student has learned is achievement.
For what was the SAT designed?
The Scholastic Aptitude Test (SAT) is, as the name suggests, a test of aptitude, not achievement. The story behind the SAT: A century ago, colleges and universities had their own entrance/placement exams. They realized this was stupid and banded together to form the (creatively named) College Board, to make a common exam, as well as other things. The test was designed to best predict success in college – nothing about how much the student learned in high school. Here’s an article that discusses that purpose. The approach they used was to assess cognitive ability via a few areas, currently Critical Reading, Math, and Writing (scores here). This isn’t a bad idea, given the purpose of the test. But the SAT definitely does not assess, for example, how much of the Science curriculum the student has learned in high school
Is there too much testing?
Connecticut obviously felt pressure from parents and teachers that there was too much testing for 11th graders; this article in a local paper shows the sentiment. I don’t want to dispute that. What concerns me is that Connecticut seemed to have a choice of eliminating their achievement tests or the SAT. They need an achievement test to track student learning. Instead, they dropped the achievement test built for that purpose and kept the SAT, which is designed for a completely different purpose. What irks me even more is that people who should know better, such as the Superintendent in that article, are apparently not even aware of the basic concepts of assessment. Instead, he is happy to have to go through the work of trying to set new cutscores on a test that is being misapplied.
So how did this happen?
I think the biggest reason for this, and many other misuses of assessment, is that the vast majority of people, including educators and politicians, are psychometrically illiterate. Educators are incredibly hardworking, but their focus is instruction (and rightly so). They have never had exposure to basic concepts like validity and reliability. As a scientific field as well as a business industry, we need to better educate end users and stakeholders. This is not limited to K12 Education, by the way.
As for Connecticut, I’d suggest one thing to start: follow the money. That local article says that the SAT is free for all CT students. We all know there is no such thing as a free lunch. Someone made a very good sale!
Want to improve the quality of your assessments?
Sign up for our newsletter and hear about our free tools, product updates, and blog posts first! Don’t worry, we would never sell your email address, and we promise not to spam you with too many emails.
Nathan Thompson, PhD
Latest posts by Nathan Thompson, PhD (see all)
- Adaptive Testing Research: JCAT - February 9, 2021
- Test Information Function from Item Response Theory - February 2, 2021
- What is a question bank? - December 9, 2020