"Timed Tests Are Biased Against Your Kids"

Above should be a free link to NYT article, arguing that timed tests are not an accurate test of ability or knowledge, but only of ability to work accurately under severe time pressure. It reports that the new digital version of the SAT will rely much less on time pressure.

If this is truly the case, it would be, finally, a solution to the advantage conferred by extended time on the SAT. If some students are given the significant advantage of extended time, without the colleges being able to see that the test was taken under altered circumstances, then clearly the solution is to take time pressure out of the design of the exam. Here’s hoping that the ACT will do this, too.

4 Likes

My S24 is really good at timed test, and yet I totally believe that is a largely useless life skill outside of taking timed tests. So I very much hope this becomes the norm.

4 Likes

I am an advocate for either giving everyone extra time and let kids leave when their done so only those that truly need it will use it or create them untimed.

4 Likes

Students will still be able to qualify for extended time on digital SAT and ACT (as well as other accommodations), so nothing there changes.

Digital SAT test will be 2:14 minutes vs 3:00 hours paper test. Per question time on digital SAT for Math is 95 seconds (vs. 83 seconds paper SAT and 60 seconds ACT). Per question time of EBRW on digital SAT is 71 seconds (vs. 62 seconds paper SAT, ACT Eng 36 seconds and ACT Reading 53 seconds).

There are a few reasons that the digital SAT can offer a shorter test including:

  • it’s adaptive (two modules in each of Math and EBRW. Student completes module 1 which is a mix of easy/medium/difficult questions. Based on their performance in Module 1, module 2 will consist of an ‘easier’ module or ‘harder’ module. The scores of students who receive the easier Module 2 will in effect have their ultimate score capped.)
  • questions are worth differing amounts of credit (harder questions worth more)
  • less reading in math questions
  • significantly shorter EBRW passages with only one question per passage

AFAIK, ACT has plans to offer a digital ACT, but it will be the same as the paper test, just in digital form (so same length, not adaptive).

I expect many HS students are going to choose the SAT test because it is shorter overall.

Students with extra time accommodations may still lean ACT depending on how much extra time they get, although each student will have to make the best determination for them.

I am not sure anyone has yet evaluated whether the SAT digital test will still be at a higher grade level in reading than the ACT, or how the reduced passage complexity of the digital SAT compares to ACT.

I think that we should continue moving towards computer adaptive testing for these high stake tests. That eliminates the worry about timing and gaming accommodations (to be clear I’m talking about schools like New Trier, a wealthy district with 4 times the number of average accommodations, not students who truly have a need).

3 Likes

This does not necessarily mean that the students don’t have a need. It definitely reflects better access and ability to pay for the testing that enable a student to receive accommodations.

My point is because so many students (probably most) don’t have access to adequate neuropsych testing, we don’t really know what the ‘average’ prevalence is.

Also to amplify my point from above, students can still qualify in all the same ways for extra time/other accommodations on the digital SAT.

5 Likes

The SATs were re-normed (re-scaled because students were achieving lower scores, on average) in 1995, 2005, and 2016. Back in the late '70s, individual scores in the low 700s were 99th percentile. A student who had scored in the mid 1400s on the SAT had likely scored high enough on the PSAT to make National Merit in a highly competitive state; those mid 700s scores on individual subjects before re-norming were considered equivalent to 800s today.

Now, with the SATs having been re-normed three times because the overall scores were dropping, and yet with some highly motivated, bright, prepped students, it is not that unusual to score in the 1500s, even to achieve a perfect 800 on math or English, or both. So the only thing left to differentiate among the high scoring test-takers, to string them out along a continuum, is time pressure, which measures how fast a student can do the test correctly, not necessarily a useful measure of ability or of mastery of the material.

When tests administered under altered circumstances were flagged, colleges could take that into account, but once this was banned, there was absolutely no reason for students and their families to not seek out the significant advantage conferred by having “extra” (meaning enough) time to do the test at a more human, relaxed pace, and demonstrate what they knew, as opposed to whether or not they could do it at breakneck speed. And so began the arms race of extra time, leaving even farther behind those without advocates. Ever since the banning of flagging scores from altered testing conditions, it has been obvious that the only way to make the tests valid again was to redesign them to remove time pressure as a consideration, but of course, since it would cost money to redesign the tests, the College Board and the ACT didn’t do it, until now, at least for the SAT. The big change is that if everyone is given a reasonable amount of time to do the test, thus removing time pressure from the equation, then extra time wouldn’t make much difference (except to that individual kid who really does have a disability that causes them to have to work more slowly, but who can still demonstrate their mastery of the material).

It is very interesting to hear that students will be given different tests based upon their performance on the first module in the subject. I can see how this would be worthwhile for students on the extreme ends of the testing spectrum, but what about those in the middle? It doesn’t seem fair to me that a kid who barely misses the cutoff for the more difficult 2nd module should miss the chance at a higher score, because they’re not offered the more difficult 2nd module. I would not be surprised if there are challenges to this. But maybe not, since all of this seems to be too little, too late for the SAT (and the ACT, even if they were to make the same changes, would still be in the same boat). The reality is that these tests have become almost irrelevant, what with nearly every school in the nation having gone test-optional, and all of the public CA schools having gone test-blind. I hear of many, many more students who just don’t bother to take either test, because they don’t see it as being worth it (and although they won’t openly say it, because they don’t expect to do well on it). On the other hand, nearly half the states now require students to take the SAT or the ACT as a condition of receiving a high school diploma, so that’s a pretty sure revenue stream for the College Board and the ACT.

I have no expertise at all in the field of education, learning, assessment,
 so as a lay person I have often wondered what the purpose of a timed test is, for most subject areas.

If a student has the knowledge and skill to produce the correct answer, specially in STEM, why does it matter how long it takes to produce the result?

There may be a point in limiting the amount of words in an essay, to test the skill to produce a meaningful essence - but why test how quickly one can translate concepts into words and how fast fingers produce the words on a keyboard?

If one’s personal “speed” is really an important factor / selection criteria, then this should be a separate test category - unrelated to testing academic skill/knowledge.

1 Like

Aren’t timed tests somewhat useful in measuring preparation for some real-world occupations – ones in which workers face hard deadlines, where they are under pressure to be efficient, etc.?

I’ve posted this in other discussions before, but there is a lot of science to support the conclusion your instincts are on track:

Not really. In another discussion we went into this in some detail, but there were famous cases like the US Navy trying out using timed tests for prospective naval aviators to see if they could predict things like crashes during training. Their internal data ended up showing that no, that didn’t really work. Instead, the Navy had to develop very specific tests of things like situational awareness, the ability to translate 2D displays into 3D mental models, and so on, in order to meaningfully predict piloting ability.

In general, if you look at this from a management perspective, it is generally a kind of mismanagement if people are regularly being given deadlines where they don’t have enough time to do their best work. Of course every once in a while unexpected emergency situations may occur where good enough work in a timely manner is preferable to better work but too late. But if that is your normal way of doing things, you are likely not going to be very competitive with organizations that resource and manage their employees to mostly get their best work. Including by giving them enough time to do that.

3 Likes

If only boss and underling could agree on the amount of time necessary to complete a task.

“I asked for that report an hour ago, Murphy.”
“But, sir, you asked me to get the donuts and coffee
”
“Well
 you still had plenty of time.”
“
and then you asked me to train Martha for 30 minutes
”
“Never mind.”

1 Like

In which case that should be a separate measure.

But for the purpose of college admission - we are attempting to assess academic mastery/readiness - and instead, the test result is a hodge-podge of personal coping with time-stress, motor skills, speed and only from among those who excel at that, we learn something about their relative academics?

Is having to “ponder” an answer to obtain the most considered, correct response really a negative? In my professional experience, I have put out too many fires for exactly the lack of sufficient thought putting behind implemented “solutions”.

Is our current testing advancing “fly-over” solvers, who then end up in “real-world occupations”.

3 Likes

There are professions where the correct answer must be found under time pressure, such as emergency room physician / nurse / etc
 However, this is not the case for all professions that a high school student may eventually want to enter.

1 Like


 seemingly correct ? :wink:

I agree that the need/scenario exists, and there is no alternative to quick decisions – I’m less convinced that the “correct” answer can reliably be found, just the “best available”.

Maybe what should be aptitude-tested for entering those professions is coping with time stress, and how good & quickly one can “interpolate” to come up with “good enough” answers - which is not what the current standardized “academic” testing does (which also don’t target any particular profession!)

Isn’t the reason for timed testing scheduling convenience?

The test needs to schedule a location. The test needs to schedule proctors. Test takers may not want a longer test cutting into whatever else they want to do that day.

But the test writer wants to add as much content within the allocated time as possible in order to make the most “accurate” assessment to the extent that the test can proxy the desired characteristic.

1 Like

But as explained at length in the meta-review article I linked, that thought is a mistake. If you try to pack too much content into too little time, more and more you test for pace of work and not actual knowledge or ability.

So a good test writer will be very careful to make sure even people with slower paces of work will have enough time to do their best work. Which means a lot of other people will finish early, and that is fine.

And if you want the test to provide more discrimination, meaning make it easier to distinguish better and worse performance, you can make the test harder substantively.

So part of the historic problem with the SAT/ACT was they were limited in how hard they could be given how they were supposed to be doable by all of the wide variety of college intenders. And when a test with a low common denominator in terms of substance doesn’t discriminate enough among some of the test-takers, you can ramp up the required pace of work for top scores. But that is actually lowering, not increasing, its ability to test for substantive ability.

But adaptive computer tests are a potential way around this problem. Because they can actually make the test harder as the computer learns that this test taker in particular can handle it. And in theory, you can entirely eliminate pace of work as a factor and yet still get the score discrimination you need across the full spectrum of college intenders.

2 Likes

The UCs aside, I think it’s a mistake to assume testing doesn’t matter. Some colleges have published results showing that students who submit test scores get in at higher rates. Here is one example from the University of Virginia:

The submission of standardized test scores was made optional for this year’s application cycle, something the University later announcedwould continue through at least 2023. In total, 42 percent of applicants opted to not submit standardized test scores, and 26 percent of the offers were given to students who chose to not submit standardized test scores.

It’s difficult to sort through the details (because we don’t know them) when schools like UVA publish that type of data. It is likely that the test submitters had overall stronger applications than did the test non-submitters
so it’s not necessarily true that the TO applicants weren’t accepted at a higher rate because they didn’t have test scores.

It does seem like some test optional schools (not including schools that communicate they are test preferred or test flexible) really prefer tests, and I would probably put UVA in that bucket. I would also say that about Northwestern and Michigan too. But I don’t have any data that support that! :woman_shrugging:

1 Like

I suspect timed tests have their place for certain types of schools. If content is delivered in a way that requires an enormous amount of reading or mathematical information is sequential and presented quickly, it’s going to be hard for someone who can’t get through the material to keep up. That doesn’t that the kid who can do it more quickly is smarter, more creative, or less capable of insightful analysis.

If the challenge isn’t simply to cover the marathon distance but to be at the finish line in x amount of time, speed matters. Otoh, there are many learning environments that do not require speed. And there are plenty of speedy workers who don’t have the endurance or desire to make anything of the “extra” time their speedy completion gives them.

It’s more a matter of discerning what a test measures and how critical those factors are to success at an institution.

Would more time help people figure out the answers better? where do you draw the line on giving more time? a half hour more? three hours? 15 more minutes? I’m sure for some extra mintues would really help; but for those who dont know the material, does more time really help? There are timed tests everywhere - in colleges, schools, for PE exams, actuary licensing, LSATs etc. etc. Are they all biased? Honestly - none of this makes sense to me. i guess my first thought in saying there is a bias in timed tests is making an excuse for those who just wont do well with tests.

2 Likes