I don’t think that time pressure is much of an issue in professional licensing exams, maybe not even in standardized tests to enter grad schools. The last standardized test that I took was my specialty boards, and before that the licensing boards. I don’t remember time pressure having been an issue for any of those; in fact, I don’t recall time pressure having been much of an issue for the MCATs, either. But I still can remember time pressure having been a HUGE issue for SATs and PSATs, 45 years ago. I really do think that it’s the test makers having trouble making a test that is limited to high school level math and English, that can separate out the top scorers so that you don’t wind up with a perfect score being 95th%, unless you add in severe time pressure as an additional factor, so that’s exactly what they’ve done. The test then measures not only mastery, but the ability to perform at lightning speed, which is not academically relevant. Add in the fact that so many students are able to obtain 150% time on the tests, and the test become an unfair measure of mastery of the material or academic potential, whatever it is that they’re supposed to be measuring.
Ask anyone who has recently taken the bar exam about time pressure.
It’s not just timed- it’s lengthy (two days in many places). And so it is testing stamina, recall, reasoning, etc. which for many lawyers is a decent proxy for what it’s like to lawyer.
It’s perfectly possible to do that in math, most Olympiad questions don’t require more than geometry as a baseline.
But with potential low scorers deciding not to take the tests and going test-optional instead, there’s no incentive to make the test harder when you remove the time pressure: consumers (students) would prefer it if they got an even higher score, so long as the colleges they want to apply to still take account of those scores.
An adaptive test which can distinguish meaningfully between the top 1% and 0.1% might be useful to (some) colleges, but wouldn’t be popular with 90% of those top students!
Pace of learning and pace of work are really two different things, though.
A fast reader is not necessarily a fast writer, for example.
So the science says these tests are correlated in the sense they all tend to test for a certain type of pace of work.
They are therefore all biased to the extent they are testing for that rather than the actual substantive knowledge and abilities they are nominally testing for.
I note there is an ongoing parallel discussion about the utility of tests like the LSAT, including the problem of US college curriculums and grading systems being just as non-standardized as high school curriculums and grading systems. So the reasons why law schools would use such a thing for filtering even despite its known flaws are very similar, but so are the reasons why they might prefer an alternative if it was as easy to obtain for large volumes of applications.
I think the new digital SAT is going to try and do that - at least from what I’ve read here. The algorithm gives test takers increasingly difficult questions if they pass some threshold. Others are routed to easier questions, but their scoring potential is capped. I think the test is only 2 hours.
I would not describe it as increasingly difficult (such as MAP test that is constantly reassessing).
From what I understand everyone takes the same first module. Depending on how you do you get one of two other modules for the second part. The easier mode caps your score because it will not include the more difficult questions that are worth more points.
It’s important to point out that the threshold’s target is in the area of a predicted 600 score, and a high achieving student may not notice a significant difference in difficulty between modules. Both have a combination of easy-medium-hard questions, skewing one way or the other.
I acknowledge that fast in one thing isn’t fast in another. But if content is delivered in a way that a certain speed of A is required, it makes sense to test for speed of A.
Whether there is another way to deliver content that doesn’t require speed of A, well that’s a different issue.
And of course, working at that speed is no test for creativity or ability to sustain that speed.
But with all of that said, in many non-academic realms, speed will matter.
There are several practice tests on CB’s websites for students to use.
I have attended a number of digital SAT sessions run by test prep companies, and their feedback has been similar…the difference between Module 2’s easier and harder options is very noticeable. For many students, when they walk out of the test thinking it was ‘easy’, they likely had the easier second module.
Test prep experts also report that the max score for someone who had the easier Module 2 is capped in the 650 range, give or take (because all questions are not worth the same, which is a big change in how SAT scoring works).
So does that mean the highest total score for those getting the easier module would be around a 1300?
One would think you should be able to get hard in one and easy in the other - that would make more sense.
Yes, that is my understanding, if the student got the easier modules in Module 2 both the math and reading sections.
The student’s performance on the first section in each of math and reading determines which module two they get for the second section in that subject. Math and reading are independent of each other, meaning a student could get the harder module 2 for math, and the easier module 2 for reading, to take one example.
Here are some more resources:
This Compass resource (from Dec 2022) does show a max 600 on Math for students who got the easier second module, but another national test prep company said in a recent webinar they are seeing around 650 max (again give or take).
Compass testing guide here, available for free with email: Guide - Compass Education Group
Some good info here too, including graphics:
Deeper dive into practice tests:
I agree that could be potentially relevant, but how would you test for how fast a student can pick up new concepts during a term with a test like the SAT/ACT?
The standard way of monitoring this in an actual class is with regular quizzes. The quizzes themselves don’t need to be time-pressured, but you are checking in to make sure the student has learned what they needed to have learned during the last period in order to be ready for the next period.
I am not sure how you could incorporate that sort of thing into these types of one-time tests.
I wonder why the SAT didn’t choose a continuously adaptive test. I think the GMAT went to a continuously adaptive test over 15 years ago.
The time pressure factor of these tests is so extreme that it becomes the major point assessed. This is totally inappropriate, irrelevant to future academic potential, and most of all, in wealthy areas, a large percentage of the students are getting extended time! There is just no excuse for making time pressure the major factor of these tests, especially when it is not applied equally.
You’re just testing reading comprehension, for example, because it’simpossibleto know what is "novel for each student. . And yes, reading about something you know about is easier, which is where the original issues with bias came in. The reading passages are testing for speed of reading with comprehension, a test that could be relevant to a student who will need to read hundreds of pages each week.
It would be interesting to extend time for everyone because at some point, it’d be more valuable to change difficulty than pace required, which is presumably the goal of the digital tests described below.
This is exactly what College Board and ACT should have done at the time that they stopped flagging tests that were administered under altered conditions - extended the time for everyone, or minimized time pressure on the test. But they didn’t, because after all, they cannot make it more difficult by testing on material that is not covered in the standard high school curriculum, so if they had extended the time for everyone, scores would have skewed higher, and the differentiating ability of the test would have been diminished.
You give CB too much credit! (I’m ignoring ACT because I doubt their ability to continuing playing in this sandbox).
I think CB went with a digital test primarily to appeal more to students by offering an easier test (does remain to be seen, but some international students and/or test prep experts are reporting this), shorter test (in the face of declining demand for these tests) and because a digital test shuts down many of the ways to cheat (quite common especially in the international market).
ETA: to clarify what I mean by easier, I don’t know that the content of the test is necessarily easier for students in terms of average reading level or what’s covered in math. Many feel it is easier simply because this test is 1/3 shorter, so a student doesn’t need to have nearly as much physical and mental endurance taking this test. There’s also more time per question. Will scores reflect these changes? We won’t know that for awhile.
They also got rid of long passages with multiple questions. I think that makes a significant difference for some kids.