. . . but not for everyone. Details are here: https://www.washingtonpost.com/news/answer-sheet/wp/2016/02/27/something-you-should-know-about-the-new-sat/
In a nutshell, students who opt in to the Essay won’t have experimental questions. Those who opt out, will likely have an extra 20-minute section. College Board won’t give up much info, belying its new commitment to transparency and equity.
Unbelievable! Thank you for letting us all know.
Why is it that the ACT is able to create an accurate test without using the students who are taking the test as guinea pigs?
The SAT is long enough without compelling the test takers to do "free research" for the College Board.
Well ACT seems to be using students as guinea pigs while it figures out its new rubric for the writing portion.
This is really shocking news.
Do you really think that the equating section might be the FIRST section of the test? I think it would have to come at the END of the test because otherwise it would strongly influence the difficulty of the remainder of the test. ETS has shown many times that the same question that comes later in the test is harder because the student is more fatigued.
I also think it is pretty safe to deduce that the equating section WILL NOT count towards the student’s score.
If it counted, then the scored part of the test would be different for different students.
Even if CB swapped out some scored questions from another section, so that the sections had a mix of scored and unscored items, the test would be different from a test with only scored items and no equating section because of the additional fatigue experienced by the student prior to encountering at least some scored items.
Further, if the experimental section had math calculator questions, the students would have to get out their calculators to do it. What if during the experimental section they flipped to the non-calculator section and worked on a few problems with their calculators? – Even more security issues. Having a calculator out at the end of the test introduces the same security issues that already exist, not more of them.
By contrast, if the equating section comes at the end, it will not influence the student’s energy level during the scored part of the test.
So my hypothesis is that
What do you think?
Another reason to think that it will be at the end: At the end of the article, there is an email exchange:
I am running a post on the experimental section of the new SAT. Can you comment on the following:
Here’s how it will work: On the new SAT, the essay is optional. Students who opt to do the essay will be in separate rooms from those who do not. They will not have to do an experimental section. Many, but not necessarily all, examinees who opt out of the essay will begin the exam with a twenty-minute section that could cover math, reading, or writing. Students will, as in the past, not be told that this section will not count toward their score on the SAT, and they will not receive any information on how they performed on the section after the exam.
Here’s the response:
Thanks for your note, Valerie. James Murphy from Princeton Review has already been in touch with us for this story. This sentence is inaccurate: Many, but not necessarily all, examinees who opt out of the essay will begin the exam with a twenty-minute section that could cover math, reading, or writing.
But it seems to me that the only content in the sentence that is in any way in disagreement with the College Board’s announcement is that examinees “will begin the exam with…” the experimental section. So while being vague and unhelpful, they can truly say the statement is in error.
And one more piece of irony: based on one of the released practice tests (a psat maybe? I’m already blurring them), it would seem that one concept the college board wants students to know is that predictions made from small samples are not valid if the samples are not chosen randomly. And yet they are excluding from their own random sample those students who opt for the essay. Am I missing something?
@JamesSMurphy You say that those not taking the Essay will “likely” have an extra 20-minute section. College Board’s language is that “some” will have the extra section. The actual number may not be very high. Although it is likely that this section is unscored, students should not make that assumption. College Board emphasizes that any section may contain both operational and pretesting items. It’s not impossible that they will use the extra 20 minutes to have experimental questions spread across two sections. This seems unlikely, but is it a risk worth taking?
@plotinus I agree with your hypothesis, but it is just a hypothesis.
Still not out of the question that pretest items would fall on two sections. There are ways CB could randomize within the 20-minute and non 20-minute groups to account for any additional fatigue or item order effect.
College Board’s accommodations page seems to confirm that the section will be unscored: “Extended time test-takers do not take the unscored section of the SAT.” It’s possible, though, that this language just got recycled from the old SAT page.
@BobbyBaseball ACT and all other standardizes tests – GMAT, GRE, MCAT, LSAT – all use experimental questions, passages, or sections. They are essential for test construction and equating. ACT does its testing in a more limited fashion.
True, but all students take a test with the same number of questions and know what the composition and timing of the test will be before they take it. People don’t get a bonus for writing an essay or penalized for not writing it. This is a completely unique and egregious approach to equating/pre-testing.
What is going to happen with the May test or whenever there is the first QAS? I believe by New York State law, CB has to return a test booklet to the student with all scored items. So the student will know exactly which items were scored and which were not. Everyone will find out whether the experimental questions are all in one section or spread across two sections.
Even after the March test, students who took the test will be able to form reasonable hypotheses based on the number right and wrong they get for each question type.
Can we assume that the additional section, if there is one, will be just in one area?
Can you explain this?
Suppose the 20-minute section is the last section.
Next suppose scored items 10-20 occur in section 1 of the non-20 minute group test and occur in section 5 of the 20-minute group test. Won’t items 10-20 be much harder for the 20-minute group? How can you randomize this? You can’t stick the 20-minute section in a different spot for different people in the room (the way the old equating section was done), because there are no other 20-minute sections. It would throw the timing off.
We can also suppose that the 20-minute section is the first section. But this means that ALL the questions in Sections 2,3,4 and 5 will be harder for the 20-minute group than for the non 20-minute group because the 20-minute group did a whole extra section.
An extra section can make a big difference in the scores of the students, especially if it is in the area that the student likes the least. I don’t understand how you can randomize away the effect of an extra section. Holding all other factors constant, the scores of the people who do the extra section will be lower unless the extra section comes at the end and does not include any scored items.
I found this Khan Academy video about the New Sat from about a year ago. At minute 5:00 of the video, Khan says,
“The current SAT has a 25-minute experimental section, and there might be experimental sections in the New SAT as well.”
Khan does not call it a “pre-test” but an experimental section like the current one.
If this sentence is true, does it not entail that none of the questions in the extra section will count towards the student’s score (i.e., be “operational”)? The score report that comes to the student’s CB account after each test contains the number and type of questions that student got right and wrong. If any question in the experimental section is operational, the student would receive information on how he performed on this question in the score report.
I’m just amazed that we are days away from this new era of transparency and we are all just speculating, reading tea leaves. “How many sections will I have and will they be scored?” is a straightforward question and a reasonable thing for a student to know before arriving on Saturday at the test center.
@Plotinus @pckeller I just took the New SAT today and had the same question after the instructor wasn’t able to give us any information on section 5. Regarding the QAS, have people who took the March SAT’s received the question and answer service yet to see if section 5 was included on them? I’m hoping the college board uses this section just for experimental use and said some of the questions will factor in our score just to scare us into trying hard.
The short answer is that I don’t have any more information about whether Section 5 contains operational (scored) items than I had back in February. As I wrote back then, my best guess is that Section 5 doesn’t contain operational items because studies have established that a question taken later in the test is more difficult than the same question taken earlier in the test. Students who have operational items in Section 5 would therefore be at a major disadvantage compared to students who had the same items in an earlier section.
My guess is that all tests have some pretest (non-scored) items scattered throughout the tests, and that Section 5 contains a group of additional pretest items. However, this is just a guess. I could be wrong.
There is no QAS for March, and the students I know who took the test in May are still waiting to receive their QAS’s. In any case, I don’t believe any of them had a Section 5. Further, their online May score reports contain much less information than did the old SAT online score reports. In particular, there is no breakdown in the online score report of the number of questions right and wrong in each area. This is a major decrease in transparency. I don’t see how you could tell whether Section 5 was operational from an online score report.
Instead, you would need to look at the May QAS of someone who had Section 5 in May. If Section 5 questions are included in the QAS, they were operational – otherwise not.
@Plotinus
Interestingly enough I opened my mailbox to find the QAS for the May exam! In May, I was given a section 5 for writing and there were no results for section 5 included in the QAS but rather only questions included in the actual Writing section…I think this is good news
I suppose a third possibility is that some Section 5 questions were operational but nonetheless not included in the QAS. This would violate the terms of the New York State ruling against College Board as I understand them. Also, the score report would have to be fudged to make up for the missing operational questions. Thus I consider this possibility under the heading of “conspiracy theory”.
@Plotinus
Haha good point. Either way SAT should’ve been more transparent on this issue and I’m interested to see where all this goes
I am not a big believer in coincidence, so this timing makes it look like CB was holding back the May QAS’s until after the October test so that people would not know that Section 5 is not operational. It also suggests that CB won’t be using Section 5 in the future, since now the cat is out of the bag. Maybe CB needed to pre-test some additional items during the rollout but now it can pre-test enough by distributing them inside the main test. I guess we’ll find out more when the October QAS’s come out and also the November test.
@Plotinus
Absolutely. I was thinking the same exact thing because at least CB has three exams of experimental questions to work with. And I definitely agree with what you said about the delay because why would it take all these months to send out the QAS? I also think CB plans on making the SAT online which gives them more opportunity with experimental questions but for the time being, they have enough data to experiment with and once the online SAT comes out they’ll be able to experiment more. I’m just a bit upset with CB lying to us saying that “some of the questions are pretest while others are operational” and trying to be shady with all of that haha. Those jerks
A long time ago, everyone knew which section was the experimental section. Then Princeton Review started training its students to put their heads down on their hands during the experimental section. Sometimes the whole room would be napping during the experimental section. After that, CB made the identity of non-operational items top secret.
Oh that makes a lot of sense
There was a 5th section for some students not doing the essay today. Students were told that pretest questions (i.e., experimental questions that don’t count) could appear anywhere on the exam and questions that do count could be anywhere as well. This is concerning.