Aaron Lemon-Strauss, a College Board rep, has confirmed on Twitter that ETS is no longer writing the SAT for College Board. The new SAT is being written by CB in-house with the help of “contractors for scale.”
See AaronLS2 on Twitter
@marvin100 https://twitter.com/AaronLS2/status/687462641699307521
I’m still wondering where my $195 GRE fee went.
Wow, this is a big change. I wonder if ETS will continue to write AP questions and SAT subject tests.
@MITer94 ETS lists the GRE prominently on its website. I don’t think College Board has any relation to the GRE.
@Plotinus I know, I was just sayin’…
@MITer94 To find where the bulk of your $$ went, read this up: http://www.aetr.org/the-facts/ets/.
Back to the topic: it’s still unclear how big the ETS’s part in the rSAT will be:
http://thecriticalreader.com/what-is-ets-role-in-the-new-sat/.
Who will be doing the scanning?
Will the ETS continue the scoring?
@gcf101 I am much more interested in who is doing the WRITING than in who is doing the grading.
The ETS people are no angels but I like the test CB has written less in many ways than the tests ETS used to write. I think the ACT math is now more conceptually challenging than is the SAT math…!
@gcf101 well, now I know. Good thing I didn’t pay $27 per school to have my scores sent!
Also, apparently the price went up again…their website says it costs $205 to take the general test.
GRE vocab now seems like Custer’s last stand. I wonder how students from the Class of 2017 forward will make it into grad school…
On second thought, I AM interested in who will be doing the scoring, in the sense of who will be doing the EQUATING between different test forms. Is CB handing that in house too? I have read a couple of things about the mechanics of equating, but I never saw the nitty-gritty details. Anybody have an idea or a reference? And of course, another related issue is who will be doing the CONCORDING with the old SAT . I will tweet Lemon-Strauss again…
I found this
https://www.ets.org/Media/Research/pdf/LIVINGSTON.pdf
but I am not sure which of the described methods (if any) is used for the SAT.
Yeah, that’s the best info out there, but you’ll notice it’s 12 years old at this point, @Plotinus
@marvin100 So much for truth-in-testing …
And where in the test are the embedded anchor items? Don’t the anchor items have to be the exact same items? For reading, wouldn’t that mean… the SAME passage on every test? This is obviously not happening. Does that mean that the equating is done with study groups and not the whole test population?
Here was my question to Lemon-Strauss on Twitter
:
“Now that CB is writing the rSAT in house, will CB also equate different rSAT test forms in house?”
Here is his answer:
“yup, that’s right. We’re handling all form construction and psychometric functions, with support from outside experts and vendors.”
Lemon-Strauss also says that there will only be three difficulty levels reported on the Redesigned SAT reports:
My tweet: “Do you mean there will be 5 difficulty levels of questions on the rSAT but score reports will show only 3 levels? Why?”
His answer: “true difficulty doesn’t have “levels”, a p-value exists in the infinite space btwn 0 and 1. We’re bucketing into 3 bands to report”
I’m probably the only idiot on this site, but what the heck does “a p-value… infinite space… 0 and 1… bucking… 3 bands” mean?
What kind of convoluted, non straight forward response is that??
Does that mean three levels like 0, .5, 1? 1 being the most difficult?
So much for transparency.
@mmk2015 I’ve only seen p-values in the context of statistical tests. If you collect sample data, the p-value is the probability that you would obtain a more “extreme” sample than the one you just got, assuming the null hypothesis is true.
“Infinite space between 0 and 1” – why not just say between 0 and 1? Between any two different real numbers there are uncountably many real numbers in between them.
“3 bands” – probably 3 categories, e.g. easy, medium, hard. That would be my best guess.
@nmk2015
Maybe Lemon-Strauss is defining difficulty level as the probability that a randomly selected student got the problem right.
This probability will range between 0 and 1. Probability = number of students who get the problem right/number of students who took the test.
This is how difficulty level is rated in the scoring material of the released Math Level 2 tests (and I think also the other Subject Tests.)
However, since the number of students getting the problem right and taking the test are finite integers, the set of probabilities is a finite set and not a continuous interval.
These probabilities are arbitrarily divided by CB into 3 groups: low (=hard), middle (medium), and high (easy).
The old PSAT questions were rated E, M, and H (easy, medium, and hard), but the old SAT questions were rated on a scale of 1 to 5 (from easy to hard). The SAT used to distinguish between two levels of easy problems (1 and 2) and 2 levels of hard problems (4 and 5). As far as I can tell, there aren’t any level 5 questions any more. However, Khan Academy has 4 levels.
@MITer94 You may be right but I don’t see how this definition relates to difficulty level.
@Plotinus If the designers thought #25 was really hard and hypothesized only 25% would get it right, but from a random sample, it turns out that 50% got it right and you obtained p = 0.0153 or something, then maybe #25 wasn’t too hard. That’s my best guess…?
So if I’m correct, they’re not categorizing problems into five levels but instead three levels? (e.g. easy, medium, hard)