“Well, that’s the news from Lake Wobegon, where all the women are strong, all the men are good looking, and ** all the children are above average **.”
That was what, 30 minutes between my post at 2:39, feeling very reassured of my kid’s 221 in the highest cutoff and @payn4ward’s link to that article that burst my bubble? The roller coaster continues. I need to get a life.
@suzyQ7 ah, sorry for being the purveyor of bad news.
As I keep saying, it is already out of our/our kids’ hands. Nothing will change the outcome.
Congratulate on job well done and time to move on with our lives.
Off to ACT, staying away from new SAT (for kids with below 215/1450.) ACT essay scoring problems… will deal with that later.
I still think 220+ has a good shot everywhere. It’s the -12 theory that I’m against.
That seems like a strange thing to do, no? (I agree that if they did, that would invalidate everything.) A percentile has a definite meaning. Putting 10% of your distribution in the 99th percentile is illogical.
Also, thanks for the thanks :-), and note that my username is thshadow, not theshadow :ar!
" ACT essay scoring problems… will deal with that later. " I wouldn’t go near that. What do you do if they give a terrible essay score? Much rather deal with bogus percentile scores and slow score reporting that angered those of you from schools with less overworked GCs, but for us is such a huge improvement over having our GC sit on the scores for nearly 2 months.
He mentions New York, Seattle (Washington), and Atlanta (Georgia), so high cutoff states. It makes sense that if a 1360 is 99th percentile range of selection indices 196-212, you would have a huge number of 99% on total score. Also, a huge number based on 99th percentile selection index of 205. So let’s say the 99.5+ table is correct, the cutoff for those states would be 213, so any score from 205-212 would be knocked out, which would probably be a lot of scores.
If you read the article, he states that he concorded one of the 2015 99th percentile numbers to a 95th percentile number on the 2014 test. The range for 95th percentile for the 2014 test is 194-197. So, yes a 205 (99th percentile for 2015 if the table is correct from CB) can have a concordance of a 95th percentile number in 2014.
@thshadow One would think that they would respond to the blog ASAP. Someone should inform them of the accusation and see if we can get an official response.
I suppose schools sign some sort of agreement with the College Board when proctoring these exams, but I don’t understand why a few schools don’t band together and an anonymous set of scores to one person who could put them into a single file so that no specific student scores could be deduced from them, but so they could be used for statistical investigation, rather than trusting the College Board. If we had data like that from the same sources but from 2013, 2014, and 2015, then we would know whether the curve had changed significantly and we could get a good idea where the true 99% lay. Schools keeping this information secret simply empowers the College Board to deceive. For that matter, I wish we could create an uproar and demand that the College Board release the data sets from the past several years – actual scores and the number of students who received those scores, by grade, in CSV format. Wouldn’t that just make life easier?
With the new PSAT, I would be cautious about converting your 2015 SI to an equivalent 2014 cutoff and assuming it’s an approximation of this cohort’s NMSF cutoff. The “shape” of the bell curve for this cohort’s scores is going to look very different from the bell curve from previous years, because of a number of big factors:
-
It’s an intrinsically different test. There is no obscure vocab. The math topics are different.
-
No guessing penalty
-
test was given on a school day, so there was likely to be higher participation rate than on weekend-administered tests. I’ve plotted PSAT participation rate vs test scores, using % of students making Commended as a proxy for high-scoring success. There is a trend showing that as a greater proportion of kids in a state take the PSAT, the higher the %percentage of kids scoring highly.
-
the 2015 PSAT score doesn’t have a one-for-one correspondence to SI, the way previous years did, because this cohort’s PSAT score gives double weighting to the Math subscore but the SI does not. So you can’t look at your 2015 PSAT percentile and directly translate it to an SI percentile.
We’ll have an anchor point for the low-end NMSF cutoffs once the Commended cutoff is known.
14
“For all we know there are 30,000 kids, not 7500-8000 kids, being told they are at the 99+%.”
Everyone on CC is in the top 1%!!! Everyone gets a trophy!!!
The PSAT is better than Lake Woebegone (where all kids are above average).
Our house was pleasantly surprised when our kid hit the 99th percentile on the PSAT. Now, who knows what that means. Especially since that kid has the exact same stronger verbal / weaker math profile described in the blog post above. Oh well…
This supports my pet theory that CB is being extra generous with percentiles to win more kids who were planning on the ACT already.
@Studious99 Well it worked. My kids is taking the ACT (has taken practices but doesn’t like it much), but with a 1480/221 SI there is NO WAY that he’s passing up the March SAT. If he scores 1480/1600 on the SAT he will be done with testing. I wonder what that would equate to vs. the ACT??? HAHA, just kidding ( if someone can figure that out, it would be great ).
Colleges are going to have a blast with class of 2017 and all these changing test scores.
30
I agree. The working assumption for our kid was that the kid would likely do better on the ACT than the SAT. Then the 99th percentile PSAT report showed up, which called our working assumptions into question.
Most likely, we will be going back to our original ACT plan.
@GMTplus7 When will the commended cutoff be announced?
I read the applerouth article. Much of the article talks about the National Sample Percentiles being inflated. We already knew that. Also talked about the Concordance Tables not making sense. We already knew that.
The item I found interesting is his statement that the SI % table was based on a research sample: “… the “Selection Index” percentile. This was still calculated using a research sample, but the sample was limited to 11th grade students.”
I can’t find anything in the CB Guide that indicated the SI % table was based on a research sample. Does anyone else see that in the CB Guide? I wonder if he made that up. I emailed Jed Applerouth. I’ll see if he responds.
I’m also going to email CB about the SI % table. See if they respond.
^^ Thank you @Speedy2019 please share the results of that e-mail to Jed Applerouth and CB. CB has removed the bullet-point explaining the source of those SI percentiles (you do see it on the 2014 percentile tables but not the 2015). CB doesn’t actually say WHERE the 2015 SI percentiles come from. With all the suspicion of “inflated percentiles” (most of it misplaced, I believe) it would be helpful to get some clarity on that table.
Please ask them to correct the Mean and SD as well. Thanks again!
Yes, assuming no malice, but uh, carelessness, I think that the mean and stddev in the SI percentile table is a cut and paste copy error. I’ve done that a million times myself.
I also think that SI table is probably accurate because it would be more work to make it up than just dump the scores.
Now, the concordance table, that I’m skeptical about. It’s stamped preliminary and seems to contradict some of the other information. But then I don’t know anything about how to make one of those. I guess they they might be important to guidance counselors.
I think this is a clear case of where “Show your work” would really help.
My gut feels like there is some clustering at the top, so I tend to go with the higher cutoffs. I really don’t know.
@Mamelot, yes, they do say where the percentiles come from. A “research study sample”. Evidently their research study wasn’t too good. I still can’t understand what purpose all this manipulation serves. Why try to make a representative group that represents the population when you can just use the data from the population?
“For the next few years, norm groups for the score ranges, mean scores, and percentiles described below will be derived from research data, not the prior year’s test-taking populations. A norm group, also called a reference population, is the group whose data your results are compared to.” https://collegereadiness.collegeboard.org/psat-nmsqt-psat-10/scores/student-score-reports
“Nationally representative percentiles are derived via a research study sample of U.S. students in the student’s grade (10th or 11th), weighted to represent all U.S. students in that grade, regardless of whether they typically take the PSAT/NMSQT. For example, a student’s score in the 75th percentile means that 75 percent of the nationally representative group of U.S. students in the same grade would have had scores at or below that student’s score. User group percentiles are derived via a research study sample of U.S. students in the student’s grade, weighted to represent students in that grade (10th or 11th) who typically take the PSAT/NMSQT” https://collegereadiness.collegeboard.org/pdf/2015-psat-nmsqt-understanding-scores.pdf
@mathyone at the risk of sounding a bit paranoid let me explain further:
I agree that the wording on the “Student Score Report” section of the website will describe the percentiles for the score report - but the SI percentiles are NOT part of the Score Report. They are (so far) unique to the “Understanding Scores 2015” report.
The table on page 11 of the “Understanding Scores” document is labeled “Selection index (Grade 11 Only)”. Where on page 11 or page 6 does it actually describe what that is? Page 6 includes a description of the representative sample and the user sample which ARE both based on a research study - but “Grade 11 only” might cover EITHER of those categories. There are representative samples for grade 11 AND user samples for grade 11.
The reason I’m pushing this is that some are saying “research study” and others are saying “actual percentiles from the 2015 PSAT”. There seems to be some disagreement - which supports the idea that the percentiles aren’t exactly clearly defined.
That bullet point that typically explains where the percentiles came from has been removed (compare page 10 to the bernardsboe website that has the CB 2014 information). Why didn’t they just update describing a “research study” OR describing that these are ACTUAL percentages? Very odd, IMHO.
Update/Addendum: I should add that the broadened purpose of the PSAT justifies a detailed explanation of which set of percentiles is used for what assessment. Right now there are two groups of percentiles to address two determinations of the PSAT test: 1) NM qualification; and 2) college readiness tracking. Therefore, it’s imperative that CB clarify the origin of the SI percentiles, as they did all other tables in their “Understanding Scores” report. Otherwise, things can get a bit confusing.
Percentiles don’t assume any “shape”. Whether it’s normally distributed or not, if the percentiles are accurate, they should directly translate to NMSF cutoffs…