***Class of 2016 NMSF/NMF Qualifying Scores

Paying for College National Merit Aid
121

Ok, I had a bit of time and I took a look at the Summary Reports for Texas this year and the previous 6 years. I decided I would do a linear regression to see if I could find a correlation and see what that says about this year’s results and the possible cutoff NMSF cutoff value.

Please feel free to comment or poke holes in my methodology. For each of the previous 6 years I did the following:

  1. Calculated the percentages of students who scored a 75-80 for each of the 3 subject areas. (They provide the percentage to 1 digit, so I calculated it based on the actual tests to give more significant digits).
  2. I added those 3 numbers together. Typically the numbers for each subject area were near 1 percent each.
  3. I then associated this number with the cutoff number for that year
  4. I plotted the cutoff number vs the sum of the percentages for each of the 6 years
  5. I fit a best fit line and had excel give the equation of the line and the R2 value (goodness of fit)
  6. I calculated the sum of the percentages for this latest test to see where it would fit to see what a likely value would be.

Known issues:

  1. I know that a person can score high on one test and not the other two and not be NMSF. However, I thought this methodology might give an indication as to how hard the tests were from the various years.

Results:
I first did this for Texas as that is where I live. The R2 value was 0.82 which indicates a pretty good fit. So, I decided to do this for other states: CA, NY, FL (also big states) and the DE (small state which also had a different trend of cutoff scores than Texas. Texas has been trending upward while DE is doing the reverse, it seems). There was a correlation in all these states, but the R2 values were not as good.

So, what do the results say for Texas? Based on the equation of the line, this methodology would say that the cutoff will be 217.54. Lately the results have been slightly above the line, so my best singular guess would be 218.

If I had to put percentages on my guesses (and they are guesses), I would say:
40% - 218
23% - 217
20% - 219
9% - 220
8% - 216

I may have given 220 and 216 too high of numbers. Last year the cutoff was 218 and the percentages declined slightly this year from last. (0.02098 vs 0.2117 in the table below). The reason I’m interested is because my son scored a 219. Texas has never had a number as high as 220 in recent history. It would appear that my son has a 90% chance of making it.

Sorry for the formating, but here are the numbers in csv format:
TX,2008,2009,2010,2011,2012,2013,2014
CR,0.005694696,0.005212882,0.005625817,0.00416715,0.006461645,0.005365187,0.004883784
Math,0.007079737,0.009487768,0.011577417,0.005915131,0.008731431,0.01104347,0.01017712
Writing,0.005414253,0.003751981,0.007347113,0.006789122,0.007282632,0.004762496,0.005923075
Sum,0.018188686,0.018452631,0.024550348,0.016871403,0.022475708,0.021171153,0.020983979
Acutal cutoff,216,215,219,216,219,218,
Value of Line,216.056381,216.1967998,219.4407851,215.3555862,218.3370769,217.6430535,217.5434769
Abs Error,0.056381035,1.196799819,0.440785086,0.644413798,0.662923098,0.356946467,

BTW, I found the summary reports from past years by googleing: 2010-2011 college-bound high school juniors summary report . Then clicking on the first available report and changing the state abbreviation in the url to the state I wanted. I changed the year in google each time I wanted to change years.

Any comments on this?

122

@WRUAustin Great analysis. Can you do one for New York?

123

NY was one of my comparison cases. The R2 wasn’t nearly as high (0.57), however. I just had to plug in this years’ numbers. I will make the spreadsheet available soon on dropbox if anyone wants to do their state in a similar fashion. I will do that at the end of the day.

For NY, the best guess is 219.012 (using extra significant digits so that you can locate the prediction in the csv table below). The sum of the percentages is higher than any of the previous years. So a new high of 220 is possible. But I wouldn’t expect any higher than that. I hope that wasn’t bad news.

2008,2009,2010,2011,2012,2013,2014
0.00650675,0.00558277,0.006743171,0.004828651,0.007776976,0.006065201,0.005609222
0.009588553,0.012946864,0.012574754,0.007747998,0.009947911,0.015367908,0.014994542
0.005050015,0.006215269,0.007098755,0.006360005,0.008895537,0.006652596,0.007929806
0.021145317,0.024744903,0.02641668,0.018936654,0.026620424,0.028085705,0.02853357
218,217,219,215,219,218,
216.6555677,217.8038715,218.3371851,215.950982,218.4021815,218.8696208,219.012494

124

@WRUAustin‌ Very impressive. What is your estimate for California?

125

@WRUAustin Thanks! I sure hope you are right. Our DS scored a 225 as sophomore but with the same #wrongs (10) this year. got 220. Wish they had a “super score” for PSAT. lol

126

@nycuw Sounds like our offspring are both in a good, but not completely assured positions. Good luck.

127

I decided to remove the 2013 test taking data (Class of 2015) from the equation of the lines and see how close the predictions would have been.

TX: Last year predicted 217.6 and it was 218
CA: Last year predicted 221.8 and it was 222
DE: Last year predicted 216.2 and it was 215
NY: Last year predicted 219.4 and it was 218
FL: Last year predicted 211.8 and it was 211

So, while it wasn’t perfect, I think it was really pretty good.

I calculated the best guess CA score for @seal16 and it is 221.9. CA R2 is 0.61 so that isn’t as high as Texas. CA is different in that there are so many people who score highly in Math there that I was concerned it would throw it off. Last years prediction was almost dead-on accurate, though.

I have uploaded the spreadsheet to dropbox including instructions. I would prefer not to look up any more numbers as it takes about 10 minutes per state approximately. I hope this helps.

https://dl.dropboxusercontent.com/u/35818861/PSAT_cutoff_predictions2.xlsx

128

Wow! I think we’ve found Nate Silver on this site. Amazing work. I’m going to see if I can figure out how to plug in FL to your spreadsheet. Thank you!

Edit: Do you already have the FL number, as it was one of your comparison cases?

129

Amazing @WRUAustin‌

130

@Leonidas1, I have not calculated the FL number, but I have the equation already calculated. You just need to plug in the numbers for this year. You might also do last year just to verify that you can duplicate what I have in there.

The FL R2 was relatively high, so hopefully the line equation makes for a good predictor.

Good Luck.

131

Wow, that’s some impressive calculating! Have you done a comparison for MD?

132

@WRUAustin, not only has Texas not had a number as high as 220 in recent history, it has fortunately NEVER been 220. Also, as people look at the stats in their states, don’t forget to look at the numbers of students in a score category, not just the percentage. Last year Texas had a huge increase in number of test-takers, and as a result, even in score categories where the percentages went down, the actual number of students in the category went up.

Last year, @PAMom21 did some amazing statistical analyses! Maybe she will jump in here with some advice!?

133

@BarflyThanks for your comments. From what I understand, NMSF is based on the top 1% of scores in a state. That is why I did the calculation as I did. So, it seems that the percentage in the top score category would be relevant. I did calculate the percentages based on the actual number in the category because the summary report only reports to 1 significant digit.

If my NMSF basis is incorrect, please let me know. If someone can point me to @PAMom21 analysis from last year, I can see what I can learn from it.

@crimsonwarrior I have not done MD. It isn’t hard. It just takes some time to get the reports and enter the data. I would recommend writing over one of the existing entries in the spreadsheet. That way the formula in the graph should autoupdate. They you will just enter in the formula in the cells in row 14.

134

2 corrections to last post (It doesn’t let me edit it after 15 minutes).

Should be @Barfly

Also, last line should read: Then you will just enter in the formula in the cells in row 14.

135

@WRUAustin, it is actually based on the number of graduating seniors in a particular state. There will be about 16,000 NMSFs. Example: If a state has 10% of the nation’s graduating seniors, then that state’s cutoff score will be set so that about 1600 students will make NMSF. In a state like Texas where a large percentage of juniors take the PSAT, only the top .5% approximately will make NMSF. In other states, the top 2% might make NMSF. Note: not sure which year is used for number of graduating seniors. I assume a prior year, because the data on number of graduating seniors may not be available until long after the cutoff scores are set.

136

I see what @Barfly is saying. I am not a numbers person but I will say your numbers seem to match the past trends. I don’t see where you have factored in the number of NMSF allocated to the state of TX. From NationalMerit.org:

. The Selection Index scores of
students who met program participation requirements
were used to designate a pool of about 16,000
Semifinalists on a state allocation basis.

The allocation is based on the number of high school graduates, Since TX has grown in population we might have a larger allocation which would might push the cut off slightly lower.

137

@wruaustin - nice analysis. No idea if any of this works but I will take 218 for Texas!

138

@Barfly

Thanks. I understand. The total number of students in the state is more meaningful than the number taking the exam.

If I knew the number of graduating seniors in a state, I could correct for the changing number of test takers.

However, even though the changes in the number of test takers in Texas has gone up about 5% each year for the last could of years, I think the effect of that is small compared with the other changes I’m calculating. Plus, given the growth in the Texas general population, some of that growth in test takers is likely due to more students. Though I’m glad that I understand it better now, I’m not sure that it is worth it (to me) to dig any deeper. The correlations seem relevant enough for my purposes. If some others gain some usefulness, then that is a bonus.

Thanks again!

139

I’ve seen a published list of numbers of NMSFs per state in particular years, and I don’t think that changes much from year to year, at least not in Texas. But for the class of 2015 (they took the PSAT in 2013), that year had a huge increase in number of test takers in Texas, so it skewed the percentiles. Still, most of us predicted a cutoff of 218-219, and it was 218. We all used different methods, but sort of got to the same cutoff prediction. Good luck everyone!

140

@Barfly

Do you know what “most of us” are predicting for TX for the class of 2016?

Thanks