USNews 2011 - New Methodology and Stanford's strange result

College Search & Selection
1

<p>This is not a thread to complain about the final rankings; there will be plenty of those. </p>

<p>This year, USNews introduced a few changes, including a change in the Peer Assessment. The resulting Academic Index seems to exhibit a rather strange behavior. </p>

<p>The definition of the new Undergraduate academic reputation index intimates that it is a weighted average of the Peer Assessment score and the High School Counselor score. </p>

<p>*But is it? *</p>

<p>MIT
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 98 </p>

<p>Yale
Peer assessment score (out of 5) 4.8
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 97 </p>

<p>Stanford
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 93 </p>

<p>Columbia
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.8
Undergraduate academic reputation index (100=highest) 93 </p>

<p>Caltech
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 92 </p>

<p>Penn
Peer assessment score (out of 5) 4.5
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 91 </p>

<p>Inquiring minds would like to know what kind of weigh could be applied to the average to yield such results</p>

<p>Here's a simple table that shows the results could be as simple as adding the numbers:</p>

<p>School PA - GC - TOTAL
MIT 49 49 98
Yale 48 49 97
Stanford 49 49 93
Columbia 46 48 93
Caltech 46 46 92
Penn 45 46 91</p>

<p>However, we know that the PA at 15% is twice the weight of the GCs score that is 7.5%</p>

<p>So, let's present a very simple table that doubles the weight of the PA.</p>

<p>School PA - PA - GC - WEIGHTED TOTAL - ORIGINAL SCORE
MIT 49 - 49 - 49 - 98.00 - 98
Yale 48 - 48 - 49 - 96.67 - 97
Stanford 49 - 49 - 49 - 98.00 - 93
Columbia 46 - 46 - 48 - 93.33 - 93
Caltech 46 - 46 - 46 - 92.00 - 92
Penn 45 - 45 - 46 - 90.67 - 91</p>

<p>The numbers work pretty well, except for one school. One has to wonder what Stanford did to Morse? :)</p>

<p>PS
Harvard
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 98 </p>

<p>Princeton
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 98</p>

2

<p>xiggi, if you do this:</p>

<p>(PA+GA)*100%</p>

<p>You will see all of them are correct, except Stanford’s. Regardless the weights, Stanford’s should be 98. Columbia’s might be due to the rounding error.</p>

<p>Simply put, the twisted mind of Morse hates Stanford. :)</p>

3

<p>Xiggi, they increased the graduate rate weight from 5% to 7.5%. Stanford’s four year graduation rate is unsually low (<80%) for a top tier school considering it has the second most inflated GPA (average 3.56) after Brown. I think that this drags it down quite a bit.</p>

<p>Ranking aside, Stanford lags behind in two important metrics for undegraduate quality: average incoming student SATs and graduation rate.</p>

4

<p>

Stanford is at its place, just Columbia is misplaced.</p>

5

<p>Xiggi: If you could, please post the rest of the scores up to #12, Northwestern.</p>

6

<p>

</p>

<p>Sorry, but that is simply not the case. You are confusing two distinct metrics. The ARI is now 22.5% and the Grad Rate is 7.5%. However, they do not form one 30% metric.</p>

<p>The increase in the graduate rate weight from 5% to 7.5% is TOTALLY irrelevant to this metric. The Academic Rep Index is NOTHING else than the weighted average of the PA and the GC rankings.</p>

<p>FYI, I just computed the ARI for the top 11 LACs and I could reproduce the index by simply applying the same “formula” </p>

<p>Try it for yourself. </p>

<p>

</p>

<p>No, the error is in Stanford’s number. It should be 98 and not 93.</p>

7

<p>

I was answering Professor101’s question. </p>

<p>I agree with you: regardless the weights, Stanford’s number should always be 98. Do they have yield numbers in their Premium Edition?</p>

8

<p>The Mouth
Peer assessment score (out of 5) 4.3
High School Counselor score (out of 5): 4.7
Undergraduate academic reputation index (100=highest) 89 </p>

<p>Coach K Landia
Peer assessment score (out of 5) 4.4
High School Counselor score (out of 5): 4.7
Undergraduate academic reputation index (100=highest) 90 </p>

<p>ObamaU
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.5
Undergraduate academic reputation index (100=highest) 91 </p>

<p>NU
Peer assessment score (out of 5) 4.4
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 89</p>

<p>In case there are still doubts about how ARI works. </p>

<p>School PA - PA - GC - WEIGHTED TOTAL - ORIGINAL SCORE
Dart 43 43 47 88.67 89
Duke 44 44 47 90.00 90
Chica 46 46 45 91.33 91
North 44 44 46 89.33 89</p>

9

<p>“are the PA scores hidden or something?”</p>

<p>Fourth column in the Premium Edition.</p>

<p>PS Sorry, EWHO. I misread your post.</p>

10

<p>Sorry, Xiggi, I confused the issue.</p>

11

<p>Though irrelevant, I wonder why Stanford’s (four-year?) graduation rate is low. There’s only like 25 or so people transferring out each year (because that’s how many they let in). Does this factor in people who leave for a year or a quarter and then come back, because that seems somewhat common. Also maybe the rate is low because of athletes red-shirting, which is very common. I imagine a red-shirt is not common at Harvard or definitely not Caltech. Or, because Stanford has low grad rates because it is so great and students want to stay there an extra quarter or two?</p>

<p>Also, if this mistake is real and not xiggi misreading data (I’m sure he looked it over, though, beforehand), and one of the weights that Stanford lags it was increased this year, then maybe it is all an intentional effort to undermine Stanford’s rank and clearly distinguish it from the traditional elite three. Also weighting guidance counselors undermines Caltech and Berkeley (probably also a low grad rate at these institutions). East coast bias haha?</p>

12

<p>Y’all are way too worked up over this. Fact is, Stanford is a great school (well-known by Californians), but not at the very top. There are other elite schools. I cannot believe you guys are breaking down US News’ methodologies to find errors. There are better things to do with your time.</p>

13

<p>^ Well, considering the significant influence of these rankings on the general public, helicopter parents and prospective applicants, this is a big problem. These posters are simply trying to work toward a solution.</p>

14

<p>

LOL, that is the best argument that I have heard in a long time. Yea, Stanford has its own golf course and I assume that a lot of students don’t want to graduate so that they can play golf or like Tiger Wood to flirt around.</p>

15

<p>^ You don’t say–if my college were to continue providing my family need-based F.A., I would stay for another year or two.</p>

16

<p>

</p>

<p>The impact of the new PA on Cal was minuscule. There was no impact whatsoever on Caltech. Only Stanford was hit by what appears to be a small clerical mistake that carries … epic repercussions.</p>

<p>Cal
Peer assessment score (out of 5) 4.7
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 93 </p>

<p>Stanford
Peer assessment score (out of 5) 4.9
High School Counselor score (out of 5): 4.9
Undergraduate academic reputation index (100=highest) 93 </p>

<p>Caltech
Peer assessment score (out of 5) 4.6
High School Counselor score (out of 5): 4.6
Undergraduate academic reputation index (100=highest) 92 </p>

<p>The graduation rates numbers for Cal work out pretty well. The grad rank corresponds to its overall ranking and the school benefits greatly from a lower predicted graduation rate. </p>

<p>Graduation and retention rank: 23
Average freshman retention rate: 96.5%
6-year graduation rate: 90.0%
Predicted graduation rate: 85%
Overperformance(+)/Underperformance(-): 5</p>

<p>FYI, Stanford’s numbers:
Graduation and retention rank: 5
Average freshman retention rate: 98.0%
6-year graduation rate: 95.0%
Predicted graduation rate: 94%
Overperformance(+)/Underperformance(-): 1</p>

17

<p>

</p>

<p>I still suspect it’s because Stanford has more students pursuing a 5-year masters than other colleges. Stanford’s 6-year graduation rate is on par with or superior to nearby colleges in the rankings.</p>

<p>So is it simply the Academic Reputation Index that’s broken or is the incorrect figure also generating an incorrect overall ranking?</p>

18

<p>^^Xiggi what is predicted graduation rate and over/underperformance?</p>

<p>yeah a lot of people coterm at Stanford too, although I thought most of them graduated at the normal time although I’m not sure?</p>

<p>edit: Wait why do people say Stanford has a bad graduation rate when it’s ranked fifth?</p>

<p>

This is a good question, and we may never know. I can’t imagine how this could not have been caught.</p>

19

<p>

</p>

<p>The wrong ARI is costing Stanford between 2 and 5 points in the overall ranking.</p>

20

<p>

</p>

<p>That is a good argument too. I hope that Stanford students are not reading this. ;)</p>