<p>Thought,</p>
<p>The study is NOT based on matriculation rates. Again, read what they wrote. They even present a table comparing their ranking to the one that would be obtained from matriculation rates, and show that the results are quite different.</p>
<p>The hypothetical you describe is ambiguous as to whether Cornell or JHU would be ranked higher. It would depend on what decisions were made by students who had choices that included at least one of these colleges.</p>
<p>Oh, well I don't really understand what they wrote</p>
<p>I thought it was matriculation rates or something</p>
<p>Look guys, we can all get into the minutiae of why any particular ranking / methodology is flawed. I think it's safe to agree that even among a group of highly accomplished / educated people no one methodology would emerge unanimously.</p>
<p>Any given ranking will have its own particular strength(s) and particular weakness(es). With that in mind, I chose the USNWR rankings for one simple reason: it is the most established / widely recognized. I chose to average the rankings over time for two simple reasons: 1) the data was available and 2) an average over time is simply a better metric than a single snapshot for any particular year. Furthermore, the other "world" rankings out there (namely the THES, Shanghai Jiao Tong University, Newsweek rankings) don't differentiate between graduate programs and undergraduate programs - in point of fact these are really rankings of graduate / research universities RATHER than ranking the quality of undergrad programs). Some were just not established / lacked credibility (e.g. Washington Monthly). </p>
<p>The WSJ Feeder ranking does a respectable job of identifying those undergrad programs that excel in getting their respective graduates into top grad programs. But is it the "best"? Again, you can argue over "which" programs were identified as the "elite" programs, but, again this is a slippery slope into the neverending world of the minutiae. I don't know of another ranking out there that even attempts to look at graduate school placement (remember I'm talking about PLACEMENT not rankings of the any given school's actual GRAD PROGRAMS - i.e. National Research Council etc.)</p>
<p>The Revealed Preferences is also significant because it measures something that the other two do not, that is the actual student's themselves - and as Sakky has mentioned - the Revealed Preferences methodology is a well established methodology in economics (and other fields of study) for analyzing choices.</p>
<p>This "triangulation" of the three rankings (IMO) seemed like a better one than any one of the three on a stand-alone basis - and since the USNWR is the most established ranking - I gave it the highest weighting of the three.</p>
<p>Is there a perfect ranking out there? Not that I've seen.</p>
<h2>I don't know why you would average all the scores since 1991. Some schools have improved vastly since then. It would be more reasonable to average the last five years.</h2>
<p>
[quote]
I don't know why you would average all the scores since 1991. Some schools have improved vastly since then. It would be more reasonable to average the last five years.
[/quote]
</p>
<p>Have you seen the kinds of "jumps" and "freefalls" that any given university goes through in a short span of 5 years? Again here's a sampling:</p>
<p>
[quote]
- Brown makes a ten notch jump from No. 18 to No. 8 over the course of four years ('93-'97)
- UPenn also makes the "ten notch" jump from No. 16 to No. 6 over the course of five years ('94-'99)
- Northwestern does the "ten notch" in one single year! (Go Purple!) - from No. 24 ('91) up No. 14 ('92) - then proceeds another five notches in five years to No. 9 in ('97)
- Cornell jumps nine spots from No. 15 ('95) to No. 6 ('99) in a span of four years
- Cal (Berkeley) does a nose dive of fourteen spots from No. 13 ('91) to a lowly outside "Top 25" - No. 27 in ('97)
- University of Michigan: freefalls 17 spots in a 5 year span, from no. 7 ('88) to no. 24 in ('93)
- Washington University: leapfrogs 12 spots in a 5 year span, from no. 20 ('98) to no. 9 in ('03)
- CalTech goes from No. 9 (for three straight years '97-'99) then inexplicably jumps to the No. 1 spot on '00! Man I bet the geeks at MIT (always the bridesmaid never the No. 1 bride) were so livid their taped specs steamed up at the mere thought that it might be the ultimate ingenious prank from Pasadena...
- But, the "Bob Beaman-Olympian-Like-Jump" Award goes to JHU. Johns Hopkins jumps a whopping twelve spots in one year from No. 22 ('95) to No. 10 ('96) then proceeds to break into the Top 10 to go to No. 7 ('00)
[/quote]
</p>
<p>Five years - heck even ten years - is too short a time period, especially if you have a larger set of data to use.</p>
<p>Furthermore, if any given school experiences a legitimate and sustained upward (or downward) trend, that would be reflected in the overall average: </p>
<p>i.e. if College A is moving up in the rankings over time vs. College B (or vs. the rest of the field for that matter), College A's overall average ranking will reflect that upward trend.</p>
<p>take for instance UPenn's overall average rank: 8.65 - their recent upward trend is reflected in this number - esp. considering that Penn hovered in the </p>
<h1>11 to #16 range from '91-'97.</h1>
<p>cool list, hope others notice</p>
<p>thanks for the kind words - hope others will as well.</p>
<p>It's true that I don't read in detail complex studies that I think are flawed conceptually from the outset. So I could be wrong. I doubt it though.</p>
<p>IF you're telling me that the study did mark down comparable schools that weren't applied to I might look. Frankly I doubt this would have been done in a way I would find satisfying.</p>
<p>Considering excluding a school "for noise" is not the same as inserting a datum indicating it is less desirable than others to the non-applicant, and weighting this datum in the results.</p>
<p>I'm sure they tried to address their shortcomings in some fashion, but I doubt that they did it in a way I would agree with.</p>
<p>I recall glancing at the results, and seeing some school like Notre dame unrealistically high IMO. The reason this would have occured in such a study jumped out at me, and after that frankly I had no interest in looking further.</p>
<p>THe "non-revealed preferences" via non-application are more significant, and not always the same as, the revealed ones IMO.</p>
<p>The sample, pools of applicants, is a biased sample and does not prefectly represent the opinions of the underlying population which includes non-applicants.</p>
<p>^yeah, I'm with monydad on this</p>
<p>
[quote]
It's true that I don't read in detail complex studies that I think are flawed conceptually from the outset. So I could be wrong. I doubt it though.</p>
<p>IF you're telling me that the study did mark down comparable schools that weren't applied to I might look. Frankly I doubt this would have been done in a way I would find satisfying.</p>
<p>Considering excluding a school "for noise" is not the same as inserting a datum indicating it is less desirable than others to the non-applicant, and weighting this datum in the results.</p>
<p>I'm sure they tried to address their shortcomings in some fashion, but I doubt that they did it in a way I would agree with.</p>
<p>I recall glancing at the results, and seeing some school like Notre dame unrealistically high IMO. The reason this would have occured in such a study jumped out at me, and after that frankly I had no interest in looking further.</p>
<p>THe "non-revealed preferences" via non-application are more significant, and not always the same as, the revealed ones IMO.</p>
<p>The sample, pools of applicants, is a biased sample and does not prefectly represent the opinions of the underlying population which includes non-applicants.
[/quote]
</p>
<p>Guys, come on. I have to agree with afan here when I say that you guys need to ** read the paper **. Again, all of these objections that you have raised were all dealt with in the paper, particularly in Section III, and particularly in equations 3-6 with the construction of the multinomial vector. </p>
<p>Please, if you have a methodological objection, then let's hear it. However, it seems to me that the authors of the paper are using well-established and reasonable modeling assumptions and techniques that are within the mainstream of economic modeling methodologies.</p>
<p>OK, so I finally managed to input and incorporate the 2005 National Merit Scholars into my Ranking (<em>whew</em>)...</p>
<pre><code> USNWR ('91-'07) Avg (USNWR Wgt.) WSJ Feeder (WSJ Wgt.) Rev. Pref. (RP Wgt.) NMSC (NMS Wgt.) Total Weighted Average
</code></pre>
<p>1 Harvard University 1.41 0.42 1 0.30 1 0.30 3.00 0.30 1.32
2 Yale University 2.53 0.76 2 0.60 2 0.60 2.00 0.20 2.16
3 Princeton University 1.82 0.55 3 0.90 6 1.80 4.00 0.40 3.65
4 Stanford University 4.65 1.39 4 1.20 3 0.90 6.00 0.60 4.09
5 Massachusetts Inst. of Technology 4.82 1.45 8 2.40 5 1.50 5.00 0.50 5.85
6 Dartmouth College 8.47 2.54 7 2.10 10 3.00 11.00 1.10 8.74
7 Duke University 6.06 1.82 6 1.80 19 5.70 10.00 1.00 10.32
8 Columbia University 10.06 3.02 11 3.30 8 2.40 16.00 1.60 10.32
9 Brown University 13.41 4.02 12 3.60 7 2.10 14.00 1.40 11.12
10 California Inst. of Technology 5.71 1.71 28 8.40 4 1.20 1.00 0.10 11.41
11 University of Pennsylvania 8.65 2.59 16 4.80 12 3.60 18.00 1.80 12.79
12 Rice University 15.12 4.54 20 6.00 17 5.10 8.00 0.80 16.44
13 University of Chicago 11.47 3.44 14 4.20 27 8.10 17.00 1.70 17.44
14 Georgetown University 21.71 6.51 17 5.10 16 4.80 24.00 2.40 18.81
15 Northwestern University 12.65 3.79 21 6.30 21 6.30 27.00 2.70 19.09
16 Cornell University 12.12 3.64 25 7.50 15 4.50 42.00 4.20 19.84
17 Johns Hopkins University 14.18 4.25 24 7.20 28 8.40 28.00 2.80 22.65
18 University of Notre Dame 19.00 5.70 35 10.50 13 3.90 26.00 2.60 22.70
19 University of Virginia 21.06 6.32 33 9.90 20 6.00 40.00 4.00 26.22
20 University of California-Berkeley 20.47 6.14 41 12.30 23 6.90 37.00 3.70 29.04
21 University of Michigan-Ann Arbor 23.71 7.11 30 9.00 42 12.60 47.00 4.70 33.41
22 Emory University 17.86 5.36 36 10.80 61 18.30 39.00 3.90 38.36
23 Carnegie Mellon University 22.94 6.88 51 15.30 46 13.80 29.00 2.90 38.88
24 Washington University 15.94 4.78 47 14.10 62 18.60 20.00 2.00 39.48
25 Tufts University 26.50 7.95 45 13.50 40 12.00 75.00 7.50 40.95
26 University of California-Los Angeles 25.38 7.61 61 18.30 36 10.80 64.00 6.40 43.11
27 U. of North Carolina-Chapel Hill 26.29 7.89 90 27.00 31 9.30 51.00 5.10 49.29</p>
<hr>
<p>Changes from Initial Ranking:
Rank School Change</p>
<p>09 Brown University 0
11 California Inst. of Technology 1
24 Carnegie Mellon University 1
07 Columbia University -1
12 Cornell University -4
06 Dartmouth College 0
08 Duke University 1
23 Emory University 1
16 Georgetown University 2
01 Harvard University 0
17 Johns Hopkins University 0
05 Massachusetts Inst. of Technology 0
15 Northwestern University 0
03 Princeton University 0
14 Rice University 2
04 Stanford University 0
22 Tufts University -3
27 U. of North Carolina-Chapel Hill 0
20 University of California-Berkeley 0
26 University of California-Los Angeles 0
13 University of Chicago 0
21 University of Michigan-Ann Arbor 0
18 University of Notre Dame 0
10 University of Pennsylvania -1
19 University of Virginia 0
25 Washington University 1
02 Yale University 0</p>
<hr>
<p>A few notes:</p>
<p>*1) Total National Merit Scholars (less scholarships sponsored by university/college) per school</p>
<p>2) Each schools total National Merit Scholars were calculated per capita (as per USNWR 2005 incoming freshman class information)</p>
<p>3) There were less than 5 schools that freshman class numbers were not disclosed by USNWR - in which case a combination of school website and Wiki were used</p>
<p>4) The new weightings were as follows:*
30.0% - USNWR ('91-'07) Avg Ranking
30.0% - WSJ Feeder Ranking
30.0% - Revealed Preferences Ranking
10.0% - NMS Per Capita Ranking</p>
<p>5) A note of thanks to KK's initial list which I worked from (which listed those schools with at least 10 NMS) - though getting that list down to at least 5 was quite a task indeed and only necessary no thanks to Tufts (5) - thanks for introducing me to the fine institutions of Colorado School of Mines (5) and Franciscan University of Steubenville (5).</p>
<p>Just because Wall Street Journal doesn't mention Vanderbilt in its ranking, does not mean it should not be in top 25. Revealed Preference & USNews seem to use more legitimate criteria.</p>
<p>This is one of the most useful things I have ever seen on CC. Thanks.</p>
<p>If I read the table correctly, Duke and Columbia are actually tied for 7th?</p>
<p>Can you give us the correlations among these measures?</p>
<p>The three non-USNews measures are great in that they are "pure", an outcome measure (WSJ), an input measure (NMS), and a student preference measure. If one were to predict the outcome (WSJ as the dependent variable) using the other three for inputs (indepedent variables) what do you get for a multiple corellation? What are the weights of the independent variables in predicting WSJ rank?</p>
<p>Did you try playing with the weights to see how much difference it makes? </p>
<p>What does it look like if you throw out USNews altogether? It seems the ranking would be much the same. I hate to admit it, but this almost vindicates USNews.</p>
<p>Looking at the table it is striking that all 4 measures seem to generate very similar rankings, saying that students congregate at colleges where they find others similar to themselves on academic measures, and that they attend top prof schools at rates that match their entrance credentials.</p>
<p>The actual weighted averages appear to capture the scale of difference between closely ranked schools as well. </p>
<p>Can you preserve this someplace easier to find than a thread on CC?</p>
<p>Terrific job.</p>
<p>And because USNews ranks universities and LAC's separately, this excludes all LAC's. so if one's goal is to tout a favorite school this is no help. But for studying top universities overall it is fascinating.</p>
<p>afan,</p>
<p>thanks for the encouragement. i'll try to answer some of your questions as best as i can. </p>
<p>one quick answer - yes Columbia and Duke are indeed tied (didn't realize that) in the second "run" including the NMS per capita ranking. i'm glad someone is actually reading this!</p>
<p>I also really like those rankings and that's coming from someone who doesn't usually like rankings; I favor grouping universities into peer groups. </p>
<p>I think Georgetown, Rice, Notre Dame and maybe UVa are ranked too high, but otherwise, the ranking seems spot on. The fact that I only diagree with 4 spots in a ranking with 27 universities is amazing. </p>
<p>Well done!</p>
<p>WOW these are good rankings.</p>
<p>That list seems very accurate to me. The only surprising aspect to me is Georgetown and Northwestern above Cornell and Notre Dame so high.</p>
<p>I agree with the Gtown comment.</p>
<p>In my new adjusted rankings (which incorporated NMS) - you'll note that Cornell dropped 4 places:</p>
<p>this was due to three factors:</p>
<p>1) Cornell's poor showing in the newly introduced weighting: NMS per capita in which Cornell ranked a surprising lowly no. 42
2) Re-adjusted weightings which de-emphasized its USNWR avg weight by 10 percentage points (Cornell overall rank was no. 12 prior to the adjusted ranking)
3) The re-balancing of the weightings which gave its USNWR avg rank equal weight to the WSJ and Revealed Preference - both rankings in which Cornell ranked below its USNWR ranking average (WSJ, no. 25) and (RP, no. 15)</p>
<p>Again, not a perfect system as I believe Cornell to be slightly higher than Northwestern and superior to Georgetown, but it does vindicate a couple of things that I have always felt was flawed with the USNWR ranking: </p>
<p>1) Certain large research dominated institutions that rode largely on the superior coattails of its grad programs, namely, WUSTL, were ranked too high; and
2) Certain top-notch undergrad focused "LAC-like" programs such as Brown / Dartmouth were underrated.</p>