An Unbiased Way to Rank Schools: Some Changes in the Lineup

Applying to College
1

An interesting research paper on an unconventional method of ranking colleges from authors (Christopher Avery, Mark Glickman, Caroline Hoxby, Andrew Metrick) from the John F. Kennedy School of Government at Harvard University; the Department of Health Services at the Boston University School of Public Health, the Department of Economics at Harvard University, and the Department of Finance of The Wharton School at the University of Pennsylvania.

In particular, Caroline Hoxby from Harvard, has done many notable studies in this area.

Their methodology is based on student preference in where they choose to finally enroll among acceptances, in effect a series of head to head competitions.

As one of the authors (Metrick) states from the Wharton site discussion of this research paper (<a href=“http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104[/url]):”>http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104):</a> “When a student decides to enroll at one college among those that have admitted him, he effectively decides which college won in head-to-head competition. This model efficiently combines the information contained in thousands of these wins and losses, and produces a ranking that would be very difficult for a college to manipulate.”

But can colleges be judged based on who “wins” the competition for students? Metrick and his co-authors contend that they can. “First, students believe and act as though their peers matter,” the researchers say in the study. “This may be because peer quality affects the level of teaching that is offered. Alternatively, students may learn directly from their peers. It is reasonable for students to care about whether they are surrounded by peers with high college aptitude … Second, students - especially the high achieving students on whom we focus - are not ignorant about college quality. They gather information from publications, older siblings, friends who are attending college, college counselors and their own visits to colleges.”

The research article ranks the schools as follows (pages 26-28 of the PDF) (<a href=“http://knowledge.wharton.upenn.edu/papers/1287.pdf[/url]”>http://knowledge.wharton.upenn.edu/papers/1287.pdf</a>)

1 Harvard 2800
2 Yale 2738
3 Stanford 2694
4 Cal Tech 2632
5 MIT 2624
6 Princeton 2608
7 Brown 2433
8 Columbia 2392
9 Amherst 2363
10 Dartmouth 2357
11 Wellesley 2346
12 U Penn 2325
13 U Notre Dame 2279
14 Swarthmore 2270
15 Cornell 2236
16 Georgetown 2218
17 Rice 2214
18 Williams 2213
19 Duke 2209
20 U Virginia 2197
21 Northwestern 2136
22 Pomona 2132
23 Berkeley 2115
24 Georgia Tech 2115
25 Middlebury 2114
26 Wesleyan 2111
27 U Chicago 2104
28 Johns Hopkins 2096
29 USC 2072
30 Furman 2061
31 UNC 2045
32 Barnard 2034
33 Oberlin 2027
34 Carleton 2022
35 Vanderbilt 2016
36 UCLA 2012
37 Davidson 2010
38 U Texas 2008
39 NYU 1992
40 Tufts 1986
41 Washington & Lee 1983
42 U Michigan 1978
43 Vassar 1978
44 Grinnell 1977
45 U Illinois 1974
46 Carnegie Mellon 1957
47 U Maryland 1956
48 William & Mary 1954
49 Bowdoin 1953
50 Wake Forest 1940
51 Claremont 1936
52 Macalester 1926
53 Colgate 1925
54 Smith 1921
55 U Miami 1914
56 Haverford 1910
57 Mt Holyoke 1909
58 Connecticut College 1906
59 Bates 1903
60 Kenyon 1903
61 Emory 1888
62 Washington U 1887
63 Occidental 1883
64 Bryn Mawr 1871
65 SMU 1860
66 Lehigh 1858
67 Holy Cross 1839
68 Reed College 1837
69 RPI 1835
70 Florida State 1834
71 Colby 1820
72 UCSB 1818
73 GWU 1798
74 Fordham 1796
75 Sarah Lawrence 1788
76 Bucknell 1784
77 Catholic U 1784
78 U Colorado 1784
79 U Wisconsin 1780
80 Arizona State 1774
81 Wheaton (Il) 1750
82 Rose Hulman 1745
83 UCSC 1736
84 Boston U 1736
85 UCSD 1732
86 Tulane 1727
87 U Richmond 1714
88 CWRU 1704
89 Trinity College 1703
90 Colorado College 1698
91 Indiana U 1689
92 Penn State 1686
93 American U 1681
94 Hamilton 1674
95 U Washington 1629
96 U Rochester 1619
97 Lewis & Clark 1593
98 Wheaton (MA) 1564
99 Clark 1551
100 Skidmore 1548
101 Purdue 1525
102 Colorado State 1513
103 Syracuse 1506
104 Scripps 1479
105 Loyola U 1221

The list appears to be pretty intuitive. Some notable ‘drops’ from US news rankings are Duke, University of Chicago, Johns Hopkins, Washington University in St. Louis and University of Michigan.

An interesting finding is that regional preferences don’t differ much from the overall national one (p. 36 of the PDF). The exception appears to be Brigham Young which is #6 in the region that includes Utah).

The disussion of this paper from the Wharton website is linked below:

<a href=“http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104[/url]”>http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104</a>

2

<p>this is still a very biased and innaccurate representation of the country's best institutions. Duke, JHU and UChic. are at least as good as the top schools on this list, but since it considers "admissions" as opposed to actual "academics" those schools obviously lose out. What we should be trying to provide an algorithm for is what schools offer their students the best opportunities...who have the best faculty, who is willing to provide the most to their students to do whatever interests them. If we did find a way to quantify that, i am POSITIVE that those three in particular would be near the top</p>

3

<p>like i said in the other post, people dont generally prefer UCSC or UCSB over UCSD. still a very interesting paper though</p>

4

<p>maybe im missing something or read it wrong, but this rating system seems flawed. if they rank colleges based upon which college wins the most of these 'head to head competitions', thats just a rough approximation of the students' fairly unimformed impressions of a school. where do they get those impressions? probably a large portion come from college rankings. so basically this study ranks colleges based upon which has the best ranking...</p>

5

<p>agreed, it just doesn't make sense to rank schools based on their admissions performance</p>

6

<p>The study seemingly does a pretty good job of relative worth of why an individual may pick certain schools over others. Clearly prestige appears to be a very important factor to applicants (since the ranking seems to essentially follow prestige rankings and many academic discussions focus on why the rank of a particular school differs from US news). But looking at the exceptions does tell something of why certain students may prefer other schools. Utah has a high percentage of Mormons and thus students from Utah give a high preference to Brigham Young. An AP article (<a href="http://www.boston.com/news/nation/articles/2004/10/20/new_ranking_system_based_on_choice/%5B/url%5D"&gt;http://www.boston.com/news/nation/articles/2004/10/20/new_ranking_system_based_on_choice/&lt;/a&gt;) summarizes why certain other schools may do well:</p>

<p>"Wellesley's appeal as a women's college evidently helps it beat elite universities. Georgetown and Notre Dame score higher than they do in US News, probably because of their popularity with Roman Catholic students."</p>

<p>But the study is very interesting in that it basically summarizes students' choices of one school over another. Which is basically what all these forums that we participate in basically debate anyway.</p>

7

<p>agreed but that is not an accurate representation of the academic caliber of any particular institution but rather an evaluation of popularity trends</p>

8

<p>Yes, I was going to edit my post to say prestige/popularity. They go hand in hand. Popularity is based a large part on prestige and exclusiveness. As an analogy, if a tremendous find of Gold and/or diamonds were found making them as easy to get as sand, we may look for other presents to give our loved ones. If Harvard expanded its seats and took a much larger percentage of applicants, its exclusiveness and therefore its prestige would drop, even though the quality of education remained similar. Popularity effects prestige. The University of Chicago was arguably as good as any university in the country many years ago, but its popularity has fallen and with it, its exclusiveness and prestige. Brown University used to be as prestigious as Colgate. But Brown's popularity went up greatly after it was selected into the Ivy League over Colgate and consequently Brown's exclusiveness and prestige went up greatly.</p>

9

<p>there's a long thread about this exact same paper somewhere on this board...</p>

<p>EDIT: I found it: <a href="http://talk.collegeconfidential.com/showthread.php?t=1653&highlight=rank+economists%5B/url%5D"&gt;http://talk.collegeconfidential.com/showthread.php?t=1653&highlight=rank+economists&lt;/a&gt;&lt;/p>