An Unbiased Way to Rank Schools: Some Changes in the Lineup

College Search & Selection
1

<p>An interesting research paper on an unconventional method of ranking colleges from authors (Christopher Avery, Mark Glickman, Caroline Hoxby, Andrew Metrick) from the John F. Kennedy School of Government at Harvard University; the Department of Health Services at the Boston University School of Public Health, the Department of Economics at Harvard University, and the Department of Finance of The Wharton School at the University of Pennsylvania. </p>

<p>In particular, Caroline Hoxby from Harvard, has done many notable studies in this area. </p>

<p>Their methodology is based on student preference in where they choose to finally enroll among acceptances, in effect a series of head to head competitions. </p>

<p>As one of the authors (Metrick) states from the Wharton site discussion of this research paper (<a href="http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104%5B/url%5D):"&gt;http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104):&lt;/a> "When a student decides to enroll at one college among those that have admitted him, he effectively decides which college won in head-to-head competition. This model efficiently combines the information contained in thousands of these wins and losses, and produces a ranking that would be very difficult for a college to manipulate." </p>

<p>But can colleges be judged based on who "wins" the competition for students? Metrick and his co-authors contend that they can. "First, students believe and act as though their peers matter," the researchers say in the study. "This may be because peer quality affects the level of teaching that is offered. Alternatively, students may learn directly from their peers. It is reasonable for students to care about whether they are surrounded by peers with high college aptitude ... Second, students - especially the high achieving students on whom we focus - are not ignorant about college quality. They gather information from publications, older siblings, friends who are attending college, college counselors and their own visits to colleges." </p>

<p>The research article ranks the schools as follows (pages 26-28 of the PDF) (<a href="http://knowledge.wharton.upenn.edu/papers/1287.pdf%5B/url%5D"&gt;http://knowledge.wharton.upenn.edu/papers/1287.pdf&lt;/a&gt;) </p>

<p>1 Harvard 2800
2 Yale 2738
3 Stanford 2694
4 Cal Tech 2632
5 MIT 2624
6 Princeton 2608
7 Brown 2433
8 Columbia 2392
9 Amherst 2363
10 Dartmouth 2357
11 Wellesley 2346
12 U Penn 2325
13 U Notre Dame 2279
14 Swarthmore 2270
15 Cornell 2236
16 Georgetown 2218
17 Rice 2214
18 Williams 2213
19 Duke 2209
20 U Virginia 2197
21 Northwestern 2136
22 Pomona 2132
23 Berkeley 2115
24 Georgia Tech 2115
25 Middlebury 2114
26 Wesleyan 2111
27 U Chicago 2104
28 Johns Hopkins 2096
29 USC 2072
30 Furman 2061
31 UNC 2045
32 Barnard 2034
33 Oberlin 2027
34 Carleton 2022
35 Vanderbilt 2016
36 UCLA 2012
37 Davidson 2010
38 U Texas 2008
39 NYU 1992
40 Tufts 1986
41 Washington & Lee 1983
42 U Michigan 1978
43 Vassar 1978
44 Grinnell 1977
45 U Illinois 1974
46 Carnegie Mellon 1957
47 U Maryland 1956
48 William & Mary 1954
49 Bowdoin 1953
50 Wake Forest 1940
51 Claremont 1936
52 Macalester 1926
53 Colgate 1925
54 Smith 1921
55 U Miami 1914
56 Haverford 1910
57 Mt Holyoke 1909
58 Connecticut College 1906
59 Bates 1903
60 Kenyon 1903
61 Emory 1888
62 Washington U 1887
63 Occidental 1883
64 Bryn Mawr 1871
65 SMU 1860
66 Lehigh 1858
67 Holy Cross 1839
68 Reed College 1837
69 RPI 1835
70 Florida State 1834
71 Colby 1820
72 UCSB 1818
73 GWU 1798
74 Fordham 1796
75 Sarah Lawrence 1788
76 Bucknell 1784
77 Catholic U 1784
78 U Colorado 1784
79 U Wisconsin 1780
80 Arizona State 1774
81 Wheaton (Il) 1750
82 Rose Hulman 1745
83 UCSC 1736
84 Boston U 1736
85 UCSD 1732
86 Tulane 1727
87 U Richmond 1714
88 CWRU 1704
89 Trinity College 1703
90 Colorado College 1698
91 Indiana U 1689
92 Penn State 1686
93 American U 1681
94 Hamilton 1674
95 U Washington 1629
96 U Rochester 1619
97 Lewis & Clark 1593
98 Wheaton (MA) 1564
99 Clark 1551
100 Skidmore 1548
101 Purdue 1525
102 Colorado State 1513
103 Syracuse 1506
104 Scripps 1479
105 Loyola U 1221 </p>

<p>The list appears to be pretty intuitive. Some notable 'drops' from US news rankings are Duke, University of Chicago, Johns Hopkins, Washington University in St. Louis and University of Michigan. </p>

<p>An interesting finding is that regional preferences don't differ much from the overall national one (p. 36 of the PDF). The exception appears to be Brigham Young which is #6 in the region that includes Utah). </p>

<p>The disussion of this paper from the Wharton website is linked below: </p>

<p><a href="http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104%5B/url%5D"&gt;http://knowledge.wharton.upenn.edu/index.cfm?fa=viewArticle&id=1104&lt;/a&gt;&lt;/p>

2

<p>The same comment as in the other thread: this is still a very biased and innaccurate representation of the country's best institutions. Duke, JHU and UChic. are at least as good as the top schools on this list, but since it considers "admissions" as opposed to actual "academics" those schools obviously lose out. What we should be trying to provide an algorithm for is what schools offer their students the best opportunities...who have the best faculty, who is willing to provide the most to their students to do whatever interests them. If we did find a way to quantify that, i am POSITIVE that those three in particular would be near the top</p>

3

<p>very interesting. although in reality people dont generally prefer UCSB or UCSC over UCSD</p>

4

<p>What it doesn't account for is the <em>actual quality</em> of the schools. I know someone who goes to U of Chicago, and while it is certainly true many of the people there are depressed "Ivy Rejects", the acedemics are widely considered to be as tough as an Ivy, and the bigger downside is probably in its sort of lackluster social scene. The error in ranking schools solely based on prestige (which is what the list feeds off of - prestige in the minds of the students) is that some schools are only "fads" based off of an agressive marketing department and not the actual quality of education or undergrad experience. </p>

<p>Also, it can inflate the positions of some of the mid-public schools because the students didn't catch Ivy-itus. For instance, while Illinois, Maryland, etc. are good schools I would consider Wake Forest to have superior acedemics.</p>

5

<p>Yes, the HS kid is the ultimate judge of quality--why look at their music and fashion sense. Definitely!</p>

6

<p>^ That's exactly what I thought. =)</p>

7

<p>That ranking is excrement. Sorry.</p>

<p>Duke #19?
UCB #23?
Chicago #27?
Johns Hopkins #28?
Michigan #42?</p>

<p>Those 5 universities should all be ranked among the top 15 nationally. And lumping LACs with Research universities is a joke. In some ways, LACs destroy research universities...in other ways, research universities destroy LACs, and all of those "ways" impact undergraduate education equally.</p>

<p>The ranking above is a beauty contest/popularity contest. It has nothing to do with academics.</p>

8

<p>thank you, yay Wake.</p>

9

<p>This is a popularity contest perhaps, but the real question is "what makes these schools popular?" What this research paper shows is that the nation's smartest students tend to prefer schools w/ a combination of prestige and a more intimate undergrad atmosphere.</p>

10

<p>I think the ranking is perfect. This is the only unbiased way to rank schools, because STUDENTS are doing the ranking. The places that the smartest students tend to choose over others is a very important statistic. It shows which schools are seen as the best schools in the minds of STUDENTS. The opinions of the students are more important than the opinions of US News editors or British tabloid magazines. It's also the most democratic way to rank schools.</p>

11

<p>by no means perfect, but probably more reliable than us news</p>

12

<p>Nope, it's moronic (even moreso than ranking universities in the first place). You can't trust stuff like this. </p>

<p>If you could...all hail Princeton Law School...the third best law school in the nation--or so says public perception of rank...</p>

13

<p>Apparantly, Princeton chooses to admit people who fall just under the threshold for admission into Harvard, Stanford, Yale and MIT in order to ensure higher matriculation rates. This increases the APPARENT selectivity of Princeton but lowers its REAL selectivity. </p>

<p>Duke also practices bad admissions practices by encouraging mediocre people to apply just to lower the acceptance rate. The actual quality of the class is unchanged. </p>

<p>Check that article out. It's an interesting read.</p>

14

<p>Weedit, this research paper isn't a survey of public perception. It's based on the choices made by students who were admitted into the best universities in the nation. Seeing the collective decisions that students make is important. Again it shows that prestige and an intimate undergrad experience is very important - more so than say large research universities w/ "high ranking" departments & no intimacy or LACs w/ no prestige. </p>

<p>I'm not saying that students should base their decisions on this research. You should always visit the schools that appeal to you and pick the right college for your needs, but I think it's important to look at what the nation's top students prefer.</p>

15

<p>I think so too. To me, that is the best possible ranking system you can ever make. I will admit that ranking colleges isn't the best thing to do, but if you have to rank them, at least rank them according to the preferences of top students.</p>

16

<p>I think it is important that there exists a ranking system that can distinguish between statistics that can be manipulated (like those used for US News) and the preferences of top students.</p>

<p>Rooster, right you are about Princeton and Duke. </p>

<p>Princeton, in effect, practices a form of 'Tufts syndrome', rejecting some more qualified applicants because of competitive pressures from more selective colleges, eg Harvard and MIT.
From the report:
"consider Princeton admissions in Figure 1 (p. 6 of the PDF). At Princeton, the admissions probability rises to 20 percent at the 93 percentile (of SAT scores), then falls to 10 percent at the 98 percentile (precisely the region where competition is toughest), and then rises again for students with SAT scores in the top 2 percentiles."
"If a college is not practicing strategic admissions, then the probability that a student is admitted ought to rise monotonically in his or her merit (MIT is shown as an example). In contrast, a college that is strategic (Princeton is given as an example) will have non-monotonic admissions probabilities. A student's probability of admission will first rise in his or her merit and then fall as his or her merit moves into the range in which the strategic college faces stiff competition. In other words, the college will avoid admitting students in the range in which it is likely to lose in a matriculation tournament. Finally, if the student's merit is high enough, a strategic college will probably admit the student even if the competition will be stiff. This is because the prospective gains from enrolling a "star" will more than make up for the prospective losses from a higher admissions rate and lower matriculation rate." </p>

<p>As for Duke (p. 11 of the PDF):
"In addition, colleges can manipulate their admissions rate by encouraging applications from students who have little chance of actually gaining admission. A college can advertise less stringent criteria than it actually applies. By doing so, it encourages marginal students to apply, increases its number of applications, decreases its admissions rate, and raises its apparent desirability, even though its real desirability has not changed. For instance, this is how Toor (2000) described her job as an admissions officer at Duke University: "The job of admissions officers is to recruit, to boost application numbers. The more applications, the lower the admit rate, the higher the institutional ranking. Increasing application numbers is usually the No. 1 mandate of the recruiting season. Partly, that means trying to get the very best students to apply. But it also means trying to persuade those regular, old Bright Well-Rounded Kids (B.W.R.K.'s, in admissionese) to apply -- so that the college can reject them and bolster its selectivity rating." </p>

<p>A better example of what Duke does may be going on at Washington University in St. Louis. It's given as an example of the discrepancy with US News rankings (from the Harvard Crimson, (thecrimson.com/article.aspx?ref=503948):
"Washington University in St. Louis, for example, ranks 11th in U.S. News, but 62nd in the NBER study. Avery pointed to such discrepancies as a testament to the strength of the system he and his co-authors proposed. It shows that U.S. News is valuing something that is not being valued by students in the same way,” said Avery." </p>

<p>Wash U's very highly statistically qualified students but 62 preference ranking would seem to imply that applicants to Wash U when given the chance to choose another school would choose the other school. Therefore, the students going to Wash U are composed of many students rejected by other schools. So for Wash U's relatively low acceptance to be true, they must be accepting most of the very qualified students balanced by rejecting many underqualified students. This probably explains Wash U's disproportionate amount of mailers to applicants. </p>

<p>The Harvard Crimson goes on to say:
"The paper criticizes U.S. News and other rankings for having unduly influenced the college admissions process, characterizing the admission and matriculation rates used in the magazine to measure selectivity as “crude proxies.” “They induce colleges to engage in distorted conduct that decreases the colleges’ real selectivity while increasing the colleges’ apparent desirability,” the paper asserts. For example, a college may encourage “marginal” students to apply only to reject them, thus increasing the college’s apparent selectivity. “When the admission and matriculation rates are given a lot of weight in rankings, college admission officers feel obliged to play a lot of games in admissions,” Hoxby said." </p>

<p>But in the end I think manipulation has helped Duke (only 19th in the preference rankings) and now Wash U to rise in the US News rankings which will eventually pay off in being seen as increased quality and prestige. And eventually applicants will prefer these schools over other schools. </p>

<p>Other schools may not correlate with US News rankings for reasons other than admission manipulation. Brigham Young is given as an example of regional/religious preference in the report (p. 41 of the PDF). Utah has a high percentage of Mormons and thus students from Utah give a high preference to Brigham Young.
An AP article (boston.com/news/nation/articles/2004/10/20/new<em>ranking</em>system<em>based</em>on_choice/) summarizes why certain other schools may do well:
"Wellesley's appeal as a women's college evidently helps it beat elite universities. Georgetown and Notre Dame score higher than they do in US News, probably because of their popularity with Roman Catholic students." </p>

<p>I think these examples are true examples of 'self selective' schools. Students who apply to these schools are more likely to go to them in comparison to other seemingly better schools. I think this is a truer form of self-selectivity than other examples commonly given like University of Chicago (students who get in there appear, from the relatively poor rank of 27, to choose other schools if they get into both). I suspect the University of Virginia's relatively high ranking (#20) may be indicative of some form of self selection of the students who apply there.</p>

17

<p>i dont think student preference is an accurate measure of the caliber of an academic institution. rather, it is merely a reflection of perceived caliber derived from the very same rankings and we are therefor left in an inescapable vicious circularity. Rather, preference, admissions statistics and the like should be considered but only as a component to what realy maters - the overall academic value that any given school can offer. UChicago which you reference, does relatively poorly in these rankings as does Georgetown and Johns Hopkins, because their admissions statistics (which are directly related to preference because of yield) don't accurately represent the "value" of the institution. Anyone in the real world including academia recognizes the top flight education available at those institutions and likely would not draw great distinctions between them and the top 5 or so schools. We need to re-evaluate our criteria for an "elite" university or college...i don't know about you all, but i want opportunities, faculty, yes some prestige, and the resources to study whatever i want to study in whatever fashion. None of those things is reflected by any of the methods used in either this study or by the USNWR</p>

18

<p>If you think about it, wouldn't a lot of community colleges show up on this list? Almost everyone who applies to a CC gets in, and has an intention of going there.</p>

19

<p>Abrandel,</p>

<p>You are right, what is seen in this study are students current perceptions of these universities and to a degree their perceived caliber. The problem is that perceptions are unfortunately important in the real world. For whatever reason, when given a choice, students who get into both the University of Chicago and Harvard will choose Harvard. I would argue that this does affect the overall academic value of the U of C. A large part of ones education is through ones peers. If all you learned was what you learned in class, you wouldn't be too educated. As a kid, your parents were in effect your peers. In college, it's your roommates, friends, study partners etc. The fact that the very best and brightest when given the choice, choose Harvard over U of C, decrease the chance that this amazing student I could have interacted with, would be at the U of C. As far as learning from books, professors and graduate students, these opportunities still rightfully abound at U of C. ( And so far graduate school, I might choose the U of C over Harvard.) But for an undergraduate education, I want my peers to be the best they can be for my own education. (This is not to say that the interaction one can get from U of C students wont be amazing, its just that perhaps Harvard interactions may be a tad more).</p>

<p>GWColonial,</p>

<p>What you are referring to is yield. The yield at CCs and religious schools in general is high, in some cases much higher than Harvard's already amazing 80%. However in most cases, a student accepted at both a CC and Harvard will chose Harvard. Thus, though the yield at a CC will be very high, its rank in this poll probably would be rather low. Most CC applicants probably don't also apply to Harvard, thus there yield can be high, but their rank in this poll can be low.</p>

20

<p>I still think it's stupid because it doesn't take into account of what students are applying to.</p>

<p>EG: top top students often apply to all the ivies/stan ect and apply to schools like UVa/Umich/JHU as their safety. When they get into a "better" school the safeties obviously lose the head to head. This gives the aura of those high tier safties as being last-resorts. </p>

<p>Then you have schools like Notre Dame. Fierce loyalty (fans/followers/alumni) cause it to often be the highest reach some people apply to. This gives it the aura of being a first pick to it just by the nature of the school and who's applying. </p>

<p>Do you really think most students would choose ND over, say, Cornell?</p>

<p>Look at USC in the rankings. It's above a dozen schools that most people would choose hands-down before it, presumably for the same reasons.</p>

1 Like