Just how ridiculous is Peer Assessment?

College Search & Selection
1

<p>Now that the 2008 US News rankings are available online, let's take a look at the top factor in the rankings-- Peer Assessment.</p>

<p>Here's what they have to say about Peer Assessment on the US News site:</p>

<p>"Peer assessment (weighting: 25 percent). The U.S. News ranking formula gives greatest weight to the opinions of those in a position to judge a school's undergraduate academic excellence. The peer assessment survey allows the top academics we consult-presidents, provosts, and deans of admissions-to account for intangibles such as faculty dedication to teaching. Each individual is asked to rate peer schools' academic programs on a scale from 1 (marginal) to 5 (distinguished). Those who don't know enough about a school to evaluate it fairly are asked to mark "don't know." Synovate, an opinion-research firm based near Chicago, collected the data; of the 4,269 people who were sent questionnaires, 51 percent responded."
<a href="http://www.usnews.com/usnews/edu/college/rankings/about/rank_3.php%5B/url%5D"&gt;http://www.usnews.com/usnews/edu/college/rankings/about/rank_3.php&lt;/a&gt;&lt;/p>

<p>Now, I don't believe I need to explain how ridiculous it is to have leaders of one schools grade another. Other people have explained this quite thoroughly. First of all, how much time do you think the provost of Michigan spends on analyzing the quality of Dartmouth? School leaders are not experts of other schools. And who knows who has an ax to grind? Is it impossible to believe that someone at Yale would give a bad grade to Harvard? It amazes me that people believe US News to be objective, when such a subjective criteria is used as its main component. </p>

<p>So let's now look at how some schools faired in the Peer Assessment:</p>

<p>UC—Berkeley received a 4.8, which ties it with Yale and places it above every school except Harvard, Princeton, Stanford and MIT. Now, Berkeley is a great school, but who really would put it ahead of CalTech, Penn, Duke, Chicago, Columbia, etc.?</p>

<p>Michigan received a 4.5. While also a good school, is it really in the same league (figuratively as well as literally) as Penn, which it tied? And should it be ahead of Duke, Dartmouth, and Wash U?</p>

<p>Wisconsin and Texas-- 4.1. Really now. 4.1? Ahead of Vandy, Rice, Emory, Georgetown, and Notre Dame? How can anyone take these rankings seriously?</p>

<p>Notre Dame gets a 3.9, which puts it BEHIND Illinois. And puts it just ahead of academic luminaries such as UC-Davis, Penn State, and Purdue. Yes, according to these "top academics," Notre Dame rates slightly better than Purdue.</p>

<p>Tufts, Brandeis, and Boston College all receive a 3.6, putting them behind all of the schools previously mentioned, as well as Minnesota, Ohio State, and Indiana. This has to be a joke.</p>

<p>Moving further down you will find Wake Forrest at 3.5. Quite respectable, seeing the tough competition it faced. Schools ahead of it at 3.6 include Arizona, Iowa, and Texas A&M. It ties Michigan State and Colorado.</p>

<p>On this list, you will also see Tulane tied with UMass, Arizona State, Missouri, and Oregon. Tulane. I don't care if New Orleans was hit by a hurricane, tornado, volcano, flooding, lighting, blizzard, bird flu, and earthquake all at the same time, Tulane would still be better than Arizona State.</p>

<p>Going down, we come upon my favorite, Lehigh at 3.2. Yes, the school that has rejected students accepted at UVa, Michigan, and NYU receives a 3.2. This places it among its peers Iowa State, Nebraska, SUNY Stony Brook, UC-Santa Cruz, and behind all schools previously mentioned.</p>

<p>TO ALL FUTURE COLLEGE STUDENTS:</p>

<p>Please, please, please, please, please do not use these rankings to help you choose your future university. As I have shown, they are extremely flawed and in no way indicative of a college's true value. The reason why people place such emphasis on these rankings is because no other publication ranks schools on a 1-100 scale. This is because other publications know that that system does not make sense for colleges. Ranking the best colleges is like ranking the best cities; they are simply to vast and complex to be assigned a single grade value. Use other guide books that talk to real students about colleges, such as Princeton Review and College Prowle r. If you are looking at public schools, your best bet is to just stay in-state and save yourself the money. Visit schools. Get a feel for life on campus. DO NOT use ridiculous "objective" rankings like these. Thank you.</p>

<ul>
<li>Pat</li>
</ul>

2

<p>in the words of John McClane: "welcome to the party, pal"</p>

3

<p>I don't think PA is ridiculous at all. It's the best way to measure prestige and how highly others think about a certain university.</p>

4

<p>Thank you, Michigan student.</p>

<p>I believe that the majority of us in the job market can tell you that while going to a "prestigious" school might get you a couple "wows" at a cocktail party, it is in no way indicative of a school's ability to prepare someone for a career. "Prestige" is very rarely taken into account in job interviews. Now don't get me wrong, some schools that are considered prestigious are also academically excellent, but there are many schools (most of which I have already mentioned) that have somehow gained prestige that has far outstripped their true academic level.</p>

<p>I know, the_prestige, it just drives me crazy how much stock people put into these rankings. There was someone on here wondering whether he should transfer to a school that was ranked ten spots higher. And that was his only reason for transferring!</p>

5

<p>It would be ok if US News listed the rankings with PA and without PA.</p>

6

<p>You're welcome. </p>

<p>I think PA is the most important factor in ranking colleges. Prestige is definitely important. It might not be to you, but for many people it is. The fact is, people who don't care about prestige, shouldn't care about the college rankings in the first place. However, if someone does care about how respected one college is compared to another, then the PA scores in these rankings do matter.</p>

7

<p>
[quote]
There was someone on here wondering whether he should transfer to a school that was ranked ten spots higher. And that was his only reason for transferring!

[/quote]
</p>

<p>Its actually 23 spots.</p>

8

<p>MightNick, that would be a somewhat valid argument if the assessment were done by employers, who actual see and hire graduates from different universities. But it is done by other academics. The Dean of Admissions at Stanford is expected to grade (on the very helpful 1-5 scale) the merit of Maryland's overall academics. Do you think he has ever even SEEN the University of Maryland? And even if he did, what makes him an expert on what constitutes a good college? How can this be justified?</p>

<p>And the post that I'm referring to was 10 spots.</p>

9

<p>"UC—Berkeley received a 4.8, which ties it with Yale and places it above every school except Harvard, Princeton, Stanford and MIT. Now, Berkeley is a great school, but who really would put it ahead of CalTech, Penn, Duke, Chicago, Columbia, etc.?"</p>

<p>Many people would, apparently. Only here on CC do people split hairs over Berkeley vs. Cornell, etc.</p>

10

<p>
[quote]
MightNick, that would be a somewhat valid argument if the assessment were done by employers. But it is done by other academics.

[/quote]
</p>

<p>And it should be. The deans and presidents of universities are some of the most important and knowledegable people in academics. If they can't provide an assesment of another university, then who can? Sure, you can argue that employers should provide an assesment of the quality of a university, but I don't see how that would be more reliable.</p>

11

<p>One problem though is that only 40 % or so of the university deans and presidents surveyed actually return the questionnaire sent to them by USN&WR. Also, schools that have a larger alumni body tend to be favored by peer assessment rankings.</p>

12

<p>"UC—Berkeley received a 4.8, which ties it with Yale and places it above every school except Harvard, Princeton, Stanford and MIT. Now, Berkeley is a great school, but who really would put it ahead of CalTech, Penn, Duke, Chicago, Columbia, etc.?"</p>

<p>Well, you are more of a number at Berkeley, but i honestly don't think you must know much about colleges. Of CalTech,Penn,Duke,Chicago,Columbia, and almost every other school you can name, who has the best business...
Wharton of Course...but then Cal. Out of the Caltech,Penn,Duke,Chicago, and Columbia, which one has the best undergrad engineering...arguably Caltech and Berkeley are pretty equal. What about the best humanities overall... sure Berkeley may make you a number, but almost every Berkeley humanities program ranks among the top 20 for sure, and many even in top 10. Of the schools you listed, the quality of berkeley humanities programs is arguably equal to columbia and Chicago, if not a little worse in some, and a little better in some.</p>

<p>Now this is peer assesment. Does not necessarily have to do with assesment of undergrad, but also of grad school. People asses the school for the whole thing, not just undergrad. </p>

<p>At graduate level, Berkeley easily offers one of the top 10 engineering programs, as well as sciences. What about law, huh? One of the top 10 best programs in the US, and maybe even the world.</p>

<p>Humanities, is where their grad school truly shines. Berkeley humanities admissions are among the lowest in the country. Why? Maybe it is because Berkeley offers top programs in almost every field. </p>

<p>So, while Berkeley undergrad is not the best, their graduate school programs rank all among the best in the world, and Berkeley, for that, easily holds a top spot as best overall graduate school in the world in breadth and depth of study. It is easily tied with schools like Harvard,MIT,Stanford,etc for the quality of its graduate programs in a 1st place matchup.</p>

<p>And don't forget about the nuclear bomb (the manhattan project was supervised and pioneered by Cal professors).</p>

<p>So as far as peer assesment goes, it deserves that high. It is easily one of the top 10, or 15 most well known universities in the world.</p>

13

<p>I also never understand....how do these provosts rate a place like Brown at the same time as they rate a place like Berkeley? One caters to undergrads and is exceptional at graduate placement and recruiting, the other has top grad schools. They can't be ranked according to the same metrics. PA should be divided. One half should be "research prowess" and the other should be "undergrad focus."</p>

<p>And BIGTWIX, an Ivy econ degree is going to place better in the job market at the elite firms than a Berkeley business degree anyway. This just further illustrates the advantage of these more undergrad focused schools.</p>

14

<p>BIGTWIX, the rankings are for undergrad.</p>

15

<p>
[quote]
I also never understand....how do these provosts rate a place like Brown at the same time as they rate a place like Berkeley? One caters to undergrads and is exceptional at graduate placement and recruiting, the other has top grad schools. They can't be ranked according to the same metrics. PA should be divided. One half should be "research prowess" and the other should be "undergrad focus."

[/quote]
</p>

<p>You are, in effect, saying that Brown and Berkeley shouldn't even be ranked in the same list. By the same logic, Dartmouth and Princeton shouldn't either -- they're very undergrad-focused.</p>

16

<p>Yeah I'm saying either PA should be split into two categories or it needs to be better explained to those scoring it. Dartmouth, Princeton, Notre Dame, Georgetown, etc have very different value propositions than Michigan or Berkeley.</p>

17

<p>How do you know it's not explained already to those scoring it? I'm sure they're aware of the differences between tiny Dartmouth and huge Michigan.</p>

18

<p>slipper1234</p>

<p>Peer assessment and actual "research prowess" are not necessarily 100 % correlated. For example, according to the Faculty Scholarly Productivity Index, Georgia Tech ranks top 10 in research output in two engineering areas only (biomedical and mechanical). Nevertheless, in the USN&WR purely PA rankings, I guess GaTech is listed top 10 (in many cases top 5) in virtually all engineering specialties. I suppose that has to do also with the fact that schools like Georgia Tech or Michigan Ann Arbor may have a large number of alumni who participate in the PA ranking survey.</p>

19

<p>Here are the complete PA rankings for National Universities:</p>

<p>USN Rank, School, PA Score</p>

<pre><code>4 schools tied for 1st

</code></pre>

<p>1 Princeton , 4.9
2 Harvard , 4.9
4 Stanford , 4.9
7 MIT , 4.9</p>

<pre><code>2 schools tied for 5th

</code></pre>

<p>3 Yale , 4.8
21 UC Berkeley , 4.8</p>

<pre><code>1 school at 7th

</code></pre>

<p>5 Caltech , 4.7</p>

<pre><code>4 schools tied for 8th

</code></pre>

<p>9 Columbia , 4.6
9 U Chicago , 4.6
12 Cornell , 4.6
14 Johns Hopkins , 4.6</p>

<pre><code>2 schools tied for 12th

</code></pre>

<p>5 U Penn , 4.5
25 U Michigan , 4.5</p>

<pre><code>2 schools tied for 14th

</code></pre>

<p>8 Duke , 4.4
14 Brown , 4.4</p>

<pre><code>3 schools tied for 17th

</code></pre>

<p>11 Dartmouth , 4.3
14 Northwestern , 4.3
23 U Virginia , 4.3</p>

<pre><code>3 schools tied for 20th

</code></pre>

<p>22 Carnegie Mellon , 4.2
25 UCLA , 4.2
28 U North Carolina , 4.2</p>

<pre><code>3 schools tied for 23rd

</code></pre>

<p>12 Wash U , 4.1
38 U Wisconsin , 4.1
44 U Texas , 4.1</p>

<pre><code>7 schools tied for 26th

</code></pre>

<p>17 Rice , 4
17 Emory , 4
19 Vanderbilt , 4
23 Georgetown , 4
27 USC , 4
35 Georgia Tech , 4
38 U Illinois , 4</p>

<pre><code>2 schools tied for 33rd

</code></pre>

<p>19 Notre Dame , 3.9
42 U Washington , 3.9</p>

<pre><code>5 schools tied for 35th

</code></pre>

<p>34 NYU , 3.8
38 UCSD , 3.8
42 UC Davis , 3.8
48 Penn State , 3.8
64 Purdue , 3.8</p>

<pre><code>4 schools tied for 40th

</code></pre>

<p>33 W&M , 3.7
57 Ohio State , 3.7
71 U Minnesota , 3.7
75 Indiana U , 3.7</p>

<pre><code>9 schools tied for 44th

</code></pre>

<p>28 Tufts , 3.6
31 Brandeis , 3.6
35 Boston Coll , 3.6
44 UC Irvine , 3.6
49 U Florida , 3.6
54 U Maryland , 3.6
62 Texas A&M , 3.6
64 U Iowa , 3.6
96 U Arizona , 3.6</p>

<pre><code>7 schools tield for 53rd

</code></pre>

<p>30 Wake Forest , 3.5
41 Case Western , 3.5
44 Rensselaer , 3.5
44 UC Santa Barbara , 3.5
59 U Georgia , 3.5
71 Michigan State , 3.5
79 U Colorado , 3.5</p>

<pre><code>8 schools tied for 60th

</code></pre>

<p>35 U Rochester , 3.4
50 Syracuse , 3.4
54 George Washington , 3.4
57 Boston University , 3.4
59 Rutgers , 3.4
59 U Pittsburgh , 3.4
71 Virginia Tech , 3.4
85 U Kansas , 3.4</p>

<pre><code>6 schools tied for 68th

</code></pre>

<p>50 Tulane , 3.3
67 Miami U (OH) , 3.3
91 U Missouri , 3.3
96 U Mass , 3.3
112 U Oregon , 3.3
124 Arizona State , 3.3</p>

<pre><code>8 schools tied for 74th

</code></pre>

<p>31 Lehigh , 3.2
52 U Miami , 3.2
64 U Connecticut , 3.2
75 Baylor , 3.2
79 UC Santa Cruz , 3.2
85 Iowa State , 3.2
91 U Nebraska , 3.2
96 SUNY-Stony Brook , 3.2</p>

<pre><code>10 schools tied for 82nd

</code></pre>

<p>54 Pepperdine , 3.1
67 Clemson , 3.1
67 SMU , 3.1
71 U Delaware , 3.1
75 Colorado Sch of Mines , 3.1
85 NC State , 3.1
96 Auburn , 3.1
96 U Tennessee , 3.1
96 UC Riverside , 3.1
118 U Buffalo , 3.1</p>

<pre><code>10 schools tied for 92nd

</code></pre>

<p>67 Fordham , 3
82 SUNY-Bing , 3
91 U Alabama , 3
96 U Vermont , 3
96 Northeastern , 3
108 U Oklahoma , 3
112 Ohio Univ , 3
112 Florida State , 3
118 Washington State , 3
122 U Kentucky , 3</p>

<pre><code>11 schools tied for 102nd

</code></pre>

<p>79 BYU , 2.9
82 St. Louis Univ , 2.9
82 Marquette , 2.9
85 American U , 2.9
96 Howard , 2.9
108 Drexel , 2.9
108 U New Hampshire , 2.9
112 U South Carolina , 2.9
112 Loyola U (Chi) , 2.9
124 Colorado State , 2.9
124 Kansas State , 2.9</p>

<pre><code>6 schools tied for 108th

</code></pre>

<p>52 Yeshiva , 2.8
62 Worcester , 2.8
91 Clark U , 2.8
96 Illinois Tech , 2.8
122 Catholic U , 2.8
124 U Arkansas , 2.8</p>

<pre><code>7 schools tied for 114th

</code></pre>

<p>75 Stevens Institute , 2.7
85 SUNY-Envi Sci/For , 2.7
85 U Denver , 2.7
107 U San Diego , 2.7
108 Texas Christian , 2.7
118 U Missouri (Rolla) , 2.7
124 Michigan Tech , 2.7</p>

<pre><code>4 schools tied for 121st

</code></pre>

<p>91 U Tulsa , 2.6
96 U of the Pacific , 2.6
124 NJ Tech , 2.6
124 Clarkson , 2.6</p>

<pre><code>2 schools tied for 123rd

</code></pre>

<p>112 U Dayton , 2.5
118 Samford , 2.5</p>

20

<p>Bruno,</p>

<p>I'm saying that PA is very ambiguous and unscientific and looking at the list above it makes little sense after the first few schools. Can one really argue that Ohio State is better than Tufts? Sure for grad school maybe but at the undergrad level its not even close.</p>