Misleading rankings for colleges and graduate schools

College Search & Selection
1

<p>When I purchased the USNEWS rankings of Colleges and Graduate Schools for the first time I never really paid any attention to the ranking factors compiled to produce the “overall” ranking till I noticed that Berkeley was ranked as #20 on the OVERALL top Colleges list. I thought that this ranking is dubious at best as Berkeley is simply one of the very finest schools in the nation and I thought it should be ranked among the top 5 and thus almost instantly I started to doubt the credibility of USNEWS. </p>

<p>Surprisingly, it took me a long time till I finally realized that the USNEWS data could actually be 100% accurate, only the way this data is compiled is simply severely flawed. Specifically, the “academic strength” of a school is but one of 18 factors that determine USNEWS final (overall) score. The notion of a top college implies that it is strong academically so why the other factors come into play to determine the overall score of a top school? But what are these other factors anyway? Examples below.</p>

<p>Freshman retention rate:
The higher the better the overall score! So, if first years students at Berkeley find it very hard to perform and decide to leave then this means that Berkeley sucks and thus its overall ranking should be affected in a negative way! While at smaller schools, Princeton, for example, students are counseled and taken care of and thus are more inclined to stay at Princeton rather than leave out of frustration. Ok. I can see that this factor might be a good piece of information but I fail to see why it is incorporated in the “overall” score for TOP colleges????!!!! </p>

<p>Graduation and retention rank/predicted graduation rate/actual graduation rate:
These are 3 separate indicators although all 3 are basically saying the same thing, i.e. how long does it take a student to graduate. So in a large school like Berkeley, where students receive comparatively small attention, they tend to take longer to graduate and this reflects negatively on the “overall” ranking of Berkeley as a top school?!! Again, I do believe that these factors give us interesting information but they should never be incorporated in the final score!</p>

<p>Percentage of classes under 20/percentage of classes with 50 or more/student per faculty ratio:
All 3 indicators are again saying the same thing: how much attention would a student receive in class…. hardly a good enough reason to affect the overall score reflecting the strength of an academic institution! </p>

<p>So far all of the above (7 factors already) have really nothing to do with the academic prestige of a school and all of them are really nothing but indicators of how “pampered” a student would be at a certain school. </p>

<p>Alumni giving rank/average alumni giving rate:
Now, why do these two factors (both are basically the same) have anything to do with the strength of a school? Of course, money plays a pivotal role in the success of any institution but why does this have to be expressed in terms of alumni giving? Aren’t their numerous ways for a school to raise money? And what percentage does alumni giving contribute to a school’s annual budget anyway???!!!</p>

<p>Selectivity rank/SAT-ACT percentile/Freshmen in top 10% of high school class/acceptance rate
Finally, a few indicators that are related to a school’s academic competitiveness, two of which, however, are very misleading! Example:
Berkeley has selectivity rank/acceptance rate of 14/25% this makes it lower than Yale/Princeton/Upenn/Brown/Duke/Stanford among others. However, percentage of first year students at top 10% of HS class is 99% HIGHER than ANY other school! So, I don’t see why an academic institution accepting mostly excellent students could still be ranked lower in terms of selectivity?? Because of high acceptance rate? But acceptance rate could be a very misleading factor, below are two far more revealing examples.</p>

<p>Michigan—Ann Arbor has acceptance rate of 62% which is higher than schools like Jackson State Univ, NorthEASTERN univ, univ of Tennessee, univ of Delaware, univ of Arkansas, Kansas State Univ, San Diego State Univ, Alabama A&M Univ, Florida International Univ, Stevens Institute of Technology, univ of Pittsburgh, Univ of South Florida, Tennessee State Univ, Univ of Central Florida, Illinois Institute of Technology among many many other schools. Although at Michigan, the percentage of first year students at top 10% of HS class is 90% FAR MUCH HIGHER than all of the aforementioned schools, even higher than Dartmouth/Duke/Stanford/Columbia/Cornell So, how could the acceptance rate be so high? It is rather evident to see that there are 4 main factors that affect the acceptance rate: the number of applications, the size of the school and most importantly, the pool of applicants and the admission criteria. It is no coincidence that Michigan receives much smaller number of applications than many other schools for the simple fact that its tuition is extremely high for a public school that doesn’t pamper its students! In fact, even its in-state tuition is by far the highest of all state schools and its out-of-state tuition is in par with private schools like Stanford and Princeton! So it is no wonder why many applicants think twice about applying to a Public School that charges the same amount of money as a private one while scoring low at the “pampering factors” private schools usually excel at.</p>

<p>Stanford engineering Graduate school has an acceptance rate of 35%, which is higher than schools like Univ of Kentucky, Buffalo, Louisiana State, Univ of Pittsburgh, Univ of Cincinnati, Missouri State, Central Florida, Iowa, Iowa State, Michigan State, and many others. Of course, Stanford has one of the top 3 graduate engineering programs anywhere in the world and as such it attracts far more superior engineering applications than these schools, however, if we are to look naively at the acceptance rate as an indicator of the school’s selectivity then we are to conclude that Michigan State in East Lansing (11% acceptance) is far more competitive in terms of admission than Stanford!!! </p>

<p>A late example of a highly misleading ranking is the so-called TOP 500 World Universities. Coming from an engineering background I noticed that it ranks many schools I never ever heard of as top US schools. This time, I decided to look at the methodology of their ranking rather than jumping into conclusions and I noticed that it heavily favors schools that are strong in both the social and natural sciences (physics/biology etc). The authors even explicitly confess (in the FAQ section) that these rankings have nothing to do with professional schools like Medicine/Law/Engineering/Business etc. So this really explains why a school like Univ of Pittsburgh is ranked higher than Carnegie Mellon Univ!!!! </p>

<p>In the end I have to say that I am not angry at USNEWS or any other rankings anymore…. They provide accurate data, interpret them in their own way and present them to the reader as a so-called “overall” ranking that shows all the factors at play. It is the reader’s duty to interpret the data for himself.</p>

2

<p>Yeah I agree with you on alot of your reasoning...but clearly class size is important in quality of undergraduate academics. </p>

<p>I should start my own magazine, rank colleges in it, and then become famous for my wonderful college rankings (while no one reads my magazine...)</p>

3

<p>I agree with you in a lot of respects - I cannot understand what alumni giving or whether profs are full or part time have to do with whether a school is good or not. I mean, after all, would you prefer to have as a professor Maya Angelou part time, or Jill Smith full time? </p>

<p>But retention rate does indicate more than your interpretation, I think. You saw it only as a "sink or swim". Any school will have students who just don't cut it. But the retention rate reflects just as much on whether the school is as good as its ads. So if students just don't like the school, they transfer. Most schools in the top tier have between 90 - 97 % retention. The difference between 95 and 96% is pretty meaningless. But if you look at a school and its retention rate is 75%, how would you view the school? You can't honestly think that 25% of the freshman class flunked out - you think quite naturally that the school is just not a school that a whole 25% of the freshman class wants to stay in.</p>

4

<p>I guess you are right regarding the freshman retention thing... </p>

<p>Anyway all I was trying to say is that only the academic strength/HS GPA/SAT-ACT scores should be used to define top schools. In my view, the quality of the professors and students is what defines the greatness of a university/program. And misleading factors that can result in potentially wrong perception of a school's quality could be shown but NOT incorporated in the overall score. I guess the Stanford vs Michigan State Graduate Engineering example explains what I mean very clearly.</p>

<p>In any case, what really is important is that students/parents should learn how to read rankings and draw their very own personalized conclusions, and avoid naive interpretations.</p>

5

<p>well, I think you're absolutely right. And there is also the "peer assessment", which is interesting. Not sure it's 100% accurate either, but what the peers think must count for something. And to prove your point even more, if you re-rank only based on peer assessment - which you can do with the online version of USNWR - then Berkeley comes out either 5th or 6th. So Berkeley's own peers agree with you that it should be higher in the rankings!</p>

6

<p>you raise the most important point when it comes to looking at rankings:
take them with a grain of salt!</p>

<p>They are a good guide, but within any given range of schools, one edging another in any one factor is not important: it's the overall fit that is most important. Two things to remember are:
It's easy to get a poor education from a great school, and, one will do better when he enjoys what he is doing.
These points mean that you shouldn't pick schools based primarily on rankings.</p>

<p>Also, just some anecdotal evidence: I attend cornell, and this is looking within the university. The engineering college at cornell is one of the best in the country, and the top in the ivy league. It has the highest average SAT of any of cornell's seven colleges. However, it also has the highest acceptance rate, somewhere in the mid to high 30s. The reason for this is usually attributed to an elite, self-selective applicant pool, but what one stands to learn from this example is that there are often (hidden) outside factors affecting a statistic. This is the primary reason why rankings should be used as a guide to selecting a college and not as a factbook on superiority and inferiority.</p>

7

<p>I agree 100% with what you had to say.</p>

<p>However, in my opinion the best rankings ever are the National Research Council (NRC) rankings. This is the only “official” ranking (sponsored by the US government--8000 faculty members across the country contributed to the NRC report.)</p>

<p>Its only problem though is that it comes out every decade. Last time these ranking were out was in 1995, we should expect a new NRC ranking in 2006. The other disadvantage is that it doesn’t rank important professional programs like Medicine/Law/Business (it does rank Engineering programs though). Below are links for the 1995 NRC rankings.</p>

<p><a href="http://www.stat.tamu.edu/%7Ejnewton/nrc_rankings/nrc41.html#area19%5B/url%5D"&gt;http://www.stat.tamu.edu/~jnewton/nrc_rankings/nrc41.html#area19&lt;/a>
<a href="http://www.nap.edu/readingroom/books/researchdoc/%5B/url%5D"&gt;http://www.nap.edu/readingroom/books/researchdoc/&lt;/a&gt;&lt;/p>

8

<p>
[quote]
Selectivity rank/SAT-ACT percentile/Freshmen in top 10% of high school class/acceptance rate
Finally, a few indicators that are related to a school’s academic competitiveness, two of which, however, are very misleading! Example:
Berkeley has selectivity rank/acceptance rate of 14/25% this makes it lower than Yale/Princeton/Upenn/Brown/Duke/Stanford among others. However, percentage of first year students at top 10% of HS class is 99% HIGHER than ANY other school! So, I don’t see why an academic institution accepting mostly excellent students could still be ranked lower in terms of selectivity?? Because of high acceptance rate? But acceptance rate could be a very misleading factor, below are two far more revealing examples.

[/quote]
</p>

<p>Yet I think that's the whole point of why USNews uses several factors to determine selectivity. Essentially, USNews takes a mixture of selectivity + SAT + high school class rank and together comes up with an aggregate selectivity value. None of those 3 values, by itself, is a good indication of selectivity. However, put them all together and you have a better picture. A perfect picture? Of course not. But better. </p>

<p>
[quote]
Stanford engineering Graduate school has an acceptance rate of 35%, which is higher than schools like Univ of Kentucky, Buffalo, Louisiana State, Univ of Pittsburgh, Univ of Cincinnati, Missouri State, Central Florida, Iowa, Iowa State, Michigan State, and many others. Of course, Stanford has one of the top 3 graduate engineering programs anywhere in the world and as such it attracts far more superior engineering applications than these schools, however, if we are to look naively at the acceptance rate as an indicator of the school’s selectivity then we are to conclude that Michigan State in East Lansing (11% acceptance) is far more competitive in terms of admission than Stanford!!!

[/quote]
</p>

<p>And again, USNews is not looking naively at only the selectivity rate. You have to figure in other factors like GMAT scores. That's the whole point. That's what USNews tries to do. </p>

<p>
[quote]
then Berkeley comes out either 5th or 6th. So Berkeley's own peers agree with you that it should be higher in the rankings!

[/quote]
</p>

<p>Ah, but school quality is a lot more than just having your peers think highly of you. Let's face it. Selectivity matters. Academic resources matter. Freshman retention rate matters. All of these things matter. </p>

<p>The biggest problem I have is not with USNews per se, it's with schools whose administrations think they can sit around and do nothing to improve the school, and just live solely on their reputation. The rankings, if nothing else, force the administration's feet to the fire and hold them accountable to improving the school. The accountability may be flawed, but it's better than no accountability at all. </p>

<p>Look, the point is that I feel that USNews has successfully addressed the quirkiness of the notion of selectivity in a fairlly reasonable way. Is the ranking perfect? Of course not. But it makes a fairly decent attempt at trying to capture all the more important variables. </p>

<p>Like I said, I think the salutary effect of the rankings is that it forces schools to constantly improve. The REAL problem is not with flawed rankings, it's with lazy school administrators who don't really try to make their school better. . Let's face it. Without rankings, I think we all know that a lot of school administrators would choose to do nothing at all.</p>

9

<p>Doesn't it account for grad/PhD programs, whereas US News looks at undergrad?</p>

10

<p>randomwalk - If more people actually looked at how these rankings are developed they would certainly lose their "power." People want to be told which college is number ONE, and if their college is in the top TEN, or top TWENTY FIVE. Few people really want to be bothered with how the list was created. </p>

<p>An even more extreme example was the ranking of the nation's best high schools. Everyone took the list and said, "Oh look, here is the BEST high school in the country, and here are the 10 best, and oh, we are the blankety blankest best high school in the country." But if you took the time to read the methodology of how the list was developed you would see it is a measure of how many AP tests were taken at the schools.</p>

11

<p>The USNWR has a separate ranking for graduate programs. You only looked at the undergrduate ranking.</p>

12

<p>"So far all of the above (7 factors already) have really nothing to do with the academic prestige of a school and all of them are really nothing but indicators of how “pampered” a student would be at a certain school."</p>

<p>Parents care about these things. And they're the ones footing the college bill in most cases. And they're the ones who more likely make up USNEWS subscription base.</p>

13

<p>The rankings are useful for creating an initial list and for looking at certain facts and figures but I would be reluctant to use them in place of my own list and ranking based upon my own desires.</p>

<p>I just read that UC Davis and US N&WR agreed that after UCD reported the percentage of faculty holding terminal degrees as 64 instead of the correct 98% there was a drop in their standing. Under "faculty resources" UCD fell from 84 to 215th. Their overall ranking dropped from 42 to 48. Whoops! </p>

<p>Here are some other rankings:</p>

<p><a href="http://ed.sjtu.edu.cn/rank/2005/ARWU2005_Top100.htm%5B/url%5D"&gt;http://ed.sjtu.edu.cn/rank/2005/ARWU2005_Top100.htm&lt;/a&gt;&lt;/p>

<p><a href="http://www.washingtonmonthly.com/features/2005/0509.collegeguide.html%5B/url%5D"&gt;http://www.washingtonmonthly.com/features/2005/0509.collegeguide.html&lt;/a&gt;&lt;/p>

14

<p>Well, the graduate rakings are flawed, too. I think they are only based on peer opionion. Shouldn't other factors be considered? How about placement of graduates into tenure track positions, for instance?</p>

15

<p>thanks for analyzing rankings. each ranking has its own flaws, but it is flawed only because the reader doesn't agree with the values placed in the rankings. obviously, everyone has different values in measuring what they think is a good university, and even perceived prestige is relative to one's background and probably geography. none of these rankings are really wrong or rediculous if their numbers add up. it just shows that certain colleges have hidden strengths elsewhere that we often overlook; we can change the criteria and weighting all we want and we'd come out with many different rankings. whether or not we misinterpret the rankings is our problem, really. </p>

<p>i just wish more people on CC did and weren't so obsessed with rankings and "prestige" and didn't take US News as if it were the bible. it's ironic how smart these kids are, yet they don't even understand what the rankings actually measure.</p>

16

<p>Well, the graduate rakings are flawed, too. I think they are only based on peer opionion. Shouldn't other factors be considered? How about placement of graduates into tenure track positions, for instance?</p>

<p>There is so much info out there that it diffuses what you are looking for.
We found the best route to chose our own criteria- then find the sources that gave us data to make our own rankings
While D school was tops in colleges that have a high percentage of graduates recieve their Phd- that wasn't really what we were interested in.
More relevant was total loans # of students after 4 years- class size/teacher-student ratio and freshmen living on campus.
Someone else may be equally concerned with # of trips to the Rose Bowl or how many course sections in israeli-palestinian relations.</p>

<p>US news aside from their "rankings"has some good info but so do many other sources</p>

17

<p>NJRes -- my point exactly!</p>

18

<p>Greenshirt -- I understand all of this information is helpful but they shouldn't identify the academic strength of a school!</p>

<p>For example, if I were a parent, I wouldn't be enthusiastic about applying to Michigan--Ann Arbor while they charge "A LOT" of money for out-of-state tuition when they don't give the same amount of attention to their undergrad students the way private schools do. I mean why pay private school tuition while Michigan fails at giving private school care of student. So yes, I'm happy that USNEWS provide this info but I'm not happy about the way they use it... that's all.</p>

19

<p>We use rankings the same way we use brand names for choosing soups,or pasta sauce; many people just assume the quality will be what they want by the brand name, others will then proceed to read and compare the labels. There might be something to picking a major brand but there might be a lesser known brand with just the right amount of olives and mushrooms for you.</p>