Are public universities hurt or helped by USNWR methodology?

Applying to College
1

<p>Some posters believe that the USNWR rankings are not reflective of the quality of state public universities. Others believe that it gives them too many advantages in the methodology. </p>

<p>The USNWR rankings are determined according to the following weights:</p>

<p>25% Peer Assessment</p>

<p>16% 6-Year Graduation Rate
4% Freshman Retention Rate
5% Graduation Rate Differential (between what % were expected to graduate in six years and how many actually did-expected grad rates determined by undisclosed calculation involving high school record and standardized test scores)</p>

<p>20% Faculty Resources (breakdown not disclosed for items measured-% of classes with fewer than 20 students, % of classes with more than 50 students, Faculty Salary (adjusted for regional cost of living differences), % of professors with highest degree in their field, Student-Faculty ratio, and % of faculty that are full time)</p>

<p>7.5% Standardized Test Scores
6% % of Students who are ranked in Top 10% in high school
1.5% Admittance Rate</p>

<p>10% Financial Resources (measures average spending per student at the school)</p>

<p>5% Alumni Giving Rate</p>

<p>What is your view?</p>

2

<p>Aren't state schools primarily supposed to make higher education available to as many students as possible for a reasonable price? That said, I think we should not take US News too seriously.</p>

3

<p>There are a couple of factors used by USNWR that I don't understand. I don't understand peer assessment. Who is doing the assessment and what do they really know about the quality of other colleges? I suspect this is mainly a prestige/reputation rating. I am not sure if State schools win or lose based on this factor. I also do not understand the importance of graduation and retention rates. Wouldn't low rates be consistent with a rigorous and demanding school? I suspect State schools lose on this one. They are often less selective and give more students a chance at a college education and fewer students are likely to graduate or make it past freshman year.</p>

4

<p>Regarding peer assessment...</p>

<p>I heard a story on NPR about ranking, and one university president opined that he can't properly assess more than a few schools, yet the USNWR survey asks him to assess tons of schools at once. He seemed to suggest the fallibility of such an exercise.</p>

<p>Of course, he may be an exception, but I get the feeling that a few more bigshots feel the same way.</p>

5

<p>edad, concerning the graduation and retention rates:</p>

<p>I think colleges should adapt the rigor of their classes to the students taking those classes, or they should only admit students who can handle the work. If something goes wrong here, usually the retention rate suffers. It also suffers when students use it as a back up school or if something (the facilities, the faculty, ...) is so horrible that the students don't want to stay there. The graduation rate is imo also a valuable meassure of the quality of students (ambitious students are more likely to graduate than slackers) and the support students receive.</p>

6

<p>There are several areas where the public school numbers often diverge widely from the numbers of similarly ranked private schools. Some work to their benefit (4 discussed below), some to their detriment (1 discussed below). Here are my thoughts:</p>

<p>Numbers that benefit Public Universities</p>

<ol>
<li> Peer Assessment scores (representing 25% of total USNWR rank)
PA scores seem to have a high correlation with research activity. As public schools perform comparatively more research, they are beneficiaries of this favoring. For example, consider the PA scores of a few private schools that have stronger student bodies than many research-oriented public school cousins.<br></li>
</ol>

<p>Top 40 USNWR Private Schools likely getting shafted by the interpretation of what PA is and the high weighting assigned to PA scores
Notre Dame (3.9)
USC (3.9)
Tufts (3.7)
Wake Forest (3.5)
Brandeis (3.6)
Lehigh (3.2)
Boston College (3.6)
NYU (3.8)
U Rochester (3.4)
Case Western (3.5)</p>

<p>Top 40 USNWR Public Schools PA Scores
UC Berkeley (4.7)
U Michigan (4.5)
U Virginia (4.3)
UCLA (4.3)
U North Carolina (4.2)
U Wisconsin (4.2)</p>

<p>Two Top 40 Public Schools that don’t get much of a bump from PA scores
W&M (3.8)
UC SD (3.8)</p>

<ol>
<li><p>6-Year Graduation Rate (representing 16% of total USNWR rank)
For the USNWR Top 40, the public schools compete very well with the top privates on this measure. But is this a proper comparison? Many people favor state universities because they often provide a cost advantage. However, what is often missed in this calculation is how long it takes to finish college. Many (most?) families will budget for 4 years of college. If 4-year graduation rates were considered rather than 6-year graduation rates, then public universities would compare very differently as public university students often struggle to get the classes they need to complete their requirements for graduation. This has an extension effect and increases the cost of attending the public university. This extended 6-year calculation lowers the bar for this measurement and state universities are the clear beneficiary. </p></li>
<li><p>% of Top 10% high school students (representing 6% of total USNWR rank)
Getting top 10% ranked students is frequently a top priority for state universities and is even a mandated requirement in some states. By contrast, private universities and take a more “holistic” approach to college applications. Consider the following Top 10% numbers for some Top 40 state universities that get a boost from this number.</p></li>
</ol>

<p>UC Berkeley (99%)
U Michigan (89%)
U Virginia (87%)
UCLA (98%)
UC SD (99%)</p>

<p>Consider some Top 40 USNWR private universities that have lower than you might expect numbers of Top 10% students.</p>

<p>Duke (88%)
U Chicago (79%)
Dartmouth (87%)
Cornell (81%)
Northwestern (82%)
Johns Hopkins (81%)
Rice (88%)
Vanderbilt (77%)
Notre Dame (86%)
Carnegie Mellon (71%)
Georgetown (86%)
USC (85%)
Tufts (80%)
Wake Forest (61%)
Brandeis (74%)
Lehigh (78%)
BC (75%)
NYU (68%)</p>

<ol>
<li> Admittance Rate (1.5% of total USNWR rank)
Many posters mistakenly believe that admittance rate is a critical factor in the calculation for USNWR ranks. It is not. Because it is not, some public universities are not penalized for this and many private universities get almost no ranking benefit from their low acceptance rate. I personally believe that the USNWR is about right or maybe slightly underweight, but either way this works to the benefit of the state universities.<br></li>
</ol>

<p>Some Top 40 USNWR public universities helped by the low weight assigned to acceptance rate</p>

<p>U Michigan (57%)
U Wisconsin (68%)
UC SD (44%)</p>

<p>Some Top 40 USNWR privates are also helped by this low weighting.</p>

<p>U Chicago (40%)
Johns Hopkins (35%)
Emory (37%)
Vanderbilt (35%)
Carnegie Mellon (39%)
Wake Forest (39%)
Brandeis (38%)
Lehigh (41%)
NYU (37%)</p>

<p>Numbers that hurt Public Universities</p>

<ol>
<li> Alumni Giving rate (5% of the total USNWR rank)
While some state universities engender tremendous loyalty from their students and alumni, nearly all of these public schools are negatively impacted by this calculation. The high numbers of state school students and alumni combined with the belief that their tax dollars (for IS students) are already footing the bill cause this number to be relatively useless in making comparisons with smaller, privately funded schools.<br></li>
</ol>

<p>Public schools in USNWR Top 40 that get shafted by the Alumni Giving numbers</p>

<p>UC Berkeley (14% of alumni give money, ranked 109th)
U Michigan (15%, 105th)
U Virginia (26%, 33rd) The impact to U Virginia is obviously less.
UCLA (15%, 105th)
U North Carolina (23%, 45th)
W&M (25%, 33rd) Same as U Virginia
U Wisconsin (14%, 109th)
UC SD (10%, 163rd)</p>

<p>On balance, I conclude that public universities are HELPED by the USNWR methodology as the positive factors, particularly the Peer Assessment score, are much more heavily weighted in the calculations. If PA is to be maintained as part of the rankings, as others have suggested, it might be useful (and better) to break out the PA scores separately and rank schools without this element.</p>

7

<p>Great analysis hawkette</p>

8

<p>In addition to hawkette's analysis...</p>

<p>'Faculty resources' may be detrimental to public schools, as the student:faculty ratio tends to be higher there, and the salary is probably somewhat lower.</p>

<p>The graduation rate difference can be hurtful, too, as the standards for 'expected graduation rate' tend to be a bit higher. Caltech is one of the privates who underperforms, and as Gerard Casper said, Caltech and other schools seem to have added "too much value."</p>

<p><a href="http://www.stanford.edu/dept/pres-provost/president/speeches/961206gcfallow.html%5B/url%5D"&gt;http://www.stanford.edu/dept/pres-provost/president/speeches/961206gcfallow.html&lt;/a&gt;&lt;/p>

<p>'Standardized test scores' too may have a negative impact (however small), as many state schools stress GPA and rank much more than test scores.</p>

<p>'Financial resources' is negative too, as state schools tend to have more students and a smaller endowment.</p>

<p>As for the % students ranked in the top 10%, state schools obviously benefit from this, but that isn't to say that having high percentages in this measure is bad. Privates may use a somewhat more holistic approach to admissions than most publics (minus schools like Berkeley), but Harvard--an appropriate example--is right behind top publics, with 95% students in the top 10%. Other privates, as you can see, aren't at that level. This is one of the measures that puzzles me in the variance among great schools.</p>

<p>Overall, I think US News has a bias against public schools; you'll notice that a public school will very rarely break the top 20. If you were to rank schools based on endowment, you'd see that it wouldn't be too different from US News' ranking.</p>

9

<p>kyledavid,
I have enjoyed reading your posts over the past several months and your comments on another current thread actually are the reason that I created this thread. We probably disagree on how the publics get treated by USNWR but I appreciate your thoughts and the California perspective that you bring. </p>

<p>As for your comments above, I think you are absolutely right about Faculty Resources and faculty-student ratios as this measure likely hurts public schools in the ranking process. However, unlike my position on PA, I don’t think you’d support eliminating or separating out this measure, would you? Same for SAT scores, right?</p>

<p>For class rank, I share your surprise that the top privates don’t have a higher % of Top 10% students and this number, probably better than any other, highlights the difference in mandate between public and private universities. Given that there are 340,000 Top 10% scorers finishing high school this year, you’d expect the USNWR Top 20 schools (all privates and representing 32,000 freshman entry places) to have a higher number than they have historically shown. I think that this clearly shows the “holistic” approach at work for private school admissions. </p>

<p>Financial Resources is probably a negative for public schools, but not because of their size as some of the largest endowments are at public schools (U Texas, UC system, U Michigan, Texas A&M, U Virginia, U Minnesota are all in top 25 for size). In my view, it is because they have large undergraduate AND graduate population. Truth be told, if these numbers broke out to show spending on undergraduates, the results would be even worse for the publics as the lion’s share of the money gets spent by the publics on their graduate programs. Frankly, for undergraduate students, public colleges probably are benefited from this measurement as currently constructed.</p>

<p>Finally, I am curious to know how you would tinker with the rankings methodology if it were up to you. My personal belief is that the rankings don’t do enough to tell students about the experience that they will have on campus (a ranking combining Faculty Resources, Selectivity, Financial Resources would provide the greatest insight) and too much on the brand name among academics as personified by PA.</p>

10

<p>Just one quick note:</p>

<p>UCSD, like UCLA, has no space between the UC and SD.</p>

<p>:)</p>

11

<p>hawkette,</p>

<p>The problem with much of your posts about USNews is that you consider the points chosen by USNews to all be important. </p>

<p>For many people, like myself, graduation and retention scores don't mean anything. I think if you were really to look at things, the reason people don't graduate in 4 years isn't because they "couldn't get into certain classes." Over the course of 8 semesters, you can get into a class you want. I, who go to a public school, have never not gotten into a class I wanted, when I wanted it. Some people don't take class seriously, lose track of credits, fail a class, take a semester or a year off, red shirt a year for sports, etc. All of these things are more managable at a public school because tuition is much less. 4 years of instate tuition at UVa, for example, is the same as 1 year tuition at Georgetown.</p>

<p>With that in mind, financial resources is another factor which hurts public schools. USNews does not take into account of economy of scale. For example, Michigan can build a $100mil science building that does the same thing a $70mil science building harvard would build...but Michigan is 3x the size of Harvard therefore hurting it.</p>

<p>Also, students in the top 10% doesn't favor publics. As I've mentioned in the NW/Michigan thread, Public schools tend to get students who are in the top 10% and with slightly lower SAT scores. SAT scores matter more than percent of students in the top 10%, so its a push.</p>

12

<p>jags861,
I think you misstate my position on the USNWR data. I believe that the data generally is a positive to have and to use as a first step in the college selection process. I don't normally in a X vs Y comparison get too involved in telling people how to interpret the data. Everyone has their own needs and I have no particular insight into which pieces of the data have the most value to the reader. However, I do believe that much of the data is good and provides some sense of perspective in making comparisons between and among colleges. </p>

<p>Re the specifics of your post, I have read many posters who attend prominent publics (UC Berkeley and others) who bemoan their inability to get the classes they need in order to finish on time. I don't know how widespread a problem this is and the exceptions that you list are certainly valid. Still, many (most?) people enter the college process with the intention of finishing in four years and if a school has a particularly good or bad record of their students doing this, then I think it should be noted and publicized. </p>

<p>I'm not sure I follow your comments on financial resources and Top 10%/SATs. Could you run that by me again?</p>

13

<p>hawkette,</p>

<p>Re: financial resources
Lets say you have two schools, Harvard and Michigan--and Harvard has 300 biology students and Michigan has 1000. Harvard builds a great new biology building, which costs $50 million, which suits the need of all 300 students and the entire biology department. Michigan then builds a great biology building that suits the needs of all 1000 students and the entire department, and the entire building costs $100 million. Because Harvards building cost half as much, but is going towards less than 1/3 the amount of students, Harvard counts as "spending more per student"--which is the basis for financial resources--even though in reality, Harvard didn't do anything above and beyond what Michigan did. So, the USNews methodology ignores economies of scale (the increasing amount you pay gives you increasing returns). Now of course, I don't think that Michigan spends as much as other top privates, but when USNews says that Michigan or UVa or Berkeley have crappy financial resources, its silly--they just are much larger and therefore can do comparatively "less" and still get the job done.</p>

<p>Re: Top 10%. I don't follow your logic on the top 10% favoring publics. Publics tend to have more students in the top 10%, and privates tend to have higher SATs. Using your logic, I could just say "using SAT scores favors private schools because they use a more holistic approach to admissions." So its a "push" in that sense.</p>

14

<p>
[quote]
3. % of Top 10% high school students (representing 6% of total USNWR rank)
Getting top 10% ranked students is frequently a top priority for state universities and is even a mandated requirement in some states. By contrast, private universities and take a more “holistic” approach to college applications. Consider the following Top 10% numbers for some Top 40 state universities that get a boost from this number.</p>

<p>UC Berkeley (99%)
U Michigan (89%)
U Virginia (87%)
UCLA (98%)
UC SD (99%)

[/quote]
</p>

<p>Publics use the top 10% as a means of increasing diversity w/o falling foul of laws barring race as a factor.</p>

<p>However, systems like UT have found this method less than ideal since a material % of "top 10%" students have been found to need remedial classes.</p>

15

<p>
[quote]
the lion’s share of the money gets spent by the publics on their graduate programs

[/quote]
</p>

<p>Can you quantify that a bit more? How much is a lion's share? I mean, I recognize that it's open to interpretation, but it would be helpful to know what this is in your view.</p>

16

<p>hoedown,</p>

<p>the problem is that people make a blanket statement like that, and it just gets tossed around as if it means something. I'm sure the lions share of money at harvard, yale, stanford, mit, caltech, etc is spent on graduate programs, too.</p>

17

<p>There are so many ways to count money in higher ed it becomes very difficult to compare numbers. For example medical school faculty can either be paid a salary from the school or get paid on their practice earnings. One would show up as school spending and the other would not as the hospital revenues and expenses are not usually included in college operating costs.</p>

18

<p>hoedown,
You are probably in a better position than me or most of us to know what are the rough levels of spending by colleges on their undergraduates vs their undergraduates. Rules of thumb that I have previously heard is that university spending for graduate students, on a per student basis, is well more than twice that for undergraduates. So the ratio of undergrads to grads would drive which group actually gets the majority, or "lion's share."</p>

19

<p>jags861,
Here is how USNWR defines their methodology for Financial Resources:</p>

<p>"Financial resources (10 percent). Generous per-student spending indicates that a college can offer a wide variety of programs and services. U.S. News measures the average spending per student on instruction, research, student services, and related educational expenditures in the 2004 and 2005 fiscal years. " </p>

<p>I'm not entirely sure what this means or how they are measuring it. Do you or anyone else understand what they are measuring and how?</p>

20

<p>no one knows what it means except for the editors. thats why you shouldn't put too much stock into making a declaration that each factor favors one type of school over the other. </p>

<p>if i had to guess, they just took the accounts payable total in each category and divided it by the number of students. This is all fine and dandy, except in the cases which i described above. </p>

<p>regardless, breaking down the USNews' flawed formula into 1% increments is incredibly banal--and defeats the purpose of rankings, imo.</p>