How to Improve USNWR Rankings? What to Drop/Add? What Criteria/Weightings to Change?

College Search & Selection
1

<p>Current USNWR rankings Criteria/Weightings:</p>

<p>1) Peer Assessment (Weighting: 25%) Measures top academics' opinions, presidents, provosts and deans of admissions to account for intangibles such as faculty dedication to teaching. Rating based on 5-point scale.</p>

<p>2) Retention ( 20%) This measure has two components; 6-yr graduation rate (80% of the retention score) and freshmen retention rate (20%).</p>

<p>3) Faculty Resources (20%) Uses six factors:</p>

<ul>
<li><p>Class Size. two components: proportion of classes fewer than 20 students (30% of Faculty Resources Score) and Proportion with 50 or more (10% of the FR Score)</p></li>
<li><p>Faculty Salary (35%) Measures average faculty pay, plus benefits, adjusted for regional differences in the cost of living.</p></li>
<li><p>Proportion of professors with the highest degree in their fields (15%)</p></li>
<li><p>Student-Faculty Ratio (5%)</p></li>
<li><p>Proportion of full-time faculty (5%)</p></li>
</ul>

<p>4) Student Selectivity (15%) Components: SAT Critical Reading/Math Scores or ACT (50%), proportion of enrolled freshmen who graduated in top 10% of their high school classes (40%) and acceptance rate (10%).</p>

<p>5) Financial Resources (10%) Measures average spending per student on instruction, research, student services, and related educational expenditures. Spending on sports, dorms, and hospitals doesn't count, only the part of a school's budget that goes towards educating students.</p>

<p>6) Graduation Rate Performance (5%) Measures the difference between a school's 6-year graduation rate and the predicted rate. Actual vs Predicted. The logic: if the actual graduation rate is higher than the predicted rate, the college is enhancing achievement.</p>

<p>7) Alumni Giving Rate (5%) The average percentage of living alumni giving to their school is an indirect measure of student satisfaction.</p>

<p>Methodology: To arrive at a school's rank, USNWR first calculated the weighted sum of its scores. The final scores were rescaled: The top school in each category was assigned a value of 100, and the other shools' weighted were calculated as a proportion of that top score. Final scores for each ranked school were rounded to the nearest whole number and ranked in descending order. Schools that receive the same rank are tied and are listed in alphabetical order.</p>

<p>Overall speaking, people see the benefits of the rankings and the pros far outweight the cons. What have been debatable, AND THIS IS WHAT THIS THREAD SPECIFICALLY ASKS FOR, include but are not limited to the following three areas:</p>

<p>1) The Creteria Used. For example, PA is not a true measure of academic quality, seems to reflect more of graduate program reputation rather than that of the undergrad. Faculty Resources, particularly class size and faculty pay components, favor rich private schools. Alumni Giving Rate almost has nothing to do with quality of education. Criteria such as expenditures per student, alumni giving rate and even acceptance rate encourage a school's efforts, budget, and resources, often times with non-academic motive but simply to boost up the school ratings.</p>

<p>2) Weightings. Some criteria are valid but need to be modified and weightings adjusted. Faculty Resources is a valid criterion but weightings need to be adjusted, particularly the faculty pay and class size components.
Retention is okay but why 20%. PA is good but why 25%. There are a number of weightings that are seen as a little bit too much, a little too less.</p>

<p>3) What Criteria to be Dropped or Added? Should we drop this? Delete that? Add this in? More Criteria that truly measure academic quality (the factors related to students academic output) need to be added to the survey. Very much open to dabate.</p>

<p>I'm sorry that the OP is lengthy but I feel a need to give sufficient briefing/background and make it clear as to what the thread asks for so that cc participants can make better, precise contributions to the topics discussed. Thank you very much for your participation and contribution/comments.</p>

2

<p>One idea I've seen advocated here (and elsewhere) is to provide these measures in an unweighted fashion. The magazine would not rank colleges but instead list them, along with these particular measures (avg. class size, graduate rate, SATs, etc). Students could "rank" colleges on the individual metrics they cared about and ignore the factors that were less important to their own sense of fit. The "weighting" would be individual, not decided by USNews.</p>

3

<p>do you mean to say that let usnwr provide a list of schools and a menu of measures with raw data such as ap, standardized test scores, avg. class size and so on. Based on these, students can pick and choose the criteria and assign the weightings as they see fit, a sort of personalized college rankings as students see appropriate. But how can that be done systematically and scientifically? Should there be, to start with, a few core criteria like pa measuring overall academic strengths, percentage of top 10% hs graduates, sat scores, then other measures (like academic strengths in majors, faculty pay, alumni giving rate, class size etc.) that students have the options to add or ignore?</p>

4

<p>Too bad everybody knows that won't sell issues, and people will just look elsewhere for a definitive list to tell them what to think.</p>

5

<p>I would remove: </p>

<ol>
<li><p>"Alumni giving rate" because that is the same thing as "School fundraiser mailing rate."</p></li>
<li><p>Faculty salary ranking (why is this a factor?)</p></li>
<li><p>Financial resources rating.... if we were to judge resulting educational quality by how much money is thrown at it, then we must also believe that the United States has by far the greatest healthcare system in the world.</p></li>
</ol>

<p>Add:</p>

<ol>
<li><p>Financial aid ranking instead of financial resources because the sheer amount of money thrown at student aid actually does correlate with financial aid quality.</p></li>
<li><p>post-graduation employment survey results</p></li>
</ol>

6

<p>how about weightings on certain criteria? how should they be adjusted, if any?</p>

7

<p>


</p>

<p>This is incorrect. Financial resources rating includes "expenditures per student category".</p>

<p>Description from USNWR:

[quote]
Expenditures per student. Financial resources are measured by the average spending per full-time-equivalent student on instruction, research, public service, academic support, student services, institutional support, and operations and maintenance (for public institutions only) during the 2005 and 2006 fiscal years. The number of full-time-equivalent students is equal to the number of full-time students plus one third of the number of part-time students. (Note: This includes both undergraduate and graduate students.) We first scaled the public service and research values by the percentage of full-time-equivalent undergraduate students attending the school. Next, we added in total instruction, academic support, student services, institutional support, and operations and maintenance (for public institutions only) and then divided by the number of full-time-equivalent students. After calculating this value, we applied a logarithmic transformation to the spending per full-time-equivalent student, prior to standardizing the value. This calculation process was done for all schools.

[/quote]
</p>

<p>This category makes no claim for separating undergraduate from graduate expenditures. It also favors schools with medical schools. Compare Berkeley's and UCLA's financial resources rank...Both are comparable UC universities in size and cost...UCLA has a medical school, Cal doesn't...UCLA has higher financial resources rank than Cal (UCLA=26, Cal=40).</p>

<p>If USNWR rankings are for undergraduate, why are they including medical school costs for financial resources?</p>

<p>Also this category rewards spending inefficiencies.</p>

8

<p>ucbchemegrad: i used the 2008 edition usnwr, the article titled "How We Calculate the Rankings", pages 76-79, as reference. here's the exact quote on page79.</p>

<p>"Financial Resources (10 percent) Generous per-student spending indicates that a college can offer a wide variety of programs and services. U.S. News measures financial resources by using the average spending per student on instruction, research, student services, and related educational expenditures in the 2005 and 2006 fiscal years. Spending on sports, dorms, and hospitals doesn't count, only the part of a school's that goes toward educating students."</p>

<p>i did delete the first intro-sentence and some words that do not change the meaning of the text as you may check the OP and the exact quote here.</p>

9

<p>^ spending for hospitals is different than spending for medical schools. I can't explain the difference in UCLA's and Cal's financial resources rank other than UCLA has a medical school.</p>

10

<p>if average per student expenditure includes both undergrad and grad students, it seems to me that the methodology, intended or not, would favor grad school program on several counts. 1} per student spending on grad programs like research funds, labs, facilities, etc. would be bigger compared to that for undergrad programs. therefore, average per-student spending would be higher for schools with bigger budgets for grads. 2) schools with more grads than undergrads would benefit from this methodology. 3) schools with medical programs also would benefit as per-student spending is bigger than most other majors.</p>

<p>so i guess there's a hidden benefit built into the Financial Resources rating for grad and medical schools, this benefit gets more magnified when the school has a greater number of grads vs undergrads.</p>

<p>it's an accepted fact that it takes more resources, financial and human, to educate grads when compared to undergrads.</p>

11

<p>


</p>

<p>I'd like to see some figures on this, if you have them. I'm not sure it works in such a straight-line fashion as you suggest. Grad students typically don't get their own labs or other facilities, and research funds available to them are usually quite limited. Faculty get those things, and it can be quite expensive in some fields; less so in others. A philosopher, for example, typically needs nothing more than an office, books, a computer, and maybe some limited travel funds. Sciences and engineering, on the other hand, are going to be quite costly, and I suspect the US News "expenditures per student" category is going to favor science- and engineering-heavy schools over humanioties-oriented schools. </p>

<p>But other things equal, wouldn't a school that gets to charge all (or a higher percentage) of these fixed faculty research costs to undergraduates as opposed to graduate students have an advantage, the way US News calculates it? It's going to cost a physics professor the same money to run his lab in either case, regardless of the mix of graduate and undergraduate students he's teaching. And perhaps more fundamentally, I'm not sure graduate students are the driver of educational costs, so much as they are the cheap labor that allows professors to carry out their costly high-end research.</p>

12

<p>bclintonk:</p>

<p>sorry i don't have data to back up the generally conceived notion that it costs for grads than for undergrads and i mean average per-student cost per credit hour, year or whatever, almost like a common sense. I could be totally wrong but it's pretty much generally accepted that way. i sure would try my best to search for figures. i know we should have the figures to back up the claim but since people generally feel that way, at least that's the way i thought, and that's i put a note so.</p>

<p>and don't get me wrong. i agree that if educational costs in terms of fixed assets and other necessary operational/educational budgets can be more or less spread out and allocated to both undergrads and grads to make the overall tution fees most affordable to all, that would seem appropriate. imo, this would benefit all concerned anyway.</p>

13

<p>one such study on public universities and colleges by Higher Education Coordinating Board on HECB 2001-02 Education Cost Study, Higher Education Expenditures for Instruction, State Support Plus Tuition Collections, seems to confirm the higher instructional cost at graduate level. Total instructional cost is calculated using both direct and indirect costs. Direct costs include salaries and benefits of instructional faculty and staff, supplies and equipment used for instruction. Indirect costs include admissions, registration, and student services not financed by student services and activities fees as well as proportional share of libraries, administration and facilities/maintenance.</p>

<p>there're many details involved that this post cannot cover. you may check detailed information on <a href="http://www.hecb.wa.gov/Docs/leg/ECS3-26-2003.pdf%5B/url%5D"&gt;www.hecb.wa.gov/Docs/leg/ECS3-26-2003.pdf&lt;/a> for accuracy.</p>

14

<p>
[quote]

  1. "Alumni giving rate" because that is the same thing as "School fundraiser mailing rate."

[/quote]
</p>

<p>I think that should stay for the exact reason stated in the first post. Just because they mail you a lot of things doesn't mean you have to donate more. Alumni won't give any money to institutions they're not satisfied with. If what you say is true, then I should just send you lots of letters asking for money, and you'd give it to me, just because I asked.</p>

<p>If US News doesn't rank the schools, then someone else will. Just look at all the threads on CC specifically about rankings.</p>

<p>The factor that I question is the proportion of full-time faculty. I've had many adjunct professors that were as good as, or better than, the full-time professors.</p>

15

<p>"Alumni won't give any money to institutions they're not satisfied with."</p>

<p>Even if this is true, that doesn't mean that all, most or many of the alumni who are satisfied with the school will give. I think alumni giving rate is a bad metric. It assumes that people with school pride will automatically give.</p>

16

<p>I think (if it is possible) they should do a salary comparrison 5 years out of school-excluding students who went on to higher education.</p>

17

<p>I think they should normalize all $$$$ figure statistics to to 1 particular place in the country, maybe a small town in Nebraska or Iowa? That way a $45,000 salary in NY and a $35,000 dollar salary in Witchata can be fairly compared as an even ~$40,000 value. Not doing this, in my opinion is the most dishonest element of the ranking system.</p>

18

<p>Here's one person's view on how to reshape the rankings and the weightings:</p>

<p>25% STUDENT BODY MEASUREMENTS for Incoming and Outgoing Students
8% Standardized Test Scores
4% Top 10% Ranks
2% Admittance Rate
8% Job Placement Statistics
3% Graduate School Statistics</p>

<p>20% FACULTY RESOURCES
4% % of classes with under 20 students
4% % of classes with over 50 students
4% Student/faculty ratio
2% % of classes taught by Tas
2% % of faculty with highest degree
2% Faculty salary
2% % of faculty that are full-time</p>

<p>20% FACULTY ASSESSMENT
8% Reputation among academics
6% Reputation among students
6% Reputation among employers</p>

<p>15.0% FINANCIAL RESOURCES
10% Money per student dedicated to research, student services, and related educational expenditures.
5% Endowment per capita (while it's not likely, ideally splitting out spending on undergraduate vs. graduate students)</p>

<p>12.5% GRADUATION/RETENTION MEASUREMENTS
2% Freshman Retention
2.5% 4-Year Graduation Rate
5% 6-Year Graduation Rate
3% Differential Measurement</p>

<p>5% FACILITIES</p>

<p>2.5% ALUMNI GIVING</p>

19

<p>hawkette:</p>

<p>i find your proposed idea on reshaping the rankings and weightings very interestiing but will you please elaborate a little more on student body measurements? are you referring to students demegraphic/geographic/ethnic groups diversity? if that's the case, what should be the appropriate mix? thank you very much.</p>

20

<p>
[quote]
Even if this is true, that doesn't mean that all, most or many of the alumni who are satisfied with the school will give. I think alumni giving rate is a bad metric. It assumes that people with school pride will automatically give.

[/quote]
</p>

<p>You're right in that those who are satisfied won't automatically give. <em>But</em> that is true across the board at all schools, and thus on average, it is a fair metric. It's not a good idea to look at the giving rate at each school individually, but it is a good idea to compare this statistic with its peers. In other words, it's poor as an absolute metric, but good as a relative metric.</p>