Building a Better Model for College UNDERGRAD Rankings

College Search & Selection
1

<p>You want to build an accurate and usable ranking of UNDERGRADUATE education in America. What inputs do you consider most important and what weighting would you give to them? </p>

<p>Based on my own experience and judgments and suggestions drawn from reading CC for over a year, here is my initial suggestion to you (and to USNWR if anyone there ever reads CC):</p>

<p>30% STUDENT BODY MEASUREMENTS for Incoming & Outgoing Students
8% Standardized Test Scores
4% Top 10% Ranks
2% Admittance Rate
10% Job Placement Statistics (% with full-time jobs, starting salary, etc)
4% Graduate School Placement Statistics
2% Average Debt of Graduates</p>

<p>20% FACULTY RESOURCES
5% % of classes with under 20 students
5% % of classes with over 50 students
4% Student/faculty ratio
2% % of classes taught by TAs
1% % of faculty with highest degree
1% Faculty salary
2% % of faculty that are full-time</p>

<p>15% FACULTY ASSESSMENT
6% Reputation among academics
3% Reputation among students
3% Reputation among alumni
3% Reputation among employers</p>

<p>10% FINANCIAL RESOURCES
7% Money per student dedicated to research, student services, and related educational expenditures.
3% Endowment per capita (combined for undergraduate and graduate)</p>

<p>10% GRADUATTION/RETENTION MEASUREMENTS
1% Freshman Retention
3% 4-Year Graduation Rate
3% 6-Year Graduation Rate
3% Differential Measurement</p>

<p>10% COST OF ATTENDANCE</p>

<p>5% ALUMNI GIVING (exempt public universities)</p>

2

<p>-Why include 4-year graduation rate? It doesn't measure anything. Many students stay in school 5 years to double major and schools surely shouldn't be punished for having students that choose to do that. That's why the 6 year grad rate is currently used.
-Why would you exempt public university from alumni giving?</p>

3

<p>I feel that placement after graduation is HUGE since it does a school no good if they are extremely strong but their grads go on to do nothing (even though this happens much less frequently as one goes higher on the ladder).</p>

4

<p>adf8,
Schools already measure how students from 4-year and 5-year programs graduate. The statistics generally refer to 1.5x the length of their program and this would be continued. The students participating in the 5-year programs would be counted in a longer duration pool. The schools would NOT be punished in the statistical calculation for students that are participating in programs longer than 4-years. </p>

<p>As for my reasoning, schools currently have the ability to measure 4-year graduation rates and do so in their Common Data Sets. The thinking is that college costs a lot these days and the schools should bear some responsibility for getting its students graduated on time. A 4-year measurement (for students on the 4-year plan) would reward those schools who have better institutional attentiveness and resources dedicated to the on-time graduation of their students. Graduation beyond the expected program tenure (eg, 4 years) is an oft-missed calculation in adding up the cost of college attendance. </p>

<p>As for alumni giving, three thoughts. One, as states contribute some portion of the money via taxes, many attendees consider this as a form of alumni giving. Second, there is not the history at most public colleges of this activity (again, because of the tax history). Third, the size of the state schools is typically much larger and these schools are disadvantaged by this percentage calculation.</p>

5

<p>Although I think grad school placement and job placement would be good things to include in theory, I think in practice it would be a little difficult to get the numbers you actually want to include.</p>

<p>You couldn't just include placement statistics as they are (30% of graduates go on to graduate/professional school, 60% go on to full-time jobs), because that doesn't tell you want the students wanted to do in the first place. Perhaps a lot more of them wanted to go to graduate school. Perhaps some of those who took jobs had to take jobs they didn't like or weren't well-prepared for.</p>

<p>You would want to include the percentages of students who got the outcome they wanted. Those numbers are not generally collected.</p>

6

<p>molliebatmit,
I think nearly everyone agrees that postgraduate outcomes are important. I agree that it is difficult, but I really think that this information would be of enormous value in the college search process. Do you have any suggestions on how to measure this? </p>

<p>BTW, I have enjoyed your musings/guidance on grad school matters.</p>

7

<p>If you are ranking the quality of undergraduate education, job placement statistics and grad school placement statistics are mostly irrelevant. They depend on many factors beyond the control of academic quality. Job placement and starting salary implies a certain value system (materialistic). And, why weight job placement higher than grad school placement? Students who go directly into the workforce after graduation are probably less well educated than students who go on to grad school or professional school.</p>

<p>Cost of attendance? How does that factor in? The higher the cost the higher the rank? Or vice versa? Net cost after grant aid or published cost? Cost of attendance is irrelevant to educational quality.</p>

<p>The list in post #1 is what I would call a "conceptual" ranking formula. It includes things that are arguably related to undergraduate academic quality. But, you don't need to know all those things to come up with a "computational" ranking formula because factors associated with quality education are interrelated. If you know SAT scores, for example, you know 90% of everything there is to know about the quality of undergraduate education ("computationally").</p>

<p>You could probably boil the ranking formula down to 3 or 4 factors.</p>

8

<p>
[quote]
4% Student/faculty ratio

[/quote]
</p>

<p>I would take this out. It's very misleading. I also don't agree with the cost of attendence factor. This is overall quality. not best value.</p>

9

<p>Collegehelp,
Thanks for your comments. You pose a very good question- should job placement or grad school placement be a part of how a school ranks? I believe that it should. </p>

<p>Many students (in fact, I would contend the vast majority of students) would benefit from knowing how effective graduates of a certain college are at getting jobs and/or landing places in graduate study programs. This is part of the natural progression of life and frequently is a MAJOR factor in a student’s selection of a college. </p>

<p>Your point is well-taken on the value system connoted by a measure influenced by wage scales, but postgraduate income is an important consideration for a large portion of college graduates. As a consumer (student or family), I would definitely want to know more about the cost-benefit relationship of attending X college. Given the criticality of this information, I would definitely lean to including such information in a ranking system. I also believe that disclosure of such information would clearly reveal how much smoke and how fire actually exists to support the arguments of many elite universities about the value of their brands. </p>

<p>As for the job placement vs grad school placement weightings, this is meant to reflect the relative differences in numbers of students pursuing these postgraduate paths. Frankly, I consider this ratio as appropriate only to the universe of top universities (Top 50?) as I believe that graduate school study becomes less common for graduates of the less selective universities. But a very high percentage of students at all schools are looking to get a job after college. If anything, I would increase the weighting of job placement vs grad school placement. </p>

<p>Xeneise,
Can you explain more about why you feel that student/faculty ratio is misleading? Would you reallocate this % to the class size measures? </p>

<p>As for cost, I would argue that this is an important element of the value proposition that a college offers. Any college could increase its cost significantly and offer higher quality in other areas. But the student/family would have to weigh if the higher cost is worth it. Conversely, a college could offer a lower cost, but not provide sufficient resources to effectively carry out their responsibilities.</p>

10

<p>Honestly, in my opinion, UNDERGRADUATE educations can't be precisely ranked.
It's not about which University has this, this and that, but it's about which university has the best learning environment for an individual student.</p>

<p>Now, Universities on the whole can and should be ranked in terms of research produced and such, but not undergrad colleges.</p>

11

<p>Alright, hawkette. Now give us your top 100.</p>

12

<p>I dont think cost of attendance should be a factor.
ur rating the quality of the education, not how much its worth.
how much financial aid is awarded could be a factor.</p>

13

<p>I don't see "How much they learn" on there.</p>

14

<p>s snack,
I don't understand why you feel that universities can be ranked, but UNDERGRAD cannot. Nor can I understand why you favor research driven rankings (which are mostly about grad student activity) when 75-80% or more of all college students are outside of the technical fields that are involved in the research. </p>

<p>As far as philosophy for the learning environment, I think I share your sentiments. IMO, a great undergraduate college experience is determined by:
1. Strength of study body
2. Size of the classroom
3. Quality of the faculty (and not decided based on research output)
4. Ability and willingness of the institution to spend money to support undergraduate students and faculty</p>

<p>At least 3 of these 4 measures can be quantitatively measured.</p>

<p>Pat2323,
Give me the data that I am looking for as outlined in the opening post and I believe that I can make a pretty good list.</p>

<p>weenie,
Can you elaborate on what you'd like to measure and how to do it? Do you think that college GPA is a relevant statistic and reflects what you learn? Could a student survey of the faculty be a part of that? How about employer perceptions of how well prepared the students are to tackle postgraduate employment? Could the postgraduate record for job placement/grad school placement partly reflect how much you learned?</p>

15

<p>
[quote]
Nor can I understand why you favor research driven rankings (which are mostly about grad student activity) when 75-80% or more of all college students are outside of the technical fields that are involved in the research.

[/quote]

I think s snack was saying that while it isn't possible to rank undergrad education, it is possible to rank universities by research output. I don't think he/she is saying one should rank undergrad education by research output, only that ranking research quality is easier/more precise than ranking undergrad education.</p>

16

<p>weenie-
I think you are right. How much students learn is the key, but so difficult to assess.. I would add that the true measure of the college experience would subtract how much students knew before college from how much they knoe after college...hypothetically.</p>

17

<p>Haha Hawkette, I don't have the time for that. I just figured since you came up with the formula you could give us the rankings.</p>

18

<p>Even if you managed to draw up a ranking using your proposed methodology, it would be immediately attacked by eveyone who thought their pet college was not in the right spot. </p>

<p>And I agree with collegehelp on the inclusion of a weighted factor for starting salaries.....this would so skew the rankings toward the universities that send large numbers of graduates to Wall Street that it would immediately create an unfair advantage that has nothing to do with measuring educational quality overall.</p>

19

<p>You cannot come up with a single ranking, for reasons people have already noted. For someone who wants to go out and get a job, then immediate employment prospects would be a critical factor in choosing a college. For someone headed for grad school, this would be irrelevant. </p>

<p>How about a series of rankings, each of which addresses a particular outcome goal or interest?</p>

<p>One might have rankings for people worried about whether they can be admitted
Most selective (MS) overall
MS for those interested in technical majors
MS for those interested in conventional liberal arts
MS for those interested in professional majors (with sub rankings for nursing, business,...)</p>

<p>Ranking for those interested in grad school
Highest proportion of students entering doctoral programs (with subranking by field)
Highest proportion of students completing doctoral programs (with subranking by field)
Academic quality of the graduate schools attended, derived by weighting the grad schools according to their evaluation in the NRC study (with subranking by field)</p>

<p>Ranking for those interested in professional school
Highest proportion of students entering professional programs (with subranking by type of professional school)
Academic quality of the professional schools attended, derived by weighting the prof schools according to some valid study, preferably not USNews (with subranking by field)</p>

<p>Ranking for those interested in employment
Percent of grads not headed to further education or other experiences (Peace Corps, etc) with full time jobs by graduation
Statistics on starting income (subranking by field)</p>

<p>Ranking for those who need financial aid
Mean debt for graduates
Availability of full need-based scholarships (excluding loans)
Availability of merit scholarships</p>

<p>Ranking for those primarily interested in quality of education. Almost impossible to do, since this again means different things to different people. For quantitative purposes one might try
Mean MCAT, LSAT, GRE scores of students, corrected for the class rank at each college of those who take the test and their SAT scores from high school. This would make it possible to adjust for academic and test taking ability of students when they enter-which will account for most of the variance in grad/professional school entrance exams- and for the tendency of lower ranked students to be less likely to try to apply to certain professional schools. People in the bottom quarter at top colleges are less likely to apply to med school. People outside of the top quarter at less prestigious colleges are equally less likely to apply to medical school.</p>

<p>Then correct the above by adjusting for input factors:
SAT score distributions
Parental education distributions
Parental income distributions (the higher the parental income of the students, and the higher the parental education, the more likely the students are to go to grad or professional school, the higher grades they get in college, and the more money they are likely to earn in the future. To separate these effects from the "treatment effect" of the college, correct the above outcome measures for academic and family background measures)</p>

<p>You will get dozens of rankings, many of which will be correlated to varying degrees. This is right. Since students vary in their goals, the "best" college will depend on what they are looking for.</p>

<p>The other things listed in USNews and other such rankings (SAT scores as a direct indicator, graduation rate differential, student faculty ratio, class size) are proxies for "quality of education". If one measures the outcome variables, then the quality of education, to the extent that it affects outcomes, will differentiate among colleges.</p>

20

<p>jazzymom,
I expect the attacks. Heck, I even want the attacks. I'm trying to understand how others think about colleges. I came to CC over a year ago and it has been an education reading a lot of posts (pun intended). Parents, college administrators, students (high school and college) have made some incredibly insightful comments. My proposed "rankings methodology" was an attempt to synthesize many of their comments with my own perspective. </p>

<p>Your comments about not using starting salaries is a good example of this. I think you make a valid point about how Wall Street pay scales would impact any measurement of undergraduate pay. However, I don't automatically conclude that the entire exercise is not of use. What you can hope to make upon graduation can be a big factor for many students looking for a school. Maybe the answer is you break the numbers out by % going to a certain industry and average salary for that industry. Maybe someone else will have a good idea on how to approach this. Or maybe you're right and we should just junk the idea. </p>

<p>My hope here is to spur some suggestions on how to improve the overall understanding that a student can get of a school when doing a college search. I think my proposed methodology is a starting point for doing just that as it incorporates aspects of the college decision making process (student outcomes, cost, input from students, alumni and employers) that are important, but not yet included in the broad surveys like USNWR. As for the other factors that are now included, the weightings that I suggest are my own conclusions about what I see as appropriate weights.</p>