US NEWS Rankings: What Would They Look Like Without Peer Assessment Score?

College Search & Selection
121

<p>Lafalum84,</p>

<p>My fault for skimming. :wink: Yes, that certainly addressed it. I think we’d all agree a balance of input and output metrics with a reduction of the subjective opinions would be a huge step in the right direction.</p>

<p>At the moment, I feel the data from payscale.com, Bloomberg.com and money.com are more useful than USNews to families making decisions that will impact the rest of the children’s lives. </p>

<p>skiblack, </p>

<p>You assume if a child goes to a top school, they’ll get the “most prestigious jobs with the best pay possible.” That’s actually not true at all. Many of the “top” schools not only aren’t top in pay, they aren’t top in percentage of graduates finding jobs. That’s the dirty little secret. </p>

<p>Again, USNews dares not include a single metric that measures the success of the graduate. If they did, their rankings would be immediately scrambled and unrecognizable with what they put out today.</p>

<p>Check out these links:
[Harvey</a> Mudd College - Colleges with the highest-paid graduates - CNNMoney](<a href=“http://money.cnn.com/gallery/real_estate/2013/09/12/highest-paid-graduates/index.html]Harvey”>Harvey Mudd College - Colleges with the highest-paid graduates - CNNMoney)
[Full</a> List of Schools - PayScale College Salary Report 2013-14](<a href=“http://www.payscale.com/college-salary-report-2014/full-list-of-schools]Full”>http://www.payscale.com/college-salary-report-2014/full-list-of-schools)</p>

<p>Look how far down the lists you need to go before you include all the Ivies, much less what many of us would consider “top” schools. :-/</p>

<p>By the way, I’m not suggesting anyone should use these sites as their only resource. A blending of information needs to occur and it should only be used to create a list of interesting schools. After that, IMHO, visits and interviews should dominate the decision process.</p>

<p>kollegekid1,</p>

<p>The salary info you want is in the links I listed.</p>

122

<p>

</p>

<p>Interestingly, the latest PayScale poll includes a “happiness” metric (“%with high job meaning”), consisting of the percentage of people who responded affirmatively to the question, “The work I do makes society a better place”, or something to that effect. And, high salaries don’t seem to positively correlate with that sentiment:
[Full</a> List of Schools - PayScale College Salary Report 2013-14](<a href=“http://www.payscale.com/college-salary-report-2014/full-list-of-schools]Full”>http://www.payscale.com/college-salary-report-2014/full-list-of-schools)</p>

123

<p>The problem with using either the CNN rating or PayScale is that they suffer from selection bias. Of course a school heavy in engineering will beat out one heavier in the arts. Engineering on average simply pays more. What would be more instructive would be to compare pay by college major. Is a physics major who attended Harvey Mudd making more money five years out than one who attended Princeton? What about an English major? It could still be that the HM English major makes more than the Princetonian because the HM curriculum encourages tech skills which translate well to employability, but without a finer breakdown than CNN and PayScale provide it’s hard to know.</p>

124

<p>^^
[Best</a> Schools for English Majors - 2013 - 2014 College Salary Report](<a href=“http://www.payscale.com/college-salary-report-2014/best-schools-by-major/english]Best”>http://www.payscale.com/college-salary-report-2014/best-schools-by-major/english)</p>

125

<p>OK, so George Mason University graduates have the highest average mid-career salaries for English majors. What does that tell us? Keep in mind that payscale only records data self-reported by alumni with terminal bachelors degrees. </p>

<p>From 2007-2011, 5 George Mason University alumni earned PhDs in English.
In the same period, 15 George Washington University alumni earned PhDs in English.
18 Johns Hopkins University alumni earned PhDs in English.
23 Georgetown University alumni earned PhDs in English.
GMU enrolls ~21K undergraduates.
GWU enrolls ~11K undergraduates.
Georgetown enrolls ~8K undergraduates.
JHU enrolls ~5K undergraduates.</p>

<p>Which is the better outcome-based indicator of an English department’s strength? The average alumni mid-career salaries of terminal BA recipients? Or the per-capita number of alumni doctorates in English?</p>

<p>Note also that GWU, Georgetown, and JHU rank much higher than JMU for earnings averaged across all majors. I doubt that GMU has discovered any secret formula to jack up the earnings of its English majors in particular. I mean, maybe they do have some fabulous DC-area internship program for liberal arts majors. More likely, I suspect we’re seeing sparse data sets skewed by self-reporting outliers.</p>

<p>Anyway, there is no certainty that an excellent English department can make you happy or rich (or that an average one will make you unhappy and poor). If we’re going to measure outcomes, let’s focus on outcomes tied to what colleges actually exist to do. In most cases, the primary mission is to create and share knowledge (not personal wealth, social justice, or world peace.)</p>

126

<p>I thought I’d dust off this old tread I started about the wisdom of the Peer Assessment rating. It shows that the “subjective” Peer Assessment ratings are predictable from hard data.</p>

<p><a href=“http://talk.collegeconfidential.com/college-search-selection/721414-wisdom-us-news-peer-assessment-rating.html?highlight=peer+assessment[/url]”>http://talk.collegeconfidential.com/college-search-selection/721414-wisdom-us-news-peer-assessment-rating.html?highlight=peer+assessment&lt;/a&gt;&lt;/p&gt;

127

<p>tk21769, That’s always where the payscale argument goes. :-/</p>

<p>I don’t think anyone is saying any ranking system is perfect. Far from it. But Payscale offers a compelling counterpoint to the traditionally accepted ranking system.</p>

<p>That being said. Your argument is a little weak. Out of hundreds or thousands of graduates, you are pointing to a pitiful few PhDs. The best indicator of a department’s strength is the success of the bulk of it’s graduates, which would point to the hundreds or thousands, not the cherry picked tens.</p>

128

<p>collegehelp,</p>

<p>It seems all too obvious that much of what you are calling “hard data about college quality” would be a direct result of ranking.</p>

<p>If a college had an incredible USNews ranking, would it not seem obvious to you that many students would apply there, so the admissions percentage would be low?</p>

<p>Since the college has their pick of students, would it not also be obvious the college could cherry pick the highest GPA and Standardized test applicants?</p>

<p>Since the very best students were cherry picked as freshmen, would it not stand to reason they would also have a high graduation rate?</p>

<p>Seems like a waste of your math skills. Your analysis was an exercise in circular reasoning.</p>

<p>Also, I’d argue none of what you list represents “hard data about college quality.” Some of it (very little of it) could infer quality, like the sum expended on research and endowment per student, but that is far from a direct indicator of quality. </p>

<p>The only thing that can measure the quality of an education is the success of the educated. And considering the conflicting results from salary based ranking systems, too many of the schools with high peer assessment scores are appallingly bad, especially considering the freshmen they had to start with. </p>

<p>If you want to apply your math skills, do a statistical analysis of that.</p>

129

<p>One thing is for sure, top law schools do not respect a berkeley education the same way they do with the top privates as some on here are suggesting:</p>

<p><a href=“https://career.berkeley.edu/law/lawstats.stm#school[/url]”>https://career.berkeley.edu/law/lawstats.stm#school&lt;/a&gt;&lt;/p&gt;

<p>Across the board, the LSAT and GPA averages of admits are higher than the average admit at the law school’s normal applicant pool.</p>

130

<p>^ yeah I know right. I have pointed this out on several occasions after doing significant research that Private school students with lower GPAs get into top grad schools than those fropm say top publics.</p>

<p>This is not because the graduate schools favor particular undergrads, just that undergrads from privates are smarter so they tend to have higher test scores on average and possible much more going on for them in activities and awards.</p>

131

<p>^Not even so, students from top privates can have lower LSATs AND lower GPAs and still get into better law schools than Berkeley</p>

132

<p>Payscale looks like a good quantitative tool until you look into their methodology. </p>

<p>-Their sample sizes are small, and consist solely of alumni "who respond to the survey. "I checked the ratings for my alma mata. They had data on only 82 graduates, despite the fact that the school graduates over 5 times this number of students in a single year. For Harvey Mudd, the number one school, PayScale used data from only 116 graduates. </p>

<p>-PayScale looks only at graduates with terminal bachelor’s degree. This penalized schools with a high percentage of students who go on to attain MDs, JDs, MBAs, PhDs or other advanced degrees. Because these students tend to be the ones with the most drive and highest grade and scores their exclusion dampens the ratings of certain schools, while rewarding, for instance, the service academies.</p>

<p>-It doesn’t include international student, or those working abroad, so schools with high populations of these students will have skewed results.</p>

<p>-It also excludes the self-employed and contract-based employees. There’s bound to be a huge gap in salary between alumni whose businesses are barely limping along and those who have established highly successful businesses. I can see why PayScale would want to exclude such data, as one or two successful entrepreneurs could wildly skew the data, but it’s still an issue.</p>

<p>[Methodology</a> and FAQ - 2013 - 2014 College Salary Report](<a href=“http://www.payscale.com/college-salary-report-2014/methodology#Data%20Set]Methodology”>http://www.payscale.com/college-salary-report-2014/methodology#Data%20Set)</p>

133

<p>^ A couple of other problems with Payscale:</p>

<ul>
<li><p>It’s not adjusted for major. A very large fraction of the top schools are engineering schools or engineering-heavy schools. At the top-ranked school, Harvey Mudd, 61% of the degrees awarded in 2012 were in engineering or computer science. #2 is the Naval Academy. #3 is Caltech, where 56% of the degrees awarded in 2012 were in engineering or computer science. At the top-ranked school in the Midwest, Rose-Hulman, 80.5% of the degrees awarded were in engineering or computer science. So all Payscale is ultimately telling us is that if you’re never going to go beyond a bachelor’s degree, engineering and computer science are more lucrative than most other majors–which we already knew.</p></li>
<li><p>It’s not adjusted for regional differences in cost-of-living which contribute to regional differences in (nominal) wage and salary scales. No surprise, then, that almost all of the top schools (17 of the top 18) are in the Northeast or California, high cost-of-living regions. </p></li>
<li><p>The exclusions of alums with advanced degrees is not a trivial problem. At many top schools half or more of the graduates go on to graduate/professional schools. At a few elite LACs it’s as high as 80%. At such schools, Payscale is reporting on a relatively small and unrepresentative minority of the alums.</p></li>
</ul>

134

<p>

</p>

<p>Success at what, exactly? If we want to make that argument using an outcome-based metric, one challenge is to demonstrate that the success of a school’s graduates, in business or any other non-academic undertaking, is largely attributable to the quality of specific undergraduate schooling. That could be pretty hard to do if we’re talking about arts & science degrees. </p>

<p>It’s not that a good liberal arts education doesn’t cultivate skills that are transferable to non-academic pursuits. However, average post-graduate earnings aren’t necessarily the best indicator. The challenge goes beyond any flaws in the payscale approach.</p>

135

<p>Payscale is even flimsier than I thought. My daughter’s LAC did quite well on Payscale with a median mid-career salary of $115 K, good for 17th place overall. Intrigued, I took a closer look at the “data.” Turns out that’s based on only 36 salaries . . . at ALL levels of experience. And when you go to the city-by-city breakdowns, the medians are lower than that for all cities except Seattle, where 2 salaries had a median of $127 K. I can only conclude that the median mid-career salary must have been based on those two salaries in Seattle and a tiny handful of others, such a small N that the “data” are virtually meaningless.</p>

136

<p>bclintonk, a sample size of 36 is as good as it is going to get. Payscale is a joke. Starting salary surveys are partially valid, but mid-career surveys are completely useless.</p>

137

<p>Reputation matters. Keep the peer assessment.</p>

138

<p>

</p>

<p>Those numbers often cover periods as long as 5 and 6 years following graduation. Potential MBAs, for example, are actually encouraged to enter the job market first.</p>

<p>As for the broader criticism that single degree graduates aren’t representative of certain college’s alumni, perhaps that’s true. But, it’s certainly representative of most Americans, the vast majority of whom seem to muddle through life without a graduate degree.</p>

139

<p>If we can gather an equally valid slice of each school’s single degree graduates, fine. But, sampling problems aside, do we really believe that if anthropology majors at school A earn more money after graduation than anthropology majors at school B, it means school A has a better anthropology department? </p>

<p>The correlation may even be inverse. Alumni of schools that best motivate and prepare anthropology majors may be more inclined to take low-paying museum or teaching jobs.</p>

<p>The best outcome-based measure of department strength would be one that measures continued success in applying what that department actually teaches. PhD production is such a measure. Colleges with the highest rates in biology (e.g. Carleton, Reed, Swarthmore) generate 60-80 life science PhDs each over a 5-year period. That appears to be as much as 30% or more of all biology majors. Berkeley and Michigan generate hundreds of biology PhDs each over the same period. What the NSF counts (in contrast to what payscale counts) is virtually 100% of all earned doctorates. </p>

<p>For single degree liberal arts graduates, I don’t think any such metric currently exists. Peace Corps participation?</p>

140

<p>All the criticism of the payscale.com ranking is valid. It’s not perfect. Not by a long shot.</p>

<p>But I’ve also found very similar results when investigating schools for my son. After making appointments with the career placement offices and asking, in particular, to discuss actual placement stats (not the survey reports), again, the results are askew from what USNews would have you assume.</p>

<p>Advice for parents helping their kids:
Schools keep two sets of employment records. One is for publication. In a word, it is crap. It is survey based and therefore has several issues. They all do it only because… well… they all do it and they need to be compete with the crap data from all the other institutions.</p>

<p>If you make an appointment with the career development offices and tell them exactly what you’d like to discuss, most colleges (In my experience) will accommodate you. You’ll be able to get a much better feel for how easy/hard it is for the graduates to find jobs. How long they’ve been looking. Who applied to grad school right away. Who gave up their job search and then applied to grad school. What kinds of internships and CoOps they had along the way… exact company names, locations, and types of positions. How many of those companies chose to make offers to them. You’ll get a lot of VERY VERY useful information.</p>

<p>Without using names, I was quite unimpressed by one of the universities I visited with my son. It is a household name and if you look at their crap survey reports, you’ll think everything is just A-OK. Their real data indicates everything is far from A-OK.</p>

<p>The point is… this is journey. No one data point is the end-all, be-all. We can take any ranking system and beat the hell out of it and say it’s worthless. The fact is, they are all worthless if taken separately. IMHO, anyone fully accepting the gospel of USNews is nothing more than fool and deserves what they get. Same goes for someone who only looks at salary data. </p>

<p>In the end, as caring parents, we need to gather all ranking resources available… discounting all, but dismissing none… and couple that with our our research and visits until our children are comfortable with a short list of schools that we know will nurture their gifts and they feel they can fit into.</p>

<p>What USNews could do for us is include some metric… any metric… of graduate success. Right now they are completely devoid of it. Which, IMHO, is a ridiculous situation. Their ranking system is essentially based on university historical reputation and advertising. The better they say they are, the better we believe they are, the better they are, period… with no correlation at all to their actual performance.</p>

<p>In what world does it make sense not to judge an institution by it’s output? I guess in this crazy world, because it happens every day. :-/</p>