US NEWS Rankings: What Would They Look Like Without Peer Assessment Score?

College Search & Selection
61

<p>

</p>

<p>I think this really represents the heart of the rankings argument. People take issue with these generic national and regional rankings lists because of how they attempt to normalize what can’t be normalized. That is to say, colleges are multi-faceted, complex institutions by nature… they simply cannot be ranked using a one-size-fits-all formula that doesn’t even take in to account factors pertaining to the innerworkings of the colleges being ranked.</p>

<p>There are indeed plenty of smaller, lesser known schools out there with great programs, faculty, and facilities, yet they get overshadowed year after year by the super behemoths (some of whom, 1) don’t even offer competing programs, and 2) may actually have weaker programs in certain areas). </p>

<p>The real question then remains: Why do we even need these national and regional rankings lists? What is the point of having them? If a student is interested in engineering, that student should look at schools strong in engineering… Harvard, Dartmouth, fine schools for some liberal arts programs, but not great schools for engineering. Then there’s the issue of size of school, school setting, atmosphere, average class size, etc… these are all infinitely more important factors to consider than the national and regional rankings.</p>

62

<p>There are 3 major issues (among many) with UNNWR (or any) ranking. They lack: </p>

<ol>
<li><p>The ability to personalize the weightage of various factors and generate scores that are relevant to one’s situation. </p></li>
<li><p>A quick way to incorporate net price & the ability to visualize the location of various colleges on a graph of net price versus personalized weighted scores. Ability to do a “what-if” analysis in changing various factors on the outcome of scores on the graph. </p></li>
<li><p>The ability to incorporate the changes in learning that has occurred between the incoming freshmen and the outgoing graduates in measurable areas such as:</p>

<p>(a) Critical thinking, problem solving, synthesis & writing etc., and </p>

<p>(b) The body of knowledge that the students are expected to have mastered during their time in college. While it is understood that it may be difficult to measure the field specific knowledge in various fields, at least in the fields that do have board exams, include the certification results from various schools.</p></li>
</ol>

63

<p>Here are the PA and GC scores for the top 40 or so “national LACs”.</p>

<p>LAC PA / GC</p>

<p>Williams 4.7 / 4.6
Amherst 4.6 / 4.5
Swarthmore 4.6 / 4.5
Bowdoin 4.3 / 4.5
Middlebury 4.3 / 4.4
Pomona 4.3 / 4.5
Carleton 4.3 / 4.4
Wellesley 4.3 / 4.5
Claremont McKenna 4.1 / 4.6
Davidson 4.2 / 4.3
Haverford 4.0 / 4.2
US Naval Academy 4.1 / 4.8
Vassar 4.1 / 4.5
Hamilton 3.9 / 4.3
Washington & Lee 3.8 / 4.2
Harvey Mudd 4.2 / 4.6
Grinnell 4.2 / 4.2
US Military Academy 4.0 / 4.8
Wesleyan 4.1 / 4.4
Colgate 4.0 / 4.4
Smith 4.2 / 4.4
Bates 4.0 / 4.4
Colby 3.9 / 4.3
Macalester 4.0 / 4.2
Holy Cross 3.6 / 4.2
Oberlin 4.0 / 4.2
Scripps 3.6 / 4.4
US Air Force Academy 3.9 / 4.7
U Richmond 3.8 / 4.1
Bryn Mawr 3.9 / 4.3
Colorado College 3.7 / 4.1
Barnard 3.8 / 4.4
Bucknell 3.8 / 4.3
Kenyon 3.7 / 4.1
Pitzer 3.5 / 4.4
Lafayette 3.4 / 3.8
Trinity 3.6 / 4.2
Bard 3.4 / 4.0
Mount Holyoke 3.8 / 4.1
Sewanee 3.5 / 4.0</p>

<p>Some observations: 1) HS GCs love the service academies, giving them their highest ratings. 2) Just as with “national universities,” GCs are more generous graders of LACs. Only AWS have higher PA scores than GC scores, and then by the slightest of margins, but going in the other direction there are several schools with GC scores as much as 0.8 or 0.9 higher (on a 5-point scale) than the corresponding PA score. 3) Setting aside the service academies as a special case, GCs seem to hold the top private LACs in slightly lower regard than the top “national universities” (I’m assuming it’s the same GCs filling out the surveys, which is not the case for PA scores). 4) GCs seem to view the top 25 or so private LACs as pretty much alike, with scores spanning a narrow range from 4.2 to 4.6. PA scores for these same schools show greater variability, ranging from 3.6 to 4.7. 5) As with “national universities,” GCs show some bias toward schools in the Northeast and in California. Nine private LACs get GC scores of 4.5 or higher; all 9 are in the Northeast or California. The highest-rated Midwestern LAC is Carleton with a GC score if 4.4; the highest rated Southern LAC is Davidson with a GC score of 4.3. 6) GCs apparently love NESCAC, awarding NESCAC schools scores of 4.1 (Connecticut College), 4.2 (Trinity), 4.3 (Colby, Hamilton), 4.4 (Bates, Middlebury, Wesleyan), 4.5 (Amherst, Bowdoin, Tufts), and 4.6 (Williams). </p>

<p>Largest disparities between GC score and PA score:</p>

<p>0.9 Pitzer
0.8 Scripps, US Military Academy, US Naval Academy
0.7 US Naval Academy
0.6 Holy Cross, Barnard, Trinity, Bard
0.5 Claremont McKenna, Bucknell, Sewanee
0.4 Vassar, Hamilton, Washington & Lee, Harvey Mudd, Colgate, Bates, Colby, Bryn Mawr, Colorado College, Kenyon, Lafayette</p>

64

<p>I know that CC people are very worried about ranking and I know that this seems to be a Status symbol in the North East where kids go to college. All I can say is this, my daughter is smart and personable, just started school at a very reasonably priced State “Directional” with a campus of maybe 15K, average SAT above 1200 (reading and math) still ranked in the top four hundred or so colleges in the Country (which is first tier right?), has some interesting classes to teach her to think critically, just rushed a Sorority and was selected by her first choice, and she is having an absolute blast. Studies show that 60% of learning occurs outside the classroom. So she will be heavily involved in College, will or has already made lots of friends, will get a good value packed education and will graduate with no debt. I on the other hand am avoiding spending an arm and a leg to send her to school. To heck with the rankings.</p>

65

<p>What state do you live in where a directional has an average SAT of above 1200?</p>

66

<p>I sent you a private email with the name of the school.</p>

67

<p>Great analysis bclintonk! It would also be interesting to compare GC rankings to admission rates. I could see GC’s having a bias to those schools that are hardest for their students to get into.</p>

68

<p>Academic profile of my daughter’s public directional university. Tuition and fees in state per year. . $6500. </p>

<p>Academic Profile (Based on Fall 2012 enrollment)
Average incoming freshman GPA 3.9
Average incoming freshman SAT 1212
Average incoming freshman ACT 27</p>

69

<p>You might have noticed the similarity between the LAC GC scores posted by BCK and the 2013 I posted earlier.</p>

<p>[High</a> School Counselor Rankings | Rankings | Top National Liberal Arts Colleges | US News](<a href=“http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/national-liberal-arts-colleges/high-school-counselor]High”>http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/national-liberal-arts-colleges/high-school-counselor)</p>

<p>There are two interesting tidbits in the methodology used:</p>

<p>

</p>

<p>Averaging the last two years seems to result in “flattened” results with little changes, and a lesser impact of recent changes. </p>

<p>Considering the list of GC polled, it is clear that the bulk of the responses originates
from public schools. Also, I’m not certain what USNews means by the LARGEST private independent private schools.</p>

70

<p>The former president of Lafayette refused to participate in the peer rankings because, “How am I supposed to know what goes on in classrooms and the quality of teaching at a college where I don’t work?” A couple years ago Clemson admitted that they were intentionally lowballing their peer assessments so that they’d rise in the rankings by comparison.</p>

<p>Malcolm Gladwell did a great article on USNWR a few years ago. He quoted a statistic where judges were asked to judge the quality of a list of law schools. Included on the list was Penn State, which at the time did not HAVE a law school. That didn’t stop most judges from rating it as a good law school… which shows exactly how much their opinions were worth. I don’t doubt the peer assessments from one college president to another are just as invalid.</p>

<p>A bigger question is why USNWR insists on rating colleges on their INPUTS - the GPA and SAT of their admits and their selectivity, instead of ranking them on their OUTPUTS - GRE scores, grad school admissions, graduation rates, post-employment graduation numbers, etc.</p>

71

<p>

</p>

<p>Xiggi-I wondered about that as well. It’s possible to do a search for schools by size here</p>

<p>[Pages</a> - NAIS School Search](<a href=“http://www.nais.org/Users/Pages/SchoolSearch.aspx]Pages”>http://www.nais.org/Users/Pages/SchoolSearch.aspx)</p>

<p>But are they looking at K-12 enrollment or 9-12 enrollment? It looks like the 40 largest independent schools would all have well over 700 students if all grades are counted. What that means is that with the exception of a few of the large boarding schools pretty much none of the elite 9-12 schools are represented. Ironic, since these are schools that put tremendous resources into college counseling and have college advisors who are much better informed about colleges than the average GC.</p>

72

<p>

</p>

<p>This part of the USNWR rankings focuses on it as well:</p>

<p>

</h1>

<p>[Best</a> Undergraduate Teaching | Rankings | Top National Liberal Arts Colleges | US News](<a href=“http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/national-liberal-arts-colleges/undergraduate-teaching]Best”>http://colleges.usnews.rankingsandreviews.com/best-colleges/rankings/national-liberal-arts-colleges/undergraduate-teaching)</p>

<p>My son considered (and/or applied to) a number of these schools–few of which were the better-known ones on the list. Commitment to teaching undergraduates was the single most important factor for us.</p>

73

<p>

</p>

<p>Because that would be logical!</p>

<p>Seriously though, the rankings metrics are cherry picked in order to skew the rankings in favor of the well-established, wealthy, highly selective colleges. If they were to overhaul their methodology and look at output metrics like you have suggested, that would give an advantage to every college out there to improve, thereby giving the top schools increased competition and a less stable footing in the rankings, which they really don’t want. The rankings are cleverly set up in a way to prevent large movements within the rankings… top pack schools will forever stay on top, and the bottom pack schools will forever stay on the bottom. This is, in large part, due to the fact that top HS talent utilizes rankings to make college decisions. These students flock to the top schools, which in turn boosts the ranking of those schools (remember, the rankings are input-heavy). The middle and bottom pack schools are then left with the the scraps - with applicant stats that place these schools at a disadvantage in the rankings. It is also next to impossible for lower ranked schools to significantly improve in the rankings since selectivity drives a big part of the rankings, and selectivity is often a game of supply and demand… and demand follows selectivity (usually).</p>

<p>That’s my take anyway.</p>

74

<p>^That pretty much sums it up.</p>

75

<p>(oops - meant to write “post graduation employment numbers” above… post employment graduation numbers makes no sense!)</p>

76

<p>“If they were to overhaul their methodology and look at output metrics like you have suggested …”</p>

<p>A simple normalized GRE/LSAT/MCAT/etc minus SAT/ACT could be telling.</p>

77

<p>^^</p>

<p>The key would be to have a super intelligent normalization process. Otherwise it would turn into the same idiotic graduation expectation index used by USNews and the even worse Mother Teresa one. In so many words, it would penalize the most selective schools and reward schools such as UTEP or the lesser selective schools with average SAT.</p>

78

<p>^ Right. I can just see schools like Harvard and Swarthmore ending up at the bottom of someone’s list because their students went from the 98th percentile on standardized test scores going into the college to the 99th coming out, while some podunk school whose 3 students who actually applied to grad school went from the 15th to 24th percentile gets to crow about how it’s one of the best in the nation!</p>

79

<p>^ Not very intelligent normalization!</p>

<p>Scary thought measuring actual improvement of students!</p>

80

<p>^ Don’t do a ranking, just list the results. Applicants with a 2100 SAT can see schools with a 2100 average SAT and compare their GRE etc.</p>