USNews 2011 - New Methodology and Stanford's strange result

<p>ICalc, you should worry about yourself right now. Read your school newspaper and comments:</p>

<p>[Princeton</a> drops behind Harvard in U.S. News rankings - The Daily Princetonian](<a href=“http://www.dailyprincetonian.com/2010/08/16/26201/]Princeton”>http://www.dailyprincetonian.com/2010/08/16/26201/)</p>

<p>

Thanks bclintock, I think that points out a huge flaw in the methodology (as if there were not enough already). If USNWR only averages the opinions of those who have heard of or are familiar enough with Judson to rate it (the rest leaving it blank), and that might be a very small number of people, then there is a very good chance the number will be skewed upward. This is a well known phenomenon among survey designers who know what they are doing.</p>

<p>vossron - Thanks for posting that quote from the Reed president, especially the part about not possibly being able to really know enough about other schools to rate them except for maybe a very few. I have been saying that to people that support PA for ages, and you can imagine the crap I get back about it. Yet it seems so obvious. Now they have gone from bad to worse by using GC’s, who know even less in many ways.</p>

<p>

</p>

<p>UCB, yes, the PA is an opinion. And, that the main problem is that the current version purports it to be a scientific assessment by experts all the while asking the responders to evaluate the schools without defining the criteria very well. </p>

<p>I understand that “disputing” the opinion of 2,000 people is silly. But, it is not silly to question tha validity of the results of a number of outliers that deviate substantially from the norm. When this happens, one starts looking at what might have created the discrepancy. In the particular cases of Mudd --a school that is often compared to Caltech-- and Smith --a school mostly compared to Mt Holyoke-- the only conclusion is that the surveyors exhibit geographical cronyism and form their opinion based on decades old data. Fwiw, this same bias is clearly exhibited in the case of Pomona. Ask anyone with a modicum of knowledge of LACs to list Pomona’s closest peers and the answer will start with Swarthmore. In terms of faculty, faculty resources, student body, and academic performance, the schools are undistinguishable. It would be nice to know why the LAC responders cannot expand their vision West of the Midwest, or a bit to the South. It would also be nice to know what the school is EXPECTED to do to be rewarded by a higher reputational index. Increase selectivity to single digit admission and perfect SATs? Graduate 100% of the class in less than 4 years? Limit class sizes to 10? Abandon the consortium and move the campus to the East Coast? Become an all-female school? </p>

<p>In the end, no matter how one slices it, the PA should not be a proxy for the age of a college nor be so influenced by a nostalgic reminder of yesterday’s reputations of schools.</p>

<p>

</p>

<p>The saddest past is that the same excerpt has been posted countless times without making a dent in the attitude of the PA supporters. To them, Diver is only an expert when part of the silent mob. As soon as he speaks up, he becomes a clueless renegade with a chip on his shoulder. The fact that hundreds of officials think like Diver and that dozens have admitted or being caught manipulating the survey does NOT matter to the PA supporters.</p>

<p>Of course, the reason is crystal clear: they support the PA because it helps their school in the rankings. USNews could admit that the survey is filled by twelve drunken monkeys and they will still say … so what!</p>

<p>How drunk?</p>

<p>xiggi, </p>

<p>I agree with you that there are “outliers”…you likely think my alma mater is an outlier. I happen to think that PA for LACs is more difficult to measure because LACs have a different focus (more teaching than research) and that focus is not publicized as much in comparison to the national universities focus (more research than teaching).</p>

<p>Asking academics to fill out a survey, they are going to base their info on factors visible and important to them. Since research results are more visible and quantifiable than teaching results, LAC administrators have a greater challenge filling out the survey, IMO.</p>

<p>

It better be. I can not imagine that they generated the PA numbers by computer and had to manually re-type them for their website. It must be the “drunken monkeys”… or I could be wrong. Is it redundant to use the quotation marks?</p>

<p>

Until USNews comes up with another standard to measure overall faculty quality (for which PA is a proxy since academic department distinction is derived from its faculty moreso than the high SAT scoring 18-year old), I will continue to support the PA.</p>

<p>I see. So despite numerous administrators besides Reed’s saying things like “I am responding…to a bad survey and an impossible question”, and “Nobody’s paying us to help U.S. News produce a commodity.… When I’m being paid hundreds of dollars a day, why would I spend time reading up on South Dakota State [University] so I could give U.S. News a better answer?”, you want to support it. Interesting. That’s even besides the manipulation cases, many of which I am sure are undiscovered since most would not show their responses to a reporter.</p>

<p>Here is an entire research paper devoted to how poor PA is: <a href=“http://www.cluteinstitute-onlinejournals.com/PDFs/2005126.pdf[/url]”>http://www.cluteinstitute-onlinejournals.com/PDFs/2005126.pdf&lt;/a&gt;&lt;/p&gt;

<p>

Although I presume that the inclusion of the opinions of high school counselors in that metric has to somewhat dilute that support–at least in theory. ;)</p>

<p>lol, ewho, I am not worked up in the slightest that Princeton is ranked #2 after Harvard. Thanks for keeping track of my school’s paper.</p>

<p>

</p>

<p>UCB, but again, here’s the problem. </p>

<p>If we were to assume that the PA for the national universities is indeed a proxy (or a partial one) for research, it would non-sensical to expect schools that are not research institutions to be measured in the same manner. I think you’re are actually saying something along the same lines.</p>

<p>However, even if that would be true, do we really believe that the responders might be able to REALLY evaluate the amount and quality of research produced by 200 LACs? Fwiw, although you know that I do not like to discuss CMC (see my dislike to wave the alma mater pompoms) but you might know that in the world of ECONOMIC research, a analysis using the STANDARDS recognized to measure the quality of that precise field revealed the leader among LACs. It is not hard to guess that a school that has one of the largest UG faculty for economics came out on top. But, again, how can we expect that the Prez or Dean of Kalamazoo have the slightest idea of the SPECIALIZED research prowess of CMC or … Bucknell or Rhodes? Frankly, I would be floored if the Kalamazoo people could separate Pomona’s from Pitzer’s attributes, except for what they read on USNews. </p>

<p>And so it goes. In the meantime, we should stop debating the PA and let the academics sort out their mess. I will never change my mind about the current PA. I also realize that the changes I’d like to see will never happen. I also realize that people will keep on coming up with “explanations” of the PA is, from CollegeHelp’s extensive correlations of data attempts to the faculty uber-alles approach of Clinton or the overly simplistic “analyses” of the clueless cheerleaders. And around and around we will go. </p>

<p>In a way, it regret that this discussions HAD to veer again in a PA discussion, when I started this as a mere technical discussions about the new ARI.</p>

<p>^ Fair enough. </p>

<p>I just want to say that all the “explanations” of PA are trying to rationalize the collective result/opinion of 2,000 responses to a very simplistic survey.</p>

<p>Because the survey is simple, it generates a relatively high response rate, and IMO, deserves a simplistic analysis from this “clueless cheerleader”.</p>

<p>

Have you contacted USNews? Did you get a response?</p>

<p>I emailed the boss. No response so far. :)</p>

<p>

</p>

<p>Rest assured that I was not making a direct nor oblique reference to you.</p>

<p>The irony is that the technology enabling this discussion renders obsolete the need for one-size-fits-none mass-printed rankings. The same data can drive a web site where one’s own weightings (e.g., 0% for PA and GC) can be entered. I’d guess that a sufficient number of our 3 million high school seniors would pay $1.99 each for access, to support such a site. Maybe a bad guess!</p>

<p>

</p>

<p>I think the two of you ARE saying, close to the same thing (although, I admit I haven’t been following the thread very closely.) The PA for LACs is not a proxy for research; it’s a proxy for free-spending, or as Interesteddad likes to say, “spending per student”. The assumption is that spending per student at top universities is plowed-back in the form of relief from teaching duties, TA stipends, lab assistants, post-docs – that sort of thing. </p>

<p>Since LACs don’t generally have those alternatives available to them, they plow money into other things: generous sabbatical leaves, support staff of every description, “auxillary enterprises” (the book store, faculty housing, multiple dining halls.) But, by far the biggest single line-item in any elite LACs “academic” budget is the one for the athletic department. In order to field a comprehensive array of “helmet sports” (football, wrestling, and hockey) plus, the traditional preppy sports of swimming, field hockey, cross-country running and lacrosse, a small college has to devote a far larger proportion of its budget to those sports as a fixed cost; in fact, the smaller the LAC the better it looks because USNews has no way of differentiating between a romance languages professor at Middlebury or a “dean for jocks” at Swarthmore.</p>

<p>xiggi, have you actually attempted contact with the USNWR people? It seems so apparent that they erred w/r/t Stanford, but I wonder if they will man up and admit it and make the correction, or if they’ll be too worried about “opening the floodgates” if they concede the error. I’m not a big rankings devotee by any means, but I recently read that some other recent ranking (Princeton Review, maybe, or Forbes?) also messed up its presentation of Stanford’s information. I think that one had something to do with admission rate or yield or something, IIRC. Pretty strange, but then again I doubt any of this will keep Stanford from being the top “dream school” of students and parents. : )</p>

<p>You are all so outraged that your beloved Stanford is ranked behind Columbia, but that is where it belongs this year. Deal with it. Columbia>Stanford. Manhattan>Silicon Valley</p>