<p>ProudDad:</p>
<p>I just went to the US News website. It is heavily footnoted with exactly the sort of info you say they don't give.</p>
<p>Once again, it's all in the context of the methodology. Accept it. Don't accept it. Really, no one cares what you do. But please, stop whining.</p>
<p>I just checked the 2007 USNews hard copy. Reed's footnote says: "School refused to fill out US News statistical survey. Data that appear may be from school in previous years or from the National Center for Educ. Statistics unless otherwise noted." That's it. That does not comply with Proud Dad's list of 4. </p>
<p>I agree with what Proud Dad said. I don't think he's whining, Tarhunt; I think he and others here make legitimate arguments and I am glad to read them. You disagree with him; fine, but he's allowed to make his points here without enduring your criticism. </p>
<p>And I agree, Originaloog -- I don't understand why we need to rank colleges every year. Especially a system designed by a magazine who changes its parameters every so often.</p>
<p>sly_vt:</p>
<p>So, nothing said on this board should be subject to criticism? Interesting position.</p>
<p>Look, US News has few options here. If they drop Sarah Lawrence completely, the school will scream about exclusion. "Potential students can't even see that we EXIST!!!" they'll cry. If they give a zero for the lack of SAT data, it would drop SLC way down the rankings. If they apply an average SAT score, it would encourage all those schools below average to stop reporting SAT scores in order to get a bump. So, what they've decided to do is apply a penalty for non-reporting, but not zero credit.</p>
<p>It's not USN's fault if SLC doesn't want to provide data USN uses in its ranking. What they're doing about it is rational.</p>
<p>
[quote]
why do we even need to "rank" colleges?
[/quote]
</p>
<p>No one "needs" to do much of anything. Clearly, there is a market for rankings or so many organizations wouldn't be attempting them. In other words, there's a demand. It's not too surprising that some people will create a supply.</p>
<p>No, the problem is bigger than that. </p>
<p>SLC is unusual in that they have decided not even to college SAT data, but many colleges are going the SAT-optional route. Basically the SAT has been shown to have little bearing whatsoever on the quality of the class selected; it has minimal predictive value; it is a biased instrument in terms of race and economic class that makes it harder for colleges to achieve diversity; and the college board's accuracy in score reporting has been called into serious question the past year with some major errors. </p>
<p>A college should be free to reject the standardized tests as a consideration in admissions. Whether or not they use the tests has no bearing whatsoever on how selective they are or the quality of their students -- there are other measures they can use, some of which are far better in terms of evaluating academic ability. </p>
<p>The US News policy of ranking colleges based on reported test scores impacts every single college that might contemplate going the same route and dropping the tests. So basically, US News has decided that it will dictate how colleges in America select their students so that they can sell magazines. </p>
<p>I think colleges should be free to set their own admission criteria, and certainly dispense with outside-administered standardized tests if they feel they are not useful. </p>
<p>When US News tells colleges that any omitted data will be downgraded, they are exercising their financial clout to dictate to every college in America how to go about selecting their students.</p>
<p>None of this has anything whatsoever to do with a truly informative "ranking" system. If US News wanted to have a list of the rank-order of the most selective colleges in the country, they could simply ask for percentages: how many applied, how many were admitted. They'd probably end up with Julliard and Deep Springs at the top of the list, which obviously they don't want. They've manipulated their "methodology" every which way and back again to force Harvard, Yale & Princeton to keep coming out on top; the main way they do that is by cheating -- rather than rely on objective data, they invented the idea of "peer review" which functions something like phone-in voting on American Idol -- it's got nothing to do with how good the college is; it's all about how well known and how popular it is.</p>
<p>Different issues are mixed up here.
1. Rankings. Many of us discount rankings in college selection.
2. Type of data that is included. some of it is of little importance to prospective college applicants and should be ignored. Others, however, are extremely useful. It would be better to concentrate on what type of data applicants find useful and should be included/retained in what USN&WR reports.
3. Manipulation of data by both reporting colleges and by USN&WR. It's okay to bash USN&WR. Let's remember that colleges also manipulate data. </p>
<p>In the end, it would be far more interesting to discuss what type of data applicants need to make up their minds as to which college to select. And let's remember that not everyone has heard of Wellesley as opposed to Whitman (or vice-versa) or know if there is a difference between Skidmore and Sarah Lawrence.
As for people who get hung up on applying only to top ten or top 20 or whatever, shrug.</p>
<p>This may be of interest...</p>
<p>calmom</p>
<p>Thanks for saying it better than I've been able to. My son feels that the American character is all about simple-mindedness. Most of us don't want to/don't know how to deal with complexity or shades of gray, so we act as if everything can be summed up by "top 10" lists. Lately I've been inclined to agree with him.</p>
<p>The posters who've stated that US News developed a methodology and analyzes according to it, with the results being whatever the results are when those formulas are applied, are correct.</p>
<p>However, the issues that other posters have mentioned, that absolute rankings based on both acceptable quantification (number applied, % admitted) as well as false quantification (e.g. peer review) are a flawed concept, that those parents and students who are not up to the 'research-the-world' standard of CC can end up accepting the rankings as gospel, that the rankings do not reveal some very important facts about outcomes, etc. etc. are valid. Caveat Emptor!</p>
<p>The US News rankings facilitate the tendency to oversimplify the factors involved in a complex decision. They lend themselves to fostering insufficient analysis about a very important decision – and when taken too seriously (as their own advertising suggests one should do), they can contribute to the risk of a student making a choice that is a bad fit.</p>
<p>The other notion that has floated up, the boycott - that seems to be an appropriate response. By consumers, if not by colleges and universities.</p>
<p>Thanks Dean J. I'm proud to call you a Virginian! (I realize you were offering another objective view, not an endorsement of either extreme.)</p>
<p>
[quote]
The posters who've stated that US News developed a methodology and analyzes according to it, with the results being whatever the results are when those formulas are applied, are correct.
[/quote]
Maybe it's a methodology, but when
[quote]
the magazine will calculate the college's ranking by assuming an arbitrary average SAT score of one standard deviation (roughly 200 points) below the average score of our [SLC's] peer group
[/quote]
I wonder how close this gets to slander (calmom, you're a lawyer) when they willfully and knowingly publish such derogatory information.</p>
<p>I think it's called libel if it's in print.</p>
<p>Whoa! Where to start with this! Yikes! A lot of stuff here.</p>
<p>
[quote]
SLC is unusual in that they have decided not even to college SAT data, but many colleges are going the SAT-optional route. Basically the SAT has been shown to have little bearing whatsoever on the quality of the class selected; it has minimal predictive value; it is a biased instrument in terms of race and economic class that makes it harder for colleges to achieve diversity;
[/quote]
</p>
<p>This is strongly debated. First off, what the newspapers call a "weak relationship" is, to social scientists who deal with multivariate analysis on large numbers of factors, a rather strong relationship. Secondly, the research that I have seen has been rather poor, probably for cost reasons. It treats grades at all schools as though they are equal, e.g., an "A" at Reed equals an "A" at East Idaho Normal School. It also makes no allowance for average grades in majors. So an "A" in quantum physics equals an "A" in recreation management.</p>
<p>The idea is that, if enough schools are taken into account, grade inflation/deflation will even out. That's a nice idea, but it ignores two issues:</p>
<ol>
<li><p>SAT scores tend to cluster at schools, so the competition in the mid 50% range tends to fall within one standard deviation. An "A" against the competition at Swarthmore is not equal to an "A" against the competition at the University of Denver.</p></li>
<li><p>People with higher SAT scores tend to go to difficult majors more often than those with lower SAT scores. It is harder to obtain high grades in more difficult majors.</p></li>
</ol>
<p>As for being biased, of COURSE it's biased. It's biased towards those who have learned a fair amount of math and those who read a lot. But then, college classes tend to be biased towards those who have a fair amount of math and read a lot.</p>
<p>
[quote]
A college should be free to reject the standardized tests as a consideration in admissions. Whether or not they use the tests has no bearing whatsoever on how selective they are or the quality of their students -- there are other measures they can use, some of which are far better in terms of evaluating academic ability.
[/quote]
</p>
<p>I agree that colleges should be able to accept or reject you on just about any basis whatsoever, except for discrimination against protected groups. But I don't agree that SAT/ACT scores have no bearing whatsoever on how selective a college is or is not. As for other measures being better, I saw the measures you had at the beginning of this thread, and I find most of them not the least useful.</p>
<p>
[quote]
The US News policy of ranking colleges based on reported test scores impacts every single college that might contemplate going the same route and dropping the tests. So basically, US News has decided that it will dictate how colleges in America select their students so that they can sell magazines.
[/quote]
</p>
<p>Selling magazines is a bad thing? I'm interested in the data. I'm willing to pay them for their efforts. Does that make me a bad person? And if they have a ranking system they feel comfortable with, what should they do when a given college decides not to pay attention to some of the data? Change their entire ranking system to accommodate that college? Why?</p>
<p>
[quote]
When US News tells colleges that any omitted data will be downgraded, they are exercising their financial clout to dictate to every college in America how to go about selecting their students.
[/quote]
</p>
<p>US News can dicate nothing. Another way to look at what they're doing is that they need to account for the absence of data some way. They really don't have any good choices, so they must choose from among those bad choices. Personally, I would just give no points for missing data. But that's me. That way, I wouldn't have to make data up.</p>
<p>
[quote]
None of this has anything whatsoever to do with a truly informative "ranking" system.
[/quote]
</p>
<p>I think it's informative. Does that make me intellectually inferior to you?</p>
<p>
[quote]
If US News wanted to have a list of the rank-order of the most selective colleges in the country, they could simply ask for percentages: how many applied, how many were admitted.
[/quote]
</p>
<p>So, you're suggesting they throw out their current methodology for a worse one? This would put Lane near the top, as well, and drop Chicago way down the list, but I would venture to say that most of the kids who go to Lane would probably not be accepted to Chicago. Why would you want to throw out a multi-factor system for a single, and very misleading, one?</p>
<p>
[quote]
They'd probably end up with Julliard and Deep Springs at the top of the list, which obviously they don't want. They've manipulated their "methodology" every which way and back again to force Harvard, Yale & Princeton to keep coming out on top; the main way they do that is by cheating -- rather than rely on objective data, they invented the idea of "peer review" which functions something like phone-in voting on American Idol -- it's got nothing to do with how good the college is; it's all about how well known and how popular it is.
[/quote]
</p>
<p>Those are some fairly serious accusations, don't you think? Deep Springs wouldn't be at the top because it's a two-year college, and Juilliard's admit rate isn't of much interest to any of US News' readers except those who plan to specialize in the arts.</p>
<p>Funny thing about "manipulating" the methodology to put Harvard, Yale, and Princeton at the top. You can check out just about any world college ranking published by anyone (except for the specialty rankings like "best dorms"), and find those three schools at or near the top. In fact, if you were to use admit rates only, you'd still get those three at or near the top. I just can't conceive of anyone who thinks those schools aren't among the very top institutions in the world and, for that matter, in history. One can certainly argue, as I would, that the undergrad experience may be better at other places for a particular kind of student, but to suggest that these schools don't have stunning faculty, stunning facilities, and stunning students is simply ...</p>
<p>As I recall, USN's very first ranking was peer-review only. They've added the objective data later. So, that would tend to indicate that the trend has been towards more objectivity.</p>
<p>The social sciences often turn to subjective data when objective data needed to tell the "whole story" is simply not available. For instance, employee performance reviews generally include three sections: One objective (performance against goals), one subjective but observable (behaviors in attaining those goals), and one developmental. The reason for this is that ethical, team behaviors are desirable, but can't be measured completely objectively. The subjective section of the performance review became necessary as people achieved objectives by leaving blood on the floor. The subjective section actually improves the review (overall) and employee behavior.</p>
<p>Personally, I find the USN rankings to be more useful with the PA than without. With the PA, all but three public schools are elevated, as I think they should be.</p>
<p>As for the PA being like American Idol, I have never watched the show. But I understand it's judged by audience voters. But you are equating Joe Sixpack with high-level administrators in colleges who work in the "industry."</p>
<p>A bit of an exaggeration on your part, n'est pas?</p>
<p>
[quote]
As for people who get hung up on applying only to top ten or top 20 or whatever, shrug.
[/quote]
</p>
<p>Eloquently put.</p>
<p>
[quote]
My son feels that the American character is all about simple-mindedness. Most of us don't want to/don't know how to deal with complexity or shades of gray, so we act as if everything can be summed up by "top 10" lists. Lately I've been inclined to agree with him.
[/quote]
</p>
<p>Well, so now I'm "simple minded." It could be true, I suppose. Nothing in my experience would lead me to believe so, but I'm open to proof.</p>
<p>
[quote]
However, the issues that other posters have mentioned, that absolute rankings based on both acceptable quantification (number applied, % admitted) as well as false quantification (e.g. peer review) are a flawed concept, that those parents and students who are not up to the 'research-the-world' standard of CC can end up accepting the rankings as gospel, that the rankings do not reveal some very important facts about outcomes, etc. etc. are valid.
[/quote]
</p>
<p>So, you need to protect me from myself? You need to protect others from themselves? Information is not to be trusted in the hands of the great unwashed. You, like bethie, feel you must protect the "simple minded" from their conclusions?</p>
<p>How very noble of you.</p>
<p>
[quote]
Now there is one gutsy real President of a College!
[/quote]
How is she gutsy? She's whining because the standard ranking systems don't favor her school. Yet she still wants to benefit from being in the rankings. Can't have it both ways, I'm afraid.</p>
<p>
[quote]
...they willfully and knowingly publish such derogatory information.
[/quote]
If SATs are worthless in the eyes of SL, with no predictive value, then how can USNews' decision to use a standard deviation step-down from the group average be derogatory?</p>
<p>The rankings do remind me of Consumer Report ratings of new cars. CR includes category ratings involving number of cup holders, availability of seat warmers, and quality of leather as factors in their ranking methodology. Not as important in my eyes as gas mileage or horsepower. So I have to look beyond the ranking number and only view the data I care about. It's the same for colleges. </p>
<p>It's not as if lists of SAT optional colleges are difficult to find!</p>
<p>The Consumer Reports auto ratings are a good case in point for me, as well. They will rate a car down if the ride isn't smooth. To me, that generally means a softer suspension and better cornering. I also don't care about ease of entry and exit.</p>
<p>
[quote]
SATs are worthless in the eyes of SL, with no predictive value, then how can USNews' decision to use a standard deviation step-down from the group average be derogatory?
[/quote]
Because SL knows that decisions are made based on such numbers, and they are being denied applicants because USNews knowlingly publishes false, made up data.</p>
<p>Just because I can, let me bury the Consumer Reports analogy. CU (Consumers Union) tests cars like they test toasters. Or spray paint. Or TVs. But if you really care about what you drive or how you drive defies appliance-style ratings, they are relatively meaningless. More weight is generally given to their Frequency of Repair records which everyone assumes are based on statistically sound data. But they won't ever tell you how many responses they had per model ranked. In fact, they will tell you, if you dig a bit, that repair frequency is calculated only from numbers supplied by their subscriber base (members in the case of CU) who bother to send in the form, and nothing else. Case in point: two decades ago lots of luxury cars had electrically telescoping antenna masts. If your mast was bumped or banged while deployed (protect your mast at all costs 'cause I'll bet it hurts to have it banged while deployed!), it would likely get bent causing it to not fully retract, become slow, or not work at all. Similarly, if the mast got dirty, it would often not fully retract. Or if you went through a car wash with the radio on, the mast would get snapped-off by the brushes. In every case where the CU member had experienced an antenna mast problem, replacement, or repair, CU rated that as a data point under "electrical system". It didn't take long for major manufacturers to figure out that their BMWs, Mercedes, and Jaguars were being dinged for "electrical system" problems far "worse than average" according to those CU black dots, and that scared potential buyers. Notice that around 1990 most antennae became embedded in the glass--and radio reception suffered because of it. </p>
<p>So, manipulate statistics any way you want to prove whatever you want. But if those collecting data don't even take care to explain how it's collected or collated, and the world accepts it as gospel, you might never realize the joy of driving a BMW and forever be doomed to your four-wheeled toaster. I think you can supply the rest of the analogy from here as it refers to colleges, LACs in particular. Remember; your mileage may vary!</p>