Shake-Up Coming in Ratings Industry?

[johnwesley]

<p>Two opinion columns from America’s “national paper”, both of which seem to pivot on the value of the “alumni giving” category of the USNews poll:</p>

<p>

But what do these rankings really boil down to? </p>

<p>It turns out that for the window dressing of alumni giving, freshmen retention rates and financial resources, they are surprisingly simple. </p>

<p>Research conducted in 2005 by University of Iowa professor Ernest T. Pascarella found an extremely high correlation between standardized test scores and ranking. That means that if you or I were given the average ACT and SAT scores of a college’s enrolled students, we’d largely be able to determine that year’s U.S. News rankings.</p>

<p>[Opinion:</a> Claremont McKenna scandal proves damaging nature of college rankings | USA TODAY College](<a href=“USA Today Educate - Teacher & Student Resources & Guides”>USA Today Educate - Teacher & Student Resources & Guides)

</p>

<p>OTOH:

The rest of the categories (alumni giving! faculty resources!) illustrate that the criteria from which the rankings are born and towards which the universities strive are honorable! </p>

<p>Among other things, alumni giving provides financial aid to students in need and diminishes waste in human capital. While the ranking system may appear exclusive, it actually promotes inclusiveness in an unexpected domino effect.
[Opinion:</a> The upside of college rankings | USA TODAY College](<a href=“USA Today Educate - Teacher & Student Resources & Guides”>USA Today Educate - Teacher & Student Resources & Guides)

</p>

[xiggi]

<p>

That means that if you or I were given the average ACT and SAT scores of a college’s enrolled students, we’d largely be able to determine that year’s U.S. News rankings.

</p>

<p>Would we not be stunned if the reverse were true. How would we react to a lack of correlation between SAT/ACT scores and an attempt to rank the schools?</p>

<p>Fwiw, the correlation works only when looking at “averages.” How does that correlation works for Wellesley versus Pomona, or from Smith versus Harvey Mudd.</p>

<p>While it is (and should be) expected that the schools with the highest SAT scores are grabbing the highest spots, USNews has relied on intangibles (read PA) and graduation rates expectations to “define” the final outcomes.</p>

[johnwesley]

<p>I think what Alpert objects to is the idea that USNews has reinvented the wheel somehow. College selectivity indexes have been packaged in one form or another since the early 1960s. But, it wasn’t until The Yale Daily News began publishing the same statistics accompanied by lively commentary intended to give the general public access to “insder” information, that the college guidebook took off as a popular genre. USNews seems like a step backward in the wrong direction.</p>

[xiggi]

<p>JW, if the USNews has taken us in the wrong direction, why has it been so hard for anyone to reverse the trends? And, of course, that assumes that there is someone who could make up a compelling case for what the right direction should be.</p>

<p>I really do not see anything that could claim to deliver a better tool than USNews. This does not mean that Morse’s mousetrap is better; it only speaks to the absolute garbage produced by the wannabes and copycats. </p>

<p>Now, if you want to REALLY look at a step in the wrong direction, look at the mercenaries such as Lloyd Thacker who would like us to believe the schools actually believe in disclosure and transparency. Look at Thacker’s “partners” and you’ll know why we cannot trust the schools. See <a href=“http://www.educationconservancy.org/BEYONDRANKING.pdf[/url]”>http://www.educationconservancy.org/BEYONDRANKING.pdf&lt;/a&gt; … by invitation only and behind closed doors! </p>

<p>I have no doubt that it were up to the schools only, we would have nothing to look at it, except for glossy brochures with photoshopped students and parks.</p>

<p>The step in the right direction is a low hanging fruit. Force the school to make all their admission and graduation public and force the officials to sign off. And bring an end to all the “anonymous” surveys (a la Peer Assessment) and all unreleased student surveys. </p>

<p>In fact, that is ALL we need.</p>

[johnwesley]

<p>

I really do not see anything that could claim to deliver a better tool than USNews. This does not mean that Morse’s mousetrap is better; it only speaks to the absolute garbage produced by the wannabes and copycats. </p>

<p>

</p>

<p>Actually, I think College Confidential delivers a better tool than USNews; it takes USNews’ fetish for categorization and combines it with some of the advantages of a social network. If there is a major weakness with CC it is that there is often too much information; for every argument there is a counter-argument. Visitors have to devote more time separating the wheat from chaff than they probably bargain for, but, in the end, they get a carload of something USNews is notoriously bad at producing, despite its background in the print media: Content.</p>

[absweetmarie]

<p>What’s the problem with using content-rich sites such as CC (a lot of qualitative data with a bit of quantitative data from people who can’t restrain themselves from sharing) as well as more quantitative sources such as U.S. News? The fact that “for every argument there is a counter-argument” on CC isn’t a weakness, IMO. That’s life, kids! </p>

<p>One site I used in the early days of helping my D in her search was the site that shall not be named, the one they call College P<em>R</em>O* … My goodness! After a while on that site, I more or less concluded that every institution of higher education in the U.S. is filled with licentious, dissipated dumbos, and I didn’t want to send my child to ANY of them.</p>

[bluebayou]

<p>

in the end, they get a carload of something USNews is notoriously bad at producing, despite its background in the print media: Content.

</p>

<p>What you call ‘content’, I call opinion. Nothing wrong with opinion*, but in reality, cc is mostly nothing more than anecdotes of parents, current college students, alums, and a bunch of wet-behind-the-ears teenagers. Sure, cc includes a few, real bonafide college adcoms, but reading their posts is akin to reading spin. “No, we have no SAT cutoffs.” (Of course, what they fail to add is that 99.99% of every unhooked admission over the past decade had a test score above xx.) </p>

<p>*Of course, it is fact when it is MY opinion. :D</p>

[johnwesley]

<p>Xiggi wrote:

Now, if you want to REALLY look at a step in the wrong direction, look at the mercenaries such as Lloyd Thacker who would like us to believe the schools actually believe in disclosure and transparency. Look at Thacker’s “partners” and you’ll know why we cannot trust the schools. See <a href=“http://www.educationconservancy.org/BEYONDRANKING.pdf[/url]”>http://www.educationconservancy.org/BEYONDRANKING.pdf&lt;/a&gt; … by invitation only and behind closed doors! </p>

<p>

</p>

<p>Jeez, Xiggi. That was five years ago! It will be interesting to see whether USNews incorporates the Obama administration’s proposed “Education Scorecard”.</p>

<p>[College</a> Scorecard | The White House](<a href=“Priorities | The White House”>Priorities | The White House)</p>

[johnwesley]

<p>You’ll probably need a subscription password, but, the article goes on to question the wisdom of ranking colleges by future earning power in light of the various ways they already distend themselves (including cheating) in order to conform to the USNews rankings:
[Robin</a> Mamlet and Christine VanDeVelde: Should Colleges Be Factories for the 1%? - WSJ.com](<a href=“Robin Mamlet and Christine VanDeVelde: Should Colleges Be Factories for the 1%? - WSJ”>Robin Mamlet and Christine VanDeVelde: Should Colleges Be Factories for the 1%? - WSJ)</p>

[johnwesley]

<p>

"We can also drop the SAT in college admissions decisions. The test has become a symbol of new-upper-class privilege, as people assume (albeit wrongly) that high scores are purchased through the resources of private schools and expensive test preparation programs. </p>

<p>Instead, elite colleges should require achievement tests in specific subjects for which students can prepare the old-fashioned way, by hitting the books."
–Charles Murray

although, he admits a little further down in the piece that the:</p>

<p>

same students who score high on the SAT score high on achievement tests, and for the same reason (they’re smart and well prepared).

</p>

<p><a href=“Opinion | Reforms for the New Upper Class - The New York Times”>Opinion | Reforms for the New Upper Class - The New York Times;

[bluebayou]

<p>^^Not news, actually. </p>

<p>The Univ of California studied thousands of incoming students and their test scores. And, not surprisingly, UC found that AP Test scores were the best predictor alone of Frosh success. Subject Tests were next, as long as they included the former Writing ST. ST’s were slightly higher predicting than the SAT alone, but can’t remember if it was statistically significant (and too lazy to look up the reports).</p>

<p>Of course, UC in its infinite wisdom, promptly dropped ST’s for admissions. :rolleyes:</p>

<p>I have no doubt that the main reason for UC’s decision is that high ST scores also correlates to students taking AP/IB courses, which correlates to wealthy suburban school districts that offer AP/IB, and schools with excellent teachers.</p>

[johnwesley]

<p>^^The UC decision you refer to may not be news as far as you are concerned, but, it bears repeating since it only begins to take effect with this Fall’s entering class.</p>

[johnwesley]

<p>[College</a> Ranking System Corrupts School Values | Berkeley High Jacket](<a href=“Berkeley High Jacket — The Voice of the Students”>Berkeley High Jacket — The Voice of the Students)</p>

[johnwesley]

<p>This time involving a state reporting agency:
[Rowan</a> University’s reported SAT data fail test - NorthJersey.com](<a href=“http://www.northjersey.com/news/education/college/0315_ROWANSAT.html]Rowan”>http://www.northjersey.com/news/education/college/0315_ROWANSAT.html)</p>

[johnwesley]

<p>Apparently, international students don’t consider USNews one of the top three ranking organizations. that honor goes to the Shanghai, QS World and Times Higher Education World University rankings reports. They go on to say:</p>

<p>

There is, however, no reason to mistrust the USN&WR rankings because they partially rely on self-reported data. The publication does crosscheck self-reported data against other public sources. Further it adjusts its metrics and seeks to close loopholes on a continuing basis. The vast majority of reporting institutions do play by the rules and report accurate data. Still, international parents and potential students may want to consult other public comparison sources. </p>

<p>

<a href=“http://www.koreaherald.com/national/Detail.jsp?newsMLId=20120318000341[/url]”>http://www.koreaherald.com/national/Detail.jsp?newsMLId=20120318000341&lt;/a&gt;&lt;/p&gt;

[johnwesley]

<p>

Advocates say the point is not to measure how each college’s students perform after four years, which depends heavily on the caliber of students it enrolls in the first place, but to see how much they improve along the way. The concern is less about measuring knowledge of chemistry or literature than about harder to define skills like critical thinking and problem-solving. </p>

<p>

<a href=“http://www.nytimes.com/2012/04/08/education/trying-to-find-a-measure-for-how-well-colleges-do.html?_r=1&hp[/url]”>http://www.nytimes.com/2012/04/08/education/trying-to-find-a-measure-for-how-well-colleges-do.html?_r=1&hp&lt;/a&gt;&lt;/p&gt;

[johnwesley]

<p>Xiggi wrote @#23:

John, when all the dust settles in the Vos affaird, you might have to admit that the incentive to cheat was trivial. I know you resent my repeated argument that the impact of increasing the score from 1480 to 1510 had little to no impact on the final rankings. As Adams wrote, why did Vos risk his career and reputation for so little should be ascertained. </p>

<p>

</p>

<p>Today, as reported in the L.A.Times:</p>

<p>

The former vice president, Richard Vos, contended that he acted in response to intense pressure from Claremont McKenna President Pamela Gann to become a more selective college, the report said. He told investigators that he was not aiming to affect the college’s placing in U.S. News & World Report’s listings but to placate Gann with false statistics.</p>

<p>

[Lone</a> official overstated Claremont McKenna scores, report says - latimes.com](<a href=“http://www.latimes.com/news/local/la-me-0418-claremont-20120418,0,3407756.story]Lone”>Lone official overstated Claremont McKenna scores, report says)</p>

[xiggi]

<p>Oh! Well, there is just a bit more to the story, JW! </p>

<p>Remember that little voice that was trying to educate you on how the correct data would NOT change the ranking outcome. Well, here’s the news from the horse’s mouth. </p>

<p>[More</a> Details on Claremont McKenna’s Test Score Data - Morse Code: Inside the College Rankings (usnews.com)](<a href=“http://www.usnews.com/education/blogs/college-rankings-blog/2012/04/19/more-details-on-claremont-mckennas-test-score-data]More”>More Details on Claremont McKenna's Test Score Data)</p>

<p>

In terms of the Claremont College report and the school’s current rank in U.S. News’s Best Colleges rankings published at usnews.com, “We’re reviewing the report,” says U.S.News & World Report Editor and Chief Content Officer Brian Kelly. "Our preliminary calculation shows that based on the correct data, Claremont McKenna College’s ranking would not have changed.

</p>

<p>Unless you think it comes from the other end of the animal! :)</p>

[bluebayou]

<p>I was wondering if JW would come back to this thread? :)</p>

[johnwesley]

<p>xiggi - Frankly, I thought that “little voice” was the gentle crunch of a straw man being stuffed. You were the only poster on this thread content with the idea that this was a matter of moral relativism, or that the only thing at stake was Claremont McKenna’s actual position in the USNews ranking.</p>

<p>I always felt and have always said, that Vos’ actions could not have taken place in a vacuum and that there was more to the story than the “Lone Wolf Goes Off the Deep End” narrative being constructed by you and President Gann. My contention fifty-five posts ago was that this had more to do with President Gann’s management style and the whole cult of the “type-A personality” espoused by some of Claremont McKenna’s defenders. Now, that Vos has spoken on the record, I feel vindicated. The buck still stops at President Gann’s office.</p>