<p>"Playing</a> With Numbers" by Nicholas Thompson</p>
<p>"When Elfin was first charged with creating a ranking system, he seems to have known that the only believable methodology would be one that confirmed the prejudices of the meritocracy: The schools that the most prestigious journalists and their friends had gone to would have to come out on top. The first time that the staff had drafted up a numerical ranking system to test internally--a formula that, most controversially, awarded points for diversity--a college that Elfin cannot even remember the name of came out on top. He told me: "When you're picking the most valuable player in baseball and a utility player hitting .220 comes up as the MVP, it's not right.""</p>
<p>To Elfin, however, who has a Harvard master's diploma on his wall, there's a kind of circular logic to it all: The schools that the conventional wisdom of the meritocracy regards as the best, are in fact the best--as confirmed by the methodology, itself conclusively ratified by the presence of the most prestigious schools at the top of the list. In 1997, he told The New York Times: "We've produced a list that puts Harvard, Yale and Princeton, in whatever order, at the top. This is a nutty list? Something we pulled out of the sky?" </p>
<p>The walls around the system that confirmed the top Ivies began to crack in 1996 when Zuckerman hired James Fallows (a contributing editor of The Washington Monthly) to edit the magazine. Fallows hired former New Yorker writer Lincoln Caplan and, when Elfin left in January of '97, Fallows put Caplan in charge of special projects at the magazine, which included the annual development of the rankings. The two began to make a series of changes that improved the rankings, most noticeably by eliminating one decimal place in the scoring (schools now get grades like 77 instead of 76.8) to create more ties and reduce a spurious air of precision. Caplan also hired a statistical expert named Amy Graham to direct the magazine's data gathering and analysis. Although both Caplan and Graham have left the magazine, and both declined to be interviewed, sources within U.S. News claim that, after looking deeply into the methodology of the rankings, Graham found that U.S. News had essentially put its thumb on the scale to make sure that Harvard, Yale, and Princeton continued to come out on top, as they did every year until 1999 after Elfin selected a formula</p>
<p>This was done in large part by rejecting a common statistical technique known as standardization and employing an obscure weighting technique in the national universities category. Consider the data from the 1997 book, the last year the numbers for overall expenditures were posted publicly. Caltech spent the most of any college at $74,000 per student per year, Yale spent the fourth-most at $45,000 and Harvard spent the seventh-most at $43,000. According to the U.S. News formula applied in every single category except for national universities, the absolute rates of spending would be compared and Caltech would be credited with a huge 40-percent category advantage over Yale. Under the formula used solely in this category the difference between Caltech and Yale (first place and fourth place) was counted as essentially the same as the difference between Yale and Harvard (fourth place and seventh place) even with the vast difference in absolute spending.</p>