A Review of the USNWR Approach-What is Valuable?

<p>
[quote]
The statistical formula for finding a percentile with interval data could certainly yield a number that was not a multiple of ten. Same with imputation--which they may have done for nonreporting nonreporting students. I don't know how they figured it, but having a number that wasn't a multiple of 10 wouldn't necessarily raise a red flag.

[/quote]
Statistical formulation? Wouldn't they just rank order their students and pick the 75th percentile score?</p>

<p>Mr. Payne, this is a rare occasion where I actually agree with you entirely. Your post has been the most accurate on this particular thread, particularly regarding faculty resources. It seems like universities have managed to leap 20-30 spots in one year in this particular criterion.</p>

<p>
[quote]
The statistical formula for finding a percentile with interval data could certainly yield a number that was not a multiple of ten. Same with imputation--which they may have done for nonreporting nonreporting students. I don't know how they figured it, but having a number that wasn't a multiple of 10 wouldn't necessarily raise a red flag.

[/quote]
</p>

<p>I don't think that that is correct, Hoedown. </p>

<p>In this case, the 75th percentile is the *score * at which 25 percent of students submitting test scores to an institution scored above. The score ONLY exists in a multiples of ten. This means that the 74th percentile might be 740 and the 75th percentile 750, but it will never be 745. </p>

<p>The reported SAT numbers that represent the 25th and 75th percentiles are not averages but exact numbers. </p>

<p>PS Here's the definition of IPEDS. </p>

<p>The 75th percentile is the score at which 25 percent of students submitting test scores to an institution scored above.</p>

<p>The data is all good. As is other information provided elsewhere.</p>

<p>Which is not to say a skeptical consumer should take every item at face value (eg SAT scores for SAT-optional schools, peer reviews provided by competitors with agendas and limited actual perspective).</p>

<p>There can be no uniform agreement on how to weight each item of data; this is up to each individual applicant based on his/her own priorities and values.</p>

<p>There is nothing wrong with USNWR making its own decisions in this regard and printing its resulting "ranking", but I find this aspect of their approach to be less valuable than its role as convenient repository of some relevant data.
And this aspect can even be harmful, due to the prominence it has gained; some people actually take its use and weighting of these parameters as some sort of gospel, instead of just one opinion.</p>

<p>
[quote]
Peer assessment score. I agree with Hawkette. This is only important to students who intend on going on to graduate school.

[/quote]
</p>

<p>Strongly disagree with both. Perceived quality of the institution is vitally important to employers, as well as grad schools. For example, Wall Street and Fortune 100 companies don't often recruit or offer internships to kids at Podunk U, but send teams to top colleges to pick off thier kids. </p>

<p>Alumni Giving is possibly a double dip for the already rich privates since alumni $ goes to build endowment dollars, which is counted elsewhere.</p>

<p>bluebayou,
Are Wall Street and Fortune 100/500 companies going to ABC college to recruit because the school has a top PA or because the students there are terrific? I think it is much, much more the latter at the undergraduate level. The quality arriving on the college's doorstep is already well-screened and Wall Street and other top recruiting companies will use this.</p>

<p>I am very surprised at those who have said that Peer Assessment is important. It is considered to be the most discredited parameter by top educators in the country.</p>

<p>Would anyone care to expand on their analysis to get at that conclusion besides just saying " Important" ?? ( at Mr Payne, kyledavid80 and Alexandre )</p>

<p>The overlap between top students, top recruitment, and top PA scores is very high. The only notable exceptions are a few old-line schools in the Northeast. Also location still has some influence over recruitment. Many firms focus on the schools in their region.</p>

<p>MovieBuff,
Am I missing something or did you mean your post for me? I share your strong dislike for PA and my posts repeatedly point to its myriad flaws.</p>

<p>"I am very surprised at those who have said that Peer Assessment is important. It is considered to be the most discredited parameter by top educators in the country."</p>

<p>Moviebuff, last I checked, it is precisely "the top educators in the country" who are responsible for the Peer Assessment Score. None of the nation's top 100 universities are complaining about the PA score. </p>

<p>"Are Wall Street and Fortune 100/500 companies going to ABC college to recruit because the school has a top PA or because the students there are terrific? I think it is much, much more the latter at the undergraduate level. The quality arriving on the college's doorstep is already well-screened and Wall Street and other top recruiting companies will use this."</p>

<p>Hawkette, the universities with the top 25 PAs also happen to have the top 25 student bodies. Generally speaking, top universities attract top employers and top students independently.</p>

<p>
[quote]
Perceived quality of the institution is vitally important to employers, as well as grad schools. For example, Wall Street and Fortune 100 companies don't often recruit or offer internships to kids at Podunk U, but send teams to top colleges to pick off thier kids.

[/quote]
</p>

<p>Bluebayou, perceived quality of an institution is indeed vitally important to employers as well as graduate schools. It is even plausible that recruiters ogle the PA as part of their criteria when deciding where to send their swat teams of interviewers --although some schools with low(er) PA do seem to do better than the PA perennial favorites.</p>

<p>But, but, isn't the central question still what the Peer Assessment is SUPPOSED to measure according to the directions of USNews in the context of defining America's Best Colleges. </p>

<p>We have now written and read volumes about this precise question and we are still not much closer in transforming our wild speculations into plausible explanations, except for the knowledge that many academics have admitted using the PA as a strategic weapon, have admitted not possessing the knowledge to rank schools except their own, or have admitted not fill the darn in the first place.</p>

<p>In the end, regardless of how accurate or honest the PA truly is, there will always be two camps. The people who support it because it fits their agenda as it provides an alternative ranking that favors their school (higher PA versus higher ranking.) For example, do we really expect ANYONE associated with Cal to admit that the stratospheric PA is more a reflection more of the reputation of the graduate school and research facilities than of the ... quality and dedication of teaching undergraduates? Do we ever expect Barrons or Alexandre to change their opinions about the PA at Michigan and Wisconsin, except to want an even higher one? </p>

<p>On the other hand, the opposing camp likes to focus on the "reported" gamemanship and is annoyed by the weight given to the subjective and manipulated criteria of the USnews. Fwiw, this camp does not declare the PA to be void of value as much as it being inconsistent and misrepresented. </p>

<p>Since we know that USNews will never drop the PA from its arsenal, the best we can hope would be for two rankings to exist side-by-side. This way, both camps could glance over the one they so despise and focus on their favorite. </p>

<p>For all I care, the reputational survey --and reputational survey is all what the PA truly is-- could fill the first 50 pages of the magazine and take center stage, as long as the rankings based on objective data are maintained for the rest of us.</p>

<p>Alexandre,
Many top educators, including those at some of the nation's top 100 universities, complain about the PA score. The head of either Williams or Amherst was quoted in early summer on this exact topic where it was clear that he did not like what it did. However, in what was probably a wise business decision as his college benefits mightily from the USNWR ranking, he also stated that his institution would go along with it for now and try to effect change from the inside. Frankly, I think you'd have a hard time finding public comments from academics who voted who actually like the USNWR rankings, including the PA scoring.</p>

<p>
[quote]
Moviebuff, last I checked, it is precisely "the top educators in the country" who are responsible for the Peer Assessment Score. None of the nation's top 100 universities are complaining about the PA score.

[/quote]
</p>

<p>Alexandre, wouldn't adding "a small number of " in front of "the top educators in the country" who are responsible for the Peer Assessment Score." be a fairer assessment.</p>

<p>And for what it is worth, assuming that the top 100 universities are all ... universities, are you so sure that schools such as WashU, Emory, Rice, Notre Dame, Wake Forest, and Tufts are particularly thrilled about their PA, and the impact of the PA on their rankings?</p>

<p>And Alexandre, would you accept a small challenge? </p>

<p>Please draw a set of comparables between two closely located engineering schools, namely Caltech and Harvey Mudd, and then use the comparables to explain and justify the differences in their ... Peer Assessment. For this exercise, please disregard the graduate school as this is STRICTLY an assessment of the quality of education at the undergraduate level.</p>

<p>I would happily substitute a purely quantitative faculty ranking for the PA scores along the lines of the Top Amercian Research Universities study combined with something along the lines of the SJTU rankings.
The LAC's will have to come up with their own system. Maybe the number of students who ate dinner at my house index.</p>

<p>I have noticed the following changes in SAT percentile:</p>

<ol>
<li>For most top universities, except Harvard, Princeton and Yale, the SAT 75th percentile has shifted 10-20 points lower than 2007</li>
<li>Even for HPY the SAT 25th percentile has shifted 10 points lower.</li>
<li>For all top LAS, the 25th percentile and 75th percentile have shifted 10-20 points lower than 2007</li>
</ol>

<p>Hawkette, I am sure there are some academics who don't agree with the PA, but by and large, every single top 100 university and top 30 LAC actively participates in the PA, and we both know, actions speak much, much, much, MUCH louder than words. The head of Amherst or Williams may very well not like the PA, but his institutiton still participates in the survey.</p>

<p>
[quote]
I would happily substitute a purely quantitative faculty ranking for the PA scores along the lines of the Top Amercian Research Universities study combined with something along the lines of the SJTU rankings.
The LAC's will have to come up with their own system. Maybe the number of students who ate dinner at my house index.

[/quote]
</p>

<p>Would those TARU and SJTU "studies" happen to focus exclusively on research and predominantly on graduate research? Ah, the surprise!</p>

<p>The "Dinner at your prof house" versus "see your prof through binoculars" would indeed be an interesting index.</p>

<p>Xiggi--when you look at the factual numbers from the top research u's study for those underranked PA schools you see why they deserve pretty much the ranking they have--their faculty is not quite top tier level. I'm sure many of them are fine teachers blah blah blah but a school IS measured by its peers at least somewhat by the stars on the faculty. Otheriwse why would they be fighting so hard to gather more of them on their staffs by going on raids to other schools offering the moon and the sky?</p>

<p>"When the University of Pennsylvania tried to recruit UW-Madison professor Laura Kiessling last year, the pitch was simple.</p>

<p>"They pretty much asked us to tell them what we wanted - and they would give us that," Kiessling said, of the joint offer made to Kiessling and husband Ronald Raines, a fellow UW-Madison biochemist.</p>

<p>With outside offers presenting everything from raises of 30 percent and higher to housing allowances, from tuition waivers for faculty children to millions of dollars in startup money to set up shop someplace else, it's no surprise heads are turned"</p>

<p>
[quote]
every top 30 LAC actively participates in the PA

[/quote]
</p>

<p>Wellesley does not.</p>

<p>alexandre,
Actually it may be that NONE of the top 100 national universities or top 30 LACs responded. This information is not provided and so we have no independent verification of any college's participation. Also, as you know, the response rate is about 50% for 1300+ colleges which means that 600+ colleges did not respond.</p>