<p>I think the PA is basically a proxy for a prestige assessment - college administrators have no clue as to what the quality of education is at any other given college, but they all know which is the most prestigious.... so the people who like the PA like it for the same reason they like the rankings: the whole thing is a system both to report on relative prestige levels and to reinforce them. Which is why if in a given year HYP don't come out on top, USNews will tweak the methodology until they do. And why US News is never going to look for or include useful information such as that reported by the National Survey of Student Engagement.</p>
<p>Personally, other than a quick source of specific data, I found US News totally useless in terms of the college search for either of my kids. It didn't answer the questions they were asking.</p>
<p>And as to the data, US News didn't have anything that I couldn't find elsewhere. I actually found the old IPEDs COOL database more useful. (Unfortunately that database was given a new interface this year that makes it somewhat more cumbersome and less informative).</p>
<p>calmom,
ditto on PA and USNWR's tweaking. I suspect the same thing. As for NSSE, the PA thread on College S&S recently shed some light on this survey and its exclusion so far from USNWR rankings. The data looks very good and relevant for most high schoolers and their families, but the survey size is considerably smaller which limits its use in a national survey like USNWR.</p>
<p>With all the complaints about USN, which schools are grossly overrated or underrated?? For the majors (large universities) I really don't see many examples that are way out of line. You can argue a few spots up or down but overall the results are pretty much in line with most other evaluations. PS--I don't think you'll find the Holy Grail in NSSE. Most students rank their schools failry highly and evenly from school to school. Not too useful for rankings.</p>
<p>that certainly could turn away $52k/yr paying parents. That income group of parents is probably more rank and label conscious than joe six packs.</p>
<p>
[quote]
I think the PA is basically a proxy for a prestige assessment - college administrators have no clue as to what the quality of education is at any other given college
[/quote]
</p>
<p>I'll admit that some administrators' knowledge is limited, but, since some of my best friends are college administrators :) I'll stick up for them. I think lots of academic deans are incredibly informed:</p>
<p>-they talk to their own faculty daily, and many of those conversations involve their own faculty using what peer departments are doing as evidence
-they are involved in retention fights and hiring away faculty from other institutions; the processes are usually very revealing
-the see data from the NSSE, COFHE and other such assessment tools that are not public
-they talk with other administrators in similar positions in a variety of contexts
-nearly any major policy decision will involve institutional research on what data is available from peer institutions
Still, I agree with what what one college president recently said about the PA (I think it was Leon Botstein in the recent NPR series on admissions). "No one intelligent that I know fills that one out." And something like 30% of schools do refuse to fill out the PA.</p>
<p>There's another view of peer assessment, this one in The Atlantic Monthly, by Reed College president Colin Diver:
[quote]
I'm asked to rank some 220 liberal arts schools nationwide into five tiers of quality. Contemplating the latter, I wonder how any human being could possess, in the words of the cover letter, "the broad experience and expertise needed to assess the academic quality" of more than a tiny handful of these institutions. Of course, I could check off "don't know" next to any institution, but if I did so honestly, I would end up ranking only the few schools with which Reed directly competes or about which I happen to know from personal experience. Most of what I may think I know about the others is based on badly outdated information, fragmentary impressions, or the relative place of a school in the rankings-validated and rankings-influenced pecking order.
<p>Using WashDad's analogy of cars and Consumer Reports. If peer assessments are so valuable, why doesn't Consumer Reports ask Ford executives to rate Toyota's cars? Or Mercedes executives to rate Volvo's? </p>
<p>After all, I'm sure the executives at car firms could make the case that they're "informed" about their competitors and can give an honest assessment of both their competitors' manufacturing operations and the experience potential car buyers will get from driving those competitor vehicles. I'm sure they would also give an unbiased assessment of their competitors' reputations. They probably also are involved in "recruitment battles" to lure top talent from their competitors. So, certainly they'd be happy and qualified to give their competitors a "fair" rating, right? ;)</p>
<p>But would you buy a car based on their opinions?</p>
<p>I there were 500 carmakers I think you would get some pretty good idea of who is best. Most U admins are not lifers but have been at a number of schools as students and on their way up. I think they have a broader view than even carmakers.</p>
<p>You make a good argument, but you are misrepresenting what I wrote, which is that the more I know about a subject, the more I develop my own criteria, and the less the opinions of others matter to me. It's true about colleges, and it's true about televisions, toaster ovens, and SUVs. I didn't say anything about peer assessment.</p>
<p>I'm the last one to defend USNews rankings, but find myself in an odd bind. The rankings are deplorable; I think that any principled institution should refuse to fill out the PA question. But since, for better or worse, the rankings exist and a vast majority of schools do provide PA-ratings, some expertise and inside knowledge (which I do belive admins acquire) filters through, however imperfectly. </p>
<p>You see it especially when the PAs are incommensurate with the overall rankings; that suggests that a lot of administrators felt that the other measures misrepresented the quality of the educational experience. I think that the high PAs relative to USNWR ranking of Smith, Earlham, or St. John's, MD (to cite examples from several tiers) probably reflect real esteem for those schools in the ranks of higher education and are not just a measure of prestige or name recognition.</p>
<p>I’d like to try another analogy for Peer Assessment similar to Carolyn’s car analogy. But in response to barron’s response about knowing the top car makers amongst a field of 220, let’s consider a different group. How about handbags? There are literally scores, if not hundreds, of handbag makers in America. As one who knows handbags, I guarantee you that the top execs of handbag companies could not effectively rank other companies. Oh, they would have opinions, but they wouldn’t know if you needed a handbag for day use or evening use. Or maybe beach use. Or how about fashion-they’d have plenty of opinions about what looks good, but some might like feathers, some might like suede, some might like leather, etc. Some might like pockets, others might like no pockets. Some might want over the shoulder, some might like clutch. Some might think you are interested in handbags that are for the ultra luxe set, while other handbag executives might think you mean handbags for the soccer moms. At the end of this whole process, they might have responded to surveys about what is good in handbags, but they could easily all be responding to different things. </p>
<p>And so it goes with college Provosts, Presidents & Deans of Admissions and their responses to the issue of Peer Assessment. It’s an impossible task to measure academic quality in this subjective manner and badly mars what otherwise is pretty good data coming out of USNWR. </p>
<p>P.S. If anybody has any good handbag recommendations out there, I’d love to hear ‘em. :)</p>
<p>Hawkette, my hats off to you. Handbags work better than cars! </p>
<p>WashDad -- sorry, I didn't mean to suggest you'd joined the PA frey -- Your comments about Consumer Reports just triggered the analogy in my mind. :)</p>
<p>Proud Dad, thanks for the good laugh. Now I just have to figure out which handbag has the biggest merit scholarship. :LOL:</p>
<p>Hawkette, no question that a single-number "handbag rating" would not be very useful in choosing an individual handbag. But imagine not ratings of individual handbags, but rather of handbag manufacturers (a better analogy, I think). Such ratings from several scores of industry insiders probably would give you some rough ideas about companies you'd never heard of that produced a good handbag; it would confirm your opinion about others; and it might give you pause about some big names. You could then move on to assessing the fit of their product line(s) for you, the attractiveness of individual bags, and you cold look at the many other pieces of objectve information that are available to you. It's in that limited sense that I think the PAs are useful. (Again, I think that the effects of having rankings are pernicious, but whether particular metrics provide any useful information at all is a separate question). My take is that the PA is probably about in the middle of the pack in terms of providing useful information; probably more useful than # of full-time faculty or alumni giving, for example.</p>
<p>Maybe the $64,000 question is the one raised in Colin Diver's column:</p>
<p>
[quote]
I would end up ranking only the few schools with which Reed directly competes or about which I happen to know from personal experience.
[/quote]
</p>
<p>I guess I naively hope that most institutional responders are ethical enough to not fill out all 200 circles on the PA form. USNews does reveal that 30% of responders refuse to fill out any PA rankings; I'm guess that they're not telling when it comes to the response at the level of rating individual schools.</p>
<p>Not that this is useful to the discussion, but I have a good friend who provides IT support for Kate Spade. Spade, and others in the field, have a big problem with knock-offs. When police raided a huge cache of Spade bag knock-offs the inventory was all sent to Spade after trial. The Spade employees were apparently very impressed with both the quality and the unique styles of the Chinese knock-offs. Now I don't presume to know where an "original" Kate Spade bag is manufactured, but I'd still hesitate to trust any industry's ranking of their peers by their peers simply because I wouldn't trust them to tell the truth. I never heard Kate Spade state publicly that the knock-offs were of better quality than the original and they were going to improve their product (or make and sell them more inexpensively) because of it! :)</p>
<p>While I love the handbag analogy, I suggest we change it once more. To shoes. I think a good fit in shoes is more important than handbags, because a bad fit leads to blisters and other painful ailments. Do you endure a painful fit even if the shoe is made by Manolo Blahnik -- even though the prestige rating of Blahnik is so much higher? Do you really want to pay for a Blahnik when Naturalizer fits better and looks just as nice? And who says Blahnik is so great -- do we really trust Sex in the City? What about a nice comfy pair of sneakers?</p>
<p>The problem comes from thinking of the "PA" as a measure of quality. As a measure of academic quality it has a pretty weak claim to valididty. But, like it or not, part of what folks buy with a college degree is prestige. For this question, the off the top of the head observations a reasonably well informed group of people are a pretty good measure.</p>
<p>I usual read a post before I look at the name of the poster. Yours are very distinctive and always make me smile. I'll go for the comfy sneakers every time and we're willing to pay for the fit of a college, not for its HYPe.</p>