Peer Assesment Rank

College Search & Selection
161

<p>
[quote]
My frustration is that the present sample of opinions, in many cases, is limited to only one (protected) group and ...

[/quote]
</p>

<p>A lot of the charges you've leveled so far have been at 'academics' and I haven't challenged that because most of the provosts and presidents probably are academics. However, I think it is less usual for admissions directors to be academics, and they are 1/3 of the surveyed population. So here I think you're wrong to say it's limited to "one group" (if you mean that one group is academics).</p>

162

<p>Pretty picayune stuff. Well, use whatever term you like, but they certainly are directly affiliated with and receive their compensation from academic institutions.</p>

163

<p>Hawkette, there is a huge difference between "academics" (professors, deans, provosts, college presidents) and administrative staff.</p>

164

<p>
[quote]
Pretty picayune stuff. Well, use whatever term you like, but they certainly are directly affiliated with and receive their compensation from academic institutions.

[/quote]
</p>

<p>My apologies; this is a usage that is quite distinctive on college campuses. As Momwaitingfornew indicates, "academic" has a specific meaning, at least among those affilated with institutions of higher learning--it means faculty and/or people who have scholarly degrees. I'm sorry if this is unfamiliar to you and seems picayune. Elsewhere you've seemed an advocate of specificity (i.e. disliking the vagueness of "distinguished" in the PA survey). </p>

<p>Hopefully others reading will also forgive the nitpick.</p>

<p>ETA: I guess this also raises the question--where you've accused academics of unflattering traits, you've all along meant admissions directors as well? I haven't challenged some of those statements because I thought you meant only provosts and presidents, and some of the criticisms about lack of knowledge didn't seem as far-fetched to me. I mean, I didn't fully agree, but understood the basis for the accusations. If known you'd been including admissions directors in that group, I may have spoken up earlier. In my experience, the admissions professional has different sorts (and sources) of knowledge about undergraduate education at its peers. I'd suggest that these people don't deserve the same kinds of criticism. Maybe that's immaterial (perhaps they deserve just as much, but of a different stripe, and that's really more a tangent for you)</p>

165

<p>For the purposes of this discussion, the important fact to me is who pays you. Is it a college or university in competition with its peers? Is that where your bread is buttered?
If you talk about "Detroit", it doesn't matter to most of us whether the person is an engineer or someone in the finance department.
Actually teaches or does research? Hires those that do?
Who cares!</p>

166

<br>


<br>

<p>So, wait. You're going to take a survey that you've been criticizing for about twelve pages, now and "fix" it by adding more groups, even further removed from recent college experience (alumni and employers) and students (no, bias there)? If you're so distrusting of academics why not just put the Princeton Review on a pedestal? They do that sort of thing already.</p>

167

<p>Hawkette, I look forward to the day that corporate America is asked to chip in its $0.02 worth. But I fear you will again dismiss it as being ridiculous and off-base (much as you do with the Peer Assessment Score) when schools you don't feel deserve respect for whatever statistical reason beat out schools you feel are more worthy.</p>

168

<p>johnwesley,
I favor the elimination or separating out of PA scoring as the first option. However, if USNWR has to include some subjective-based measurement, I want it to incorporate a broader spectrum of opinion than just those in the academic community. </p>

<p>I would like to present the impact of PA scores on the rankings published by USNWR. Here are most of the schools in the USNWR Top 50 that would be most affected by the elimination of PA scoring.</p>

<p>MOVE DOWN THE MOST</p>

<p>10 places from 24 to 34 U Michigan</p>

<p>8 places from 9 to 17 U Chicago
8 places from 34 to 42 U Wisconsin</p>

<p>7 places from 21 to 28 UC Berkeley</p>

<p>6 places from 4 to 10 Caltech
6 places from 41 to 47 U Illinois</p>

<p>MOVE UP THE MOST</p>

<p>10 places from 20 to 10 Notre Dame
10 places from 44 to 34 Yeshiva</p>

<p>7 places from 30 to 23 Wake Forest</p>

<p>6 places from 12 to 6 Wash U
6 places from 27 to 21 Tufts</p>

<p>5 places from 8 to 3 Duke
5 places from 15 to 10 Brown</p>

169

<p>^^^So, just like the LACs (perhaps not as pronounced), you'd wind up with a lot of ties (all around the #10 spot.)</p>

170

<p>hawkette, how did you come up with your estimates of which schools move up and down and by how much? Do we know details of UsNews PA rankings?</p>

171

<p>ramaswami,
The rankings ex-PA have been posted many times over the past year by various posters and I drew from their work. Most colleges did not have large changes, but as noted above, some change a lot, eg U Michigan and Notre Dame. </p>

<p>Here is the complete listing for the USNWR Top 50:</p>

<p>Rank with PA, Rank without PA, Change, College</p>

<p>1, 1, 0, Princeton
2, 1, 1, Harvard
3, 3, 0, Yale
4, 10, -6, Cal Tech
4, 6, -2, Stanford
4, 6, -2, MIT
7, 3, 4, U Penn
8, 3, 5, Duke
9, 17, -8, U Chicago
9, 6, 3, Dartmouth
9, 10, -1, Columbia
12, 6, 6, Wash U StL
12, 15, -3, Cornell
14, 10, 4, Northwestern
15, 10, 5, Brown
16, 17, -1, J Hopkins
17, 15, 2, Rice
18, 17, 1, Emory
18, 17, 1, Vanderbilt
20, 10, 10, Notre Dame
21, 28, -7, UC Berkeley
21, 23, -2, Carnegie Mellon
23, 21, 2, Georgetown
24, 34, -10, U Michigan
24, 23, 1, U Virginia
26, 28, -2, UCLA
27, 27, 0, USC
27, 21, 6, Tufts
27, 28, -1, U North Carolina
30, 23, 7, Wake Forest
31, 28, 3, Brandeis
31, 34, -3, W & M
33, 23, 10, Lehigh
34, 34, 0, Boston College
34, 38, -4, NYU
34, 28, 6, U Rochester
34, 42, -8, U Wisconsin
38, 38, 0, UC SD
38, 42, -4, Georgia Tech
38, 33, 5, Case Western
41, 47, -6, U Illinois UC
42, 41, 1, Rensselaer
42, na, na, U Washington
44, 46, -2, UC Irvine
44, 38, 6, Tulane
44, 34, 10, Yeshiva
47, 51, -4, UC Davis
47, 42, 5, UC S Barbara
47, 48, -1, U Florida
47, 50, -3, Penn State
47, na, na, U Texas
52, 42, 10, Syracuse</p>

172

<p>thanks, hawkette.</p>

173

<p>"9, 17, -8, U Chicago"</p>

<ul>
<li>......... :rolleyes:</li>
</ul>

<p>.... Needs more library expenditures....</p>

174

<p>momwaitingfornew, you seem to identify research universities as science universities. All these universities have strong humanities depts where research happens: do you think psychologists and linguists and historians don't do research? Your post reveals a fundamental misunderstanding about research or it is badly phrased. Yale and Harvard English depts are doing a tremendous amt of research into Middle English ,etc.</p>

<p>I often get a response that goes like this: even in research univs there are profs that love to teach, etc. Of course. But the point is in LACs nearly 100% of the profs will be teaching and most will be teaching 80% of the time or thereabouts. At research univs and univs that have grad depts 40% direct teaching will be a stretch. Also, in LACs class sizes are very small so teaching really happens. AT the large research schools in huge lecture halls with 100 + students it is not teaching but theater.</p>

175

<p>Rama,</p>

<p>You are probably aware that the answers you got would depend a lot on what you asked. </p>

<p>If you asked them to tell you how good the undergraduate programs were at places they had never worked, them I respect them for being cautious. </p>

<p>If you had asked for the top professors in each respondent's area of expertise, they probably could have told you. Since they had no idea who you were, or whether you were telling the truth about your interest, they might have remained careful in what they said. However, in a trusted survey, they could have answered. </p>

<p>Hawkette, </p>

<p>I still don't get it. You disagree with the rankings generated by the PA survey, but you don't seem to have any basis for saying that certain colleges are ranked higher or lower than they should be. Of course, by chance alone, if you change the group of raters you will change the rankings, but that hardly establishes that the new ranking will be any more valid than the old. </p>

<p>The PA tells you what answer you get if you ask this question of a set of presidents, provosts and admissions deans. You may not agree, for obscure reasons, but that is what these people think. </p>

<p>If the rank with PA is meaningless, then the rank without is equally meaningless. Who says the other factors included in USNEWS should be used at all? Where is the evidence that the weights are appropriate? What is the basis for making all the weights linear? Why are there no interaction terms? Answers to all these questions is the same "no evidence, no basis, they just made it up" From a methodologic point of view, this is worse than a mess. It is trash. </p>

<p>It is pointless to try to rank colleges, doubly pointless if you cannot agree on what characteristics count or how much.</p>

<p>Now the real question "So what?"</p>

176

<p>
[quote]
Here is the complete listing for the USNWR Top 50:</p>

<p>Rank with PA, Rank without PA, Change, College</p>

<p>1, 1, 0, Princeton
2, 1, 1, Harvard
3, 3, 0, Yale
4, 10, -6, Cal Tech
4, 6, -2, Stanford
4, 6, -2, MIT
7, 3, 4, U Penn
8, 3, 5, Duke
9, 17, -8, U Chicago
9, 6, 3, Dartmouth
9, 10, -1, Columbia
12, 6, 6, Wash U StL
12, 15, -3, Cornell
14, 10, 4, Northwestern
15, 10, 5, Brown
16, 17, -1, J Hopkins
17, 15, 2, Rice
18, 17, 1, Emory
18, 17, 1, Vanderbilt
20, 10, 10, Notre Dame
21, 28, -7, UC Berkeley
21, 23, -2, Carnegie Mellon
23, 21, 2, Georgetown
24, 34, -10, U Michigan
24, 23, 1, U Virginia
26, 28, -2, UCLA
27, 27, 0, USC
27, 21, 6, Tufts
27, 28, -1, U North Carolina
30, 23, 7, Wake Forest
31, 28, 3, Brandeis
31, 34, -3, W & M
33, 23, 10, Lehigh
34, 34, 0, Boston College
34, 38, -4, NYU
34, 28, 6, U Rochester
34, 42, -8, U Wisconsin
38, 38, 0, UC SD
38, 42, -4, Georgia Tech
38, 33, 5, Case Western
41, 47, -6, U Illinois UC
42, 41, 1, Rensselaer
42, na, na, U Washington
44, 46, -2, UC Irvine
44, 38, 6, Tulane
44, 34, 10, Yeshiva
47, 51, -4, UC Davis
47, 42, 5, UC S Barbara
47, 48, -1, U Florida
47, 50, -3, Penn State
47, na, na, U Texas
52, 42, 10, Syracuse

[/quote]
</p>

<p>No wonder Alex is such a Peer Assessment supporter. </p>

<p>No other school stands to lose more than UMichigan if it were eliminated. It would drop a whopping 10 spots.</p>

177

<p>^^^^^ouch!!</p>

178

<p>What nonsense. Wash U no change? Ridiculous. Where did you get this analysis? As far as I know, USN does not rank without PA. No number is more valid than PA, where schools are rated by experts in the field. Everything else worthless without PA. It is frankly, the only number I care about.
Top doctors are ranked by experts in that field, as are top attorneys, etc. Who else should inform the public about the status of a school's respect? You?</p>

179

<p>"No number is more valid than PA, where schools are rated by experts in the field."</p>

<p>"Top doctors are ranked by experts in that field, as are top attorneys, etc. Who else should inform the public about the status of a school's respect? You?"</p>

<p>-How do you know that any of what you said is true???? Have you seen who responds on the PA surveys- and how they respond?</p>

180

<p>I am aware of some of the reasons why the profs would have answered the way they did. I set up phone interviews, actually some returned calls to my office phone (it is a psychiatric crisis service), we developed a level of comfort, 2 of them taught my son at Cornell summer school. I believe they trusted me more than a "trusted survey". Afan, if you are referring to USNews it is not held in high regard in academe. Yes, some of the profs knew other top profs in their fields. But my point is this: they did not have an intimate knowledge of what went on in the classrooms. They made overarching comments such as the east coast ivy league engineering schools are European in their approach to engineering, Stamford is faddish, MIT follows federal money, etc. I asked some pointed questions such as how fluid mechanics is taught at X vs Y vs Z univ. I sampled researchers, those who taught more actively, and those who were administrators. The administrators seemed to know their counterpart institutions but they were mainly non engineers and did not know of teaching quality in engineering when pushed. They would begin by grand comments about how XYZ school did well by ABET requirements but when I probed further they would confess that their expertise was in anthropology or sociology of education etc and not in subject matter. They would then direct me to a prof who knew particular peer depts but this person when probed would say he had received well trained grad students but would not be able to elaborate on the training. Some of them said that Swarthmore engineers were as good as MIT engineers etc. IN any case I was not impressed. 17 of the 23 I spoke with had not visited more than 2 peer instituitions in the last 5 years but they all said they had filled out USNews rankings on 20 to 30 institutions mainly by their impressions but could not say where their impressions came from.</p>