<p>
[quote]
The short answer is both organizations use a survey of peers to rate academic units. The surveys are kept confidential. It really is not very important if the organization is for profit or not. Both use their reputation and good name to add weight to the results. US News has much to lose if they are biased as does the NRC.
[/quote]
</p>
<p>The short answer is the "good name" you speak isn't so "good", and rightly so. Criticism ranging from the Stanford President's open letter criticizing USNWR to articles blasting USNWR to entire groups of institutions defecting from the USNWR process only serves to underscore this growing wave of criticism.</p>
<p>I don't see any Presidents from elite academic institutions writing open letters protesting NRC's ranking. Gee, why not?</p>
<p>Clearly the USNWR ranking has some very vocal critics. So, yes, clearly there is a difference. </p>
<p>If there wasn't, how do you explain all of the criticism of USNWR -- specifically with regards to the Peer Assessment portion? And absolutely ZERO criticism about the NRC?</p>
<p>The fact that one is NONPROFIT and one is FOR PROFIT doesn't play into that at all? You can't even acknowledge this as a remote possibility?</p>
<p>Really?</p>
<p>p.s. you haven't answered the simple question of whether each voters actual vote for in any given NRC ranking is made public or not. and the onus to provide that is on YOU since you claimed that the NRC process and UNSWR process were "virtually the same".</p>
<p>
[quote]
Why are you capitalizing "for profit"--is that indicating profit is some nefarious concept??
[/quote]
</p>
<p>Nefarious? Is an entity operating for profit nefarious? No.</p>
<p>Should an entity posing as an objective provider of information that does so for a profit be questioned and scrutinized particularly if there are some very respectable vocal critics with regards to its lack of transparency?</p>
<p>
[quote]
I am sure we could find an ample number of critics of the NRC rankings.
[/quote]
</p>
<p>Really? Please show us these ample critics.</p>
<p>I haven't heard of a single one. For instance, is there a single educational institution that has requested to be excluded from the NRC rankings? or has refused to cooperate with the NRC?</p>
<p>barrons,
I think you again give way too much weight to the views of academics. To be honest, I had never even heard of the NRC until I came to CC. It just is not on the radar screen of the vast, vast majority of college students, familiies and alumni. </p>
<p>If you want an assessment that would have real value to the public, give the vote to all of the stakeholder groups-students, alumni, employers-and let them give their opinions of the faculty at a college. That is where the rubber meets the road, not how many awards a certain professor or department has won or how many articles get publlished or other "inside baseball" type measurements.</p>
<p>
[quote]
I suggest you use the Google--you will find numerous critics.
[/quote]
</p>
<p>Why should I? The onus is on you to provide these "ample" number of critics.</p>
<p>
[quote]
Here's a starter.
[/quote]
</p>
<p>Who??? David Fenton? Who is this nobody? Is that you Barrons? </p>
<p>Again, has a single institution refused to cooperate with the NRC? Plenty of schools have defected from the USNWR process.</p>
<p>Clearly if you have to resort to desperate Googling to support your bravado-filled claims you are seriously lacking substance. Any monkey can Google.</p>
<p>Besides, what exactly is your argument? Aren't you arguing on my side?</p>
<p>In other words, if you dig up criticism against the NRC and the NRC's methodology is = to USNWR's (as you claim), then, aren't you basically confirming that the critics of USNWR are correct?</p>
<p>Isn't the biggest difference between the NRC and USNWR the frequency with which they publish their results? Correct me if I'm wrong, but it looks to me as if it takes about three years for NRC to sift through its own survey data. I think people would give the USNews surveys a lot more credence if they weren't rushed out year after every year. No wonder they have only a 58% response rate. Much of the work that goes into filling out those questionnaires has to be redundant.</p>
<p>hoedown,
I'm not claiming that higher PA scores make for a bias. In fact, I content that public disclosure of the voting would remove any bias that might now exist (who is giving the low scores to HYPSM to give them a 4.9 and not a 5.0?). </p>
<p>Further down the ranks, I suspect there is some manipulation going on and we'd find out how some highly regarded schools are getting some surprisingly low scores. A public disclosure of the votes would reveal if there is bias going on. </p>
<p>Consider the following Top 30 USNWR schools and their PA scores:
4.1 Wash U, Rice, Vanderbilt
4.0 Emory
3.9 Notre Dame, USC
3.7 Tufts
3.5 Wake Forest </p>
<p>Please also note that all of these schools (except Tufts) are located outside of the Northeast. Is the teaching and the faculty at these schools really that inferior to that going on at schools in the Top 15 (none of which have a PA less than 4.4) or is this just inherent favor being shown those schools that have been around longer and whose names are better known? Do the rankers even know about the teaching quality at ANY of these schools? They may, they may not, but how do we know? </p>
<p>I accept that the college football coaches know a lot about the other teams (although I suspect their familiarity and knowledge about teams in other regions/conferences is lower than generally recognized), but the analogy is not perfect because all of the schools play the same game (football) while the PA graders are judging across a wide range of academic subjects and are being asked to make distinctions between undergraduate and graduate programs. But the football coaches make a public disclosure. Furthermore, there are other polls that serve as a check on the opinions of the coaches. This is sort of how I see the use of non-academic polls, ie, rankings done by other stakeholder groups. </p>
<p>Why should academia be exempt from having outsiders give opinions or at least have the ability to see if graders are being fair with their marks? This strikes me as one more example of academia wanting to be accountable to no one but themselves. This serves the interests of the academic world and the established status quo, but is it accurate and does it serve the public interest? I would argue NO.</p>
<p>The NRC is a careful scholarly study. They did an extensive public review of their survey and evaluation methods as a prelude to the current round of rankings. As part of this, they invited comments on and criticisms of their prior approach and formed a committee of scholars to recommend revisions and improvements. They published this document and are implementing the recommendations for the new survey. The decisions on how to proceed were made by scholars interested in the reliability of the results, rather than editors interested in selling magazines. This is about as far from the USNEWS approach as one can get.</p>
<p>
[quote]
I think you again give way too much weight to the views of academics
[/quote]
</p>
<p>Yeah, they are college professors. What could they possibly know about the education and outcomes of colleges? I would much rather ask random people on the street what they think of the art history department at Yale, or computer science at Case.</p>
<p>You are probably right that many people have never heard of the National Research Council. That is evidence of why one should have little interest in the opinion of these people about the quality of colleges.</p>
<p>In fact, the next time I get on an airplane, it would be a great comfort to me to know that the design team consisted of the first dozen regular folks who walked in the door of the nearest McDonald's, rather than a bunch of over educated aeronautics engineers.</p>
<p>afan,
I think you have completely missed my point. I am arguing for a public disclosure of the votes of those in academia. I am not saying that we should ignore what they have to say.</p>
<p>I am also recommending that other groups opine on the effectiveness of the faculty. Students, alumni, and employers have all sampled the product from different perspectives and many would have valid and valuable contributions to make. </p>
<p>To use your airplane analogy, consumers do have an understanding of how effectively the aeronautic engineers designed the plane and daily experience the performance and environmental differences in how various planes are designed. Aren't the passengers every bit as able to comment on the quality of the airline product that they experience as the aeronautic engineers? Can't the airline pilots comment on the engineering that went into the design of the plane? Can't federal regulators have a view on the safety of various aircraft designs (remember the DC 10 controversies) or are the engineers at Boeing the only ones who can have an opinion? </p>
<p>Academia looks like a monopoly to me and surveys like PA and the NRC only perpetuate that. Is having only one group decide serving the best interests of the consumer, ie, the student taking the classes, the family paying the bills, the alumni who has the degree and makes contributions, the employer who hires the students???? I'm not saying shut them out-they can make a very valuable contribution, but so can many others and I would, in many cases, trust the judgments of students/alumni/employers over academics.</p>
<p>USNEWS is nonsense, so I will confine the following to the NRC. </p>
<p>There is no monopoly. The whole way the study is constructed prevents that. Look at what they actually do and this is apparent in their methods. </p>
<p>They do not attempt to tell anyone "what is in the best interests of the consumer". The NRC attempts to capture an estimate of the quality of graduate departments. Going to a graduate department with a higher academic profile may or may not be in the best interests of a given student.</p>
<p>Are passengers qualified to comment on the quality of airplane engineering? No, they have not a clue. They can comment on how comfortable the planes might be, but they have no idea whether the differences are due to engineering, design, construction, maintenance, airline policies, airport services, government regulations...</p>
<p>Pilots have far more expertise, and in some cases they could identify the causes of variations in airplane quality as due to engineering as opposed to other factors. Of course, to become an airline pilot takes years of highly specialized training and experience, as well a substantial technical knowledge of airplanes. Among the things some pilots do is teach new people how to be pilots, and evaluate those who are entering or moving up in the profession. Asking pilots is way different than asking the random Joe what he thinks about an airplane or a college. In fact, it is much like asking graduate school faculty what they think of graduate school programs.</p>
<p>When the government investigates airplane disasters it relies heavily on engineers. As far as I know, surveying the person on the street for technical advice on aircraft safety is not part of the process.</p>
<p>Random Joe may have an opinion, and he may tell me if I ask, but that does not mean that hearing his response will be informative.</p>
<p>Afan,
As you can easily see, I am not an engineer nor an academic. But I am a consumer and I can sample the product and I can make observations. For example, there is more than just the mechanical engineering that goes into the plane. There is also the human engineering that goes into the design and comfort features when one is flying and the environmental engineering that goes into sound management. Do you believe that only engineers can comment on this? </p>
<p>There are operational issues related to the underlying mechanical engineering and don’t you think that pilots can comment on the quality of this underlying engineering even if they themselves did not design the aircraft? And when the government inspects an airplane, I certainly would hope that they have an individual with engineering knowledge doing the inspections or the investigations, but I also would expect that engineer NOT to be associated in any way, shape, or form with the manufacturer of the product. BTW, the FAA and the Department of Transportation are run by non- engineers. The current head of the FAA is Marion Blakey who studied International Relations at Mary Washington and the head of the DOT is Mary Peters who got a degree in management from the University of Arizona. In your construct where the engineers doing the mechanical design would have the vote, the top administrators would not be allowed to have an opinion.</p>
<p>while your mini-dissertation on airplane engineering was quite fascinating, let's get down to brass tacks:</p>
<p>do you believe that inherent bias does not exist in a methodology incorporating subjective voting? Bear in mind that this "subjectivity" could very well be extremely informed, but it is subjective nonetheless. And where there is subjectivity, there is room for bias, and where there is room for bias there is room for fudging, esp. if the collector of that data is also the distributor of that data and there is no way to check the veracity of that data... oh, and they also do it for a profit (i.e. USNWR).</p>
<p>in the end, i think what people are saying is that the Peer Assessment is fundamentally flawed in so many ways:</p>
<p>1) Inherent bias
2) Impossibility of "knowing" / having an expert opinion on dozens and dozens of colleges
3) Even if they did, it's not even their job to know, so why should we care about the Chancellor of XYZ's opinion on the relative merits of a Dartmouth vs. University of Wisconsin... opinions are just opinions they are like a-holes, everyone has one.
4) The lack of transparency (who actually is voting, how did they vote?)
5) No way to keep USNWR "honest", no way of double checking their numbers</p>
<p>In sum, just stick to verifiable facts and there will be less scrutiny, less criticism.</p>