UF President cooks USNews Rankings

[hoedown]

<p>

They have not devoted their entire professional careers to the subject of knowing the differences of other colleges located hundreds of miles away from them as you imply. They work in administration.

</p>

<p>I get what you’re saying in general, and there is some truth to it.</p>

<p>But I think you are underestimating how much time and attention administrators like Machen have had to devote over their administrative careers to issues like peer comparisons, who is hiring what faculty away, who is enrolling top cross-admits, and so on. They also have colleagues at many peer schools, and they stay in touch and network. </p>

<p>I have worked for a number of provosts and other high-level administrators and they tend to know quite about about the “quality” (perceived or otherwise) of other schools. Some of it is pretty good insider information, actually. LOL.</p>

[coollege]

<p>Kind of silly to have schools be able to rate themselves. Should include the “can’t vote for yourself” rule like we did in grade school. But you could still screw with other schools’ score.</p>

[xiggi]

<p>

@ClintonBK

They are certainly better positioned to know more than the self-appointed amateur experts on CC who bloviate endlessly on the very same subject, yet insist with equal certitude that no college president or provost—smart and competent people who have devoted their entire professional careers to the subject, and whose reputations and livelihoods rise or fall with their making correct judgments on it—could possibly know what the casual part-time outside observers know. **Such utter horse manure. Such self-importance. Such arrogance. Such hubris. Sometimes CC disgusts me. **

</p>

<p>What a wonderful statement!</p>

<p>Does it stem from the absence of possible rebuttal to what the self-important and arrogant posters who excel in posting horse manure have been writing for a long time? You, as a member of the academic world seem to expect the “self-appointed amateur experts on CC” to accept your arguments at face value. </p>

<p>The issue is partly about what the experts do know, but mostly about their integrity. Nobody here denies that the “experts” do know a LOT about their schools, SOME about their direct competitors, and just a BIT out the entire universe of USNews. Twist is as you want but a President who offers expert knowledge about every one of the schools on his survey DOES lose credibility. The alternative exists, namely to fill the last answer block! </p>

<p>However, the answers that are blatant attempts at manipulating the outcome of the survey for “competing” schools illustrate the lack of integrity the “horse manure” peddlers have been decrying for a long time. And THAT part is inexcusable, or at least should be!</p>

<p>Perhaps it is the error of the simple “peons” who look at the majesty of the world of academia from the outside to cling to the notion that it is still a world anchored in truth, integrity, honor, and selflessness. Your spirited defense (and denial) of the depth of this breach of trust by senior officials speaks volumes about your views on this issue. And this forces me to agree with you on the last sentence, “Such utter horse manure. Such self-importance. Such arrogance. Such hubris. Sometimes CC disgusts me.” Me too!</p>

[barrons]

<p>To give you an idea of how top U officials share info I know one MAJOR state U top admin sent a copy of his plans to deal with budget cuts to a similar level admin at a peer school. He was probably looking for ideas about how to handle things.</p>

[bclintonk]

<p>I’ve now gone through and compared Machen’s PA ratings for all 263 schools he was asked to rate, and compared them to the 2009 US News PA scores for these same schools. I have to say, I cam away impressed. Not only are the vast majority of Machen’s scores well within the mainstream of the industry as a whole, but I think they evidence some thoughtful judgments. I don’t agree with all his judgments, and a small percentage of the scores seem to be off base—obviously not enough to significantly damage the schools he low-balls, or to significantly help those he gives high marks. But the pattern is pretty revealing.</p>

<p>Consider this. Some CCers criticize Machen for giving Michigan where he previously served as provost, a 5 while giving the University of Chicago only a 4. But in both cases he’s well within the mainstream of PA raters. Michigan’s overall PA score is 4.4. That means at least 40% of PA raters gave it a 5—as did Machen. </p>

<p>Think about it this way: to get an average score over 4, some significant fraction of the individual scores had to be 5s. If all the scores were 4s and 5s, then 40% of the scores were 5s and 60% were 4s. But we can probably safely assume there were a few 3s, possibly even a few lower (especially if any presidents, provosts, or admissions deans hold the kind of virulent anti-public bias we hear from some on CC). But if some of the scores were lower than 4, then an even larger percentage must have been 5s to keep the average at 4.4. Bottom line: Machen’s 5 for Michigan was well within the mainstream, as at least 40% of PA survey respondents agreed with him. Also note that even if he had gone with the majority and given Michigan a 4 instead of a 5, it’s extremely unlikely that would have made any difference in Michigan’s average PA score. Three people at each of the 263 schools, or 789 persons, were invited to respond. We’re told the response rate is just under half, so let’s assume about 350 actually responded. That means Machen’s vote comprises 1/365th of the total—just a little over 2/10 of 1%. On overage it’s going to require a change of about 35 votes even to move just one tenth of a point, e.g., from 4.4 up to 4.5, or 4.4 down to 4.3.</p>

<p>What about Chicago? Machen gave it a 4; its overall PA score was 4.6. Again, Machen’s in the minority, but well within the mainstream, as we can assume roughly 40% of PA voters gave Chicago a 4 or lower. And again, the likelihood that his individual vote is the tipping point in any movement in Chicago’s PA average is practically nil. </p>

<p>What about Notre Dame? Didn’t he hose them? No, not really. Machen gave them a 3. Their average PA was 3.9. That means at least 10% of PA votes agreed with Machen and gave Notre Dame a PA score of 3 or lower; probably more, because views on Notre Dame vary widely, so there were probably some 5s as well, meaning there were probably more than 10% at 3 or lower to result in an average of 3.9. You might disagree with the score; I do, I’d give them a 4. But that’s beside the point. This score isn’t an outlier, it’s at a still fairly fat part of the tail of the curve in a normal distribution.</p>

<p>What about everyone’s favorite atrocity story, Brown? Machen gave them a 3; their PA average is 4.3, so Machen’s score falls a full 1.3 points below the average. i’ll admit to being quite surprised by this. I’d have given Brown a 4 or a 5, and most PA voters apparently agree with me. But once again, in fairness this has to be seen as another “tail of the curve” story, except Machen’s farther out on a narrower part of the tail on this one. does it reflect ignorance, carelessness, or inattention? I doubt it. I suspect he has his reasons. I especially think that because he scored Dartmouth–in many ways a similar school, smaller and more LAC-like than other research universities, not at all fitting the model of a large, multi-dimensional, strengths-in-all-disciplines university that many people, including Machen, have in mind when they think of a research university. Machen’s view of these schools, although not a true “outlier,” is pretty far out on the tail of the curve; but there’s also a sense in which these schools are themselves pretty far out on the tail of the curve for what even counts as a research university (or “national university” to use USNWR’s parlance). So perhaps it shouldn’t be too surprising that there’s the occasional mismatch in perceptions. But clearly Machen’s view doesn’t hurt them much, as the PA scores of these schools hold up pretty well without Machen’s help.</p>

<p>What about Georgetown? Hosed? Not really. Georgetown’s average PA score was 4.0, but with that average you wouldn’t expect 100% 4s, you’d expect perhaps at least 10 to 20% 5s and a roughyl equal number of 3s. Another tail-of-the-curve story.</p>

<p>Sounds like too many “tail of the curve” stories? I don’t think so. Out of the 263 or so schools Machen rated, there are only a tiny handful like this—in short, the number of stories like this itself reflects the “tail of the curve,” not the predominant story. In only 16 cases was Machen’s score more than 1 PA point away from the school’s PA average. Two of those he scored on the high side—Florida and Utah, the two schools where he’s been President, reflecting an obvious but harmless hometown bias.Three others were in his current home state, where he scored Florida Atlantic, Florida International, and Florida Institute of Technology at 1, whereas each had an overall PA average of 2.1. Hard to say he was egregiously wrong in any of these cases, and he may have had better inside information than the average PA rater, who did not hold these school in high regard in any case. The rest? Brown and Dartmouth we’ve already mentioned. Machen gave Pepperdine a 2 v. its 3.1 average. George Washington, 2 v. 3.4; Clark Atlanta, 1 v. 2.2. Boston College, 2 v. 3.5. Carnegie Mellon, 3 v. 4.1. Lehigh, 2 v. 3.2. Clemson, 2 v. 3.1. Memphis, 1 v. 2.3. Baylor, 2 v. 3.1. Tail of the curve to be sure, but not wildly aberrant, and mostly consistent with the “not-quite-full-strength-university” story I told about Brown and Dartmouth. </p>

<p>Again, you may think he’s wrong, as do, apparently, most people doing the PA scoring. But I’m impressed with the degree to which his views are broadly in accord with the mainstream views of academic administrators—except on this question which affects only a few schools in the survey and certainly doesn’t come close to biasing the results against them, since Machen’s got only one vote and on this question may be a minority of one. Broad consensus doesn’t mean the PA ratings are “right,” of course. But it does suggest they may do a pretty good job of doing what they purport to do, namely to reflect the relative standing of academic institutions in the eyes of their peers.</p>

[dstark]

<p>“They are certainly better positioned to know more than the self-appointed amateur experts on CC who bloviate endlessly on the very same subject, yet insist with equal certitude that no college president or provost—smart and competent people who have devoted their entire professional careers to the subject, and whose reputations and livelihoods rise or fall with their making correct judgments on it—could possibly know what the casual part-time outside observers know. Such utter horse manure. Such self-importance. Such arrogance. Such hubris. Sometimes CC disgusts me.”</p>

<p>Hmmm. Looks right to me.</p>

[parent2noles]

<p>Siserune:

How, exactly, would this easy group-based “cooking” work? Collusion carries large reputational risks and is vulnerable to secret defections (Prisoner’s Dilemma). Increasing the challenge, any would-be conspiracy is small compared to the total number of raters, and has to compete against other possible conspiracies negating its effects.

Others have already posted that US News in particular has garnered influence far beyond what the authors likely intended - or deserve. This has placed pressure on the respondents that I’d suggest they should not have. One possible scenario would be where the number of respondents is self-limited due to lack of information, as in where someone from CA declines to rate UF because they don’t know anything about it. This weakens the strength of a large group, which you correctly point out lends reliability to this type of assessment. So, while the top-20 schools are likely very well assessed, schools like UF, Clemson and USC, for example, have an opportunity to essentially promote themselves, correctly or not, and also a way to diminish their competition. You could call this marketing, competition, university politics, collusion or whatever but we all know it exists. Since many students/alumni/faculty move during their career, they carry the message forward, pro or con, conveying certain positions regarding their school and their competitors. It may be secret, it may be overt, it may be opportunistic or planned, but it is very plausible. Over time, aspiring regional schools especially can be helped or hurt by this propagated perception, what ever it may be. This human information is what the PA is designed to ferret out. By the way, it occurs to me you suppose I was only referring to the PA when I wrote about cooking the ratings…no. I mean the entire system can be and is gamed.</p>

<p>

The main point of the Peer Assessment, mushy as it is, is that it is less susceptible to manipulation than the objective measures. USC can hire as many 94-year old Nobel Prize winners as it wants, and no university president will rate it the equal of MIT. The PA stabilizes the rankings by adding extra information that is hard to distill from a small set of numbers, and helps to indicate whether the objective part of the ranking metric makes sense.

I agree. There is a place for peer assessments, but I’d say the current system needs to be improved, such as not being able to rate your own school, only rating schools where you have no vested interest or history (such as employment) and maybe making all responses public and traceable to the respondent. I’m sure there are many ideas with merit that could be found.

I think a lot of university presidents may automatically give their schools the maximum rating just on principle.

What value does self-reporting inflated scores have for the PA? You may think this, but in Florida this seemingly simple act caused Machen to have his integrity very publicly questioned. The question then becomes if he exaggerates this, what else is untruthful? While school spirit is fun and creates a lot of support for something like a university, there is a limit to actions that erode the credibility of the institution.</p>

[bclintonk]

<p>

@gellino

It is NOT their job to know the difference between Tulane and Brown and their ignorance shows. They have not devoted their entire professional careers to the subject of knowing the differences of other colleges located hundreds of miles away from them as you imply. They work in administration. I would say the avg CCer applying to or attending the top schools probably has a much deeper understanding of the relative merits of Tulane vs Brown than the avg college president, who has a much more narrow focus.

</p>

<p>I couldn’t disagree with you more. With rare exceptions, people get to be college and university presidents and provosts by working their way up from the ranks of academics—serving on faculty hiring committees in their own department and school, chairing hiring committees and running competitive national searches for the best in their field, serving as department chairs and deans responsible for maintaining and strengthening their departments and schools relative to their peers, participating in larger school- and university-wide strategic initiatives and strategic planning, becoming as provost the “dead of deans” responsible for keeping all academic disciplnies strong, then finally as president, usually moving up from provost either at the same school or at another school. These are exercises in comparative benchmarking at every step of the way. You simply can’t do any of those jobs without keeping a constant and comprehensive eye out for where the best talent is, and how your faculty’s talent stacks up against not only your closest competitors and peers, but more broadly within the full array of potential competitors out there. It’s a highly competitive, even cutthroat business. These people don’t spend their lives squirreled away in a dark office with green eyeshades poring over musty spreadsheets—though there is some of that, too. Basically, they spend their every waking minute trying to figure out how they can get a leg up against their competitors in the business. and that requires watching their competitors like a hawk, tracking their every move, assessing what works and what doesn’t work, and which parts of it you might be able to do better, cheaper, faster, You bet your bottom dollar they know what’s going on at other schools. In some ways, that’s their main function.</p>

<p>That said, I’ve said before that what they watch most closely, can see most clearly from a distance, and feel they have the capacity to affect most directly, is the quality of the faculty, discipline by discipline, department by department, school by school. They know who are the rising stars and who are the aging deadwood; who’s made the best recent hires and who’s suffered the most devastating losses to retirement or lateral moves; who may be movable and who’s not; who’s “underplaced” at a weaker school relative to their own individual standing and might be interested in a lateral move; which of their own stars they need to protect against poaching by hungry competitors or more pretigious schools looking to replenish their ranks, and which of their own people have less market value. Bottom line, in their world building and maintaining the strongest faculties is the ultimate measure of success. That’s their obsession. That’s what I think is primarily reflected in the PA score. But since I happen to believe a strong faculty is essential to a strong college or university, and since there’s nothing else in the US News ranking that comes remotely close to measuring faculty strength, I can accept the PA rating as a crude but broadly accurate proxy of the cumulative wisdom of college and university presidents and provosts on this question. (I’ve also said before, however, that I don’t think deans of admissions should be in the survey as they’re less well-positioned to know these things, and that a more detailed faculty-by-faculty survey would be a better indicator).</p>

[BaghDAD]

<p>Parent2Noles wrote:</p>

<p>“Baghdad, Would you mind explaining how TK Wetherell has integrity issues like Machen when he revealed the online test scandal to the NCAA and then ordered an investigation to rectify the issue?”</p>

<p>Love to,</p>

<p>First, an investigation does little to rectify a situation unless the root causes are found and addressed…hardly the case here, as the NCAA had to investigate in much more detail.</p>

<p>I’ll use a historical example for my second point. In 1989, The Exxon Valdez ran aground in Prince William Sound, causing a disaster we are all familar with. Who was at the controls, hazarding a vessel and causing billions of $ in damage? Greg Cousins. Who? Everyone thinks it was Joseph Hazlewood, the ship’s Master. See, even though not on duty, the Master (or as the general public refers to them, Captain) is always responsible for his and HIS CREW’S actions.</p>

<p>What happened with TK? He complained with “I didn’t know.” OK, who is the boss of the FSU employees who wrote the student’s papers? Directed them to take the “special” course? Yes, the kid who breaks the window and goes to the homeowner is correct to do so, however, if he starts offering up that “Jimmy didn’t catch the ball” as an excuse why he shouldn’t pay, well, he has a problem. How is what TK did and is still doing [NCAA</a> rejects Florida State’s appeal of sanctions - Sports - MiamiHerald.com](<a href=“http://www.miamiherald.com/sports/story/1104212.html]NCAA”>http://www.miamiherald.com/sports/story/1104212.html) any different? TK is the captain of the FSU ship…it happened on his watch. Yes, he resigned, but part of me thinks he was “relieved of duty.”</p>

[goufgators]

<p>So instead of addressing the academic scandal at FSU, TK decided to just rid himself of the problems by leaving. I wouldn’t be surprised if the FSU board of trustees pressured him into leaving.</p>

[tk21769]

<p>

However, on balance, not so bad and certainly not a contemptible attempt at ranking.

To so lack objectivity, to render a judgment so driven by self-interest, speaks poorly of a professional scholar. The 5 for his own school would have been not so bad if he had assigned the lowest ranking to every other school. Then his response would have amounted to an act of ridicule and protest. </p>

<p>A university is not a product like a Mercedes Benz. It is a community of people working cooperatively to try to figure out the truth, how to make things better, how to improve the quality of our choices and actions. If somebody does not believe in that mission, why would he be running a university?</p>

[Phead128]

<p>Higher education is a commodity that cost more than a Mercedes Benz. That’s for sure lol. That is besides the point. Why is Machen running a university?</p>

<p>Machen is merely taking orders for Board of Trustees who elected him into office on the premise that he will skyrocket the university into national pre-eminence…</p>

<p>Manipulation around the edges is not the hall mark of a professional scholar. I totally agree!</p>

<p>BTW: If Higher education is not like a commodity like Mercedes Benz, then why does public branding and image matter so much in USNews report? It’s people, but it’s also the quality of leadership to superimpose a standard vision for excellence. The mission to find the truth cannot be fufilled without tuition dollars from undergraduate students. It’s still a corporation selling a product, but it’s run the same way as a corporation. It’s non-profit…</p>

[midatlmom]

<p>wow Phead and tk, you guys are tough. I would equate Machen to a doting parent who thinks that their kid is amazing. While he might be a tad misguided about his kid, that doesn’t make him manipulative, completely lacking in integrity or incapable of doing a great job running a school.</p>

[Phead128]

<p>If College administrators cannot live by what they preach (honesty policy, no cheating, no plagarism, no lying), then isn’t President Machen’s actions somewhat hypocritical? :o</p>

<p>While I agree that it is absolutely understandably for a college administrator to be very proud of his or her institution… Marking other universities down consistently just drives me insane It’s not truthful or honest. </p>

<p>Midatlmom has a really good point there, I am glad you brought that one up! I agree because I have no qualms about boasting about your own institution. I draw the line when someone ranks a competitor as “marginal” when they are as good as your own institution…I bet they do the same vice versa so overall, they both lose lol</p>

[SSobick]

<p>Malicious intent was not Bernie Machen’s objective (unlike Clemson University). The Florida institions that he rated marginal were spot on. All of them have lousy graduate programs, tiny endowments, less than stellar undergraduates, and lousy research expenditures.</p>

<p>Bernie Machen could of shafted Penn State, Syracuse, George Washington, University of Maryland, UIUC, and the University of Miami so that UF could do better on the rankings. He did nothing of the sort and they received a fair assessment in my opinion.</p>

<p>Look at bclinkton’s breakdown and you will see that Machen was pretty much in tune with the peer assessment scores from the past. Granted he should of rated UF strong and not distinguished.</p>

[midatlmom]

<p>But Phead, I don’t actually believe that he lowballed his competitors. I think that he was probably fairly honest about them–he doesn’t think highly of most of the Florida schools (I don’t think that he’s going to be the most popular guy at any academic gatherings they might have). I think that his ranking of UF was highly suspect, but as bclintonk has shown, he’s not far off about the remaining Florida schools. </p>

<p>Which Florida schools do you think that he was wrong about and shafted deliberately to gain an advantage for UF? Do you think that University of Miami is a “4” or would you have given it a “3”? Do you think that Barry, FIU, Florida A&M, Florida Atlantic and Nova are good schools (they are all fourth tier in USNWR). Isn’t Florida State better than University of South Florida?</p>

[Gasdoc]

<p>I think bclintonk’s analysis is spot on…</p>

[tk21769]

<p>Yes, midatlmom does make an interesting point. I do think my own kids are amazing. However, if someone asked me to fill out a form ranking their abilities in this or that, I would hope I would not automatically give them the highest ranking and the neighbor kids one of the lowest. </p>

<p>Now, if it were not simply an academic exercise, if rewards were on the line and I knew that my child’s ranking would affect his success (and if a low ranking hurt not me but him) that might be an awful temptation. In that case, I’d be inclined to question the ranking procedure itself. Sure, take enough samples and the self-interest factor may cancel out. But how many of the other rankings are based on more than a halo effect?</p>

[midatlmom]

<p>once again, tk, I don’t think that he gave the neighbor kids low marks. I think that he gave the neighbor kids marks that he believed were warranted.</p>

<p>As to his own kid, however, he was completely off-base. Did he know it? Maybe, but maybe he’s like most people about their kids. He thinks they’re incredible, but he also sort of knows they’re not. Sometimes it’s hard to be objective and maybe he gave into the temptation to rank them too highly, figuring that most people would rank their own schools highly. Do I think that this is criminal and that he lacks integrity? No, I guess it makes him sort of normal.</p>

[SSobick]

<p>"Isn’t Florida State better than University of South Florida? "</p>

<p>FSU is on the cusp of a top 100 ranking, and USF is a Tier 3 National University. Just a few years ago USF was ranked in the Tier 4 category.</p>