<p>
</p>
<p>Should a top ranking by some Monte Carlo trials of some objective data outweigh a #3 ranking of what academics think?</p>
<p>That’s your answer, Dunnin. If you think the weightings used for the S Engineering ranking are important to you, then Brown engineering for a PhD is better than Cornell engineering. If you value the reputational ranking R results, then Cornell engineering is better than Brown.</p>
<p>
This is strikingly similar to the ARWU ranking. I guess that in the next few years, if the NRC ranking is not available, people can use ARWU as a proxy.</p>
<p>^ Heh, and ARWU probably did it in about 6 months with just a few widely published metrics compared to the arduous task the NRC undertook…</p>
<p>With the new NRC ranking out, I decided to check out how my department (Applied Linguistics @UCLA) rank within the 53 linguistic programs that were assessed. Of course I was very pleased to note that under the S-measure (survey-based quality score), Applied Linguistics@UCLA ranked a definite 2nd placing even under the lower range (#2-2). But imagine my horror when I looked under the R-measure (regression-based quality score) to find Applied Linguistics@UCLA had been ranked #12-51!!!</p>
<p>How can a department be ranked on two opposite ends of a spectrum given that the basic data for the ranking exercise used by both R-based and S-based methodology are the same? An understanding of the methods used reveals that this was a result of different weighting given to the same 20 variables used to assess a program. For a concise explanation on how the weighting for R-based and S-based quality score is determined, see: [About</a> the NRC’s Quality Scores — PhDs.org Graduate School Guide](<a href=“http://graduate-school.phds.org/about/quality_scores]About”>http://graduate-school.phds.org/about/quality_scores). </p>
<p>Basically, the R-based determined weighting is derived implicitly through the evaluators’ subjective ranking of the departments first, and then extrapolating that result to see what variables the top (or bottom) departments had in common before assigning the weighting. The S-based weighting is determined simply by asking evaluators to rank the variables they deemed most important (to least important) in the success of a program. So……what sort of weightings has resulted in the disparate placing of my department? I looked up the NRC report for the assigned R-based and S-based coefficients (a number between -1 to 1 used to represent weights of variables) for Linguistics programs. Four types of coefficients are available R5, R95, S5, S95 which determines the 5th percentile (upper range) and 95th percentile (lower range) of a ranking for R-based and S-based ranking respectively. The report shows that the S-based coefficients (or importance of variables) ranked in exactly the same order for both S5 and S95 in linguistic programs, though the value of the coefficients may differ slightly (reproduced below):</p>
<p>Rank of variable/Variable (category)/S5 value/S95 value</p>
<h1>1 / Publications per faculty (CAT1) / 0.149 / 0.162</h1>
<h1>2 / Citations per faculty (CAT1) / 0.117 / 0.130</h1>
<h1>3 / % with Research Grants (CAT1) / 0.094 / 0.105</h1>
<h1>4 / % of academic placement after degree (CAT3) / 0.081 / 0.088</h1>
<h1>5 / % Full financial support (CAT3) / 0.079 / 0.088</h1>
<h1>6 / % of Interdisciplinary work (CAT1) / 0.061 / 0.074</h1>
<h1>7 / Honors and awards per faculty (CAT1) / 0.053 / 0.063</h1>
<h1>8 / % completion of Ph.D. (CAT3) / 0.052 / 0.058</h1>
<h1>9 / Median GRE scores (CAT2) / 0.047 / 0.054</h1>
<h1>10 / # Student support activities (CAT3) / 0.041 / 0.048</h1>
<h1>11 / % with portable fellowships (CAT2) / 0.030 / 0.037</h1>
<h1>12 / # Ph.D. over five years (CAT3) / 0.021 / 0.026</h1>
<h1>13 / % international students (CAT2) / 0.018 / 0.024</h1>
<h1>14 / % of racial diversity-Student (CAT2) / 0.014 / 0.019</h1>
<h1>15 / % of gender diversity-Faculty (CAT1) / 0.011 / 0.016</h1>
<h1>16 / % of racial diversity-Faculty (CAT1) / 0.009 / 0.014</h1>
<h1>17 / % of student with Work Space (CAT3) / 0.009 / 0.013</h1>
<h1>18 / % of gender diversity-Student (CAT2) / 0.008 / 0.012</h1>
<h1>19 / Health insurance covered? (CAT3) / 0.004 / 0.006</h1>
<h1>20 / Avg. time to degree (CAT3) / -0.034 / -0.028</h1>
<p>The weight or level of importance assigned to each variable looks about right given that quality of faculty is given utmost importance as reflected in the top 3 most important S-based variable are publication, citation per faculty and percentage of faculty with research grants. I think outcome of the program as reflected in percentage of Ph.D. students being placed in an academic position after graduation (ranked #4) is also very apt as this shows the marketability and quality of the student after going through the program. Other highly relevant variables of quality such as involvement in interdisciplinary work (#6) and awards per faculty (#7) were also on top. Variables that were arguably irrelevant to the quality of a department such as percentage of students with private work space (does it matter if I prefer to work at Starbucks?!?!), if the university covers health insurance for the student (What has this got to do with quality of the department??!?!) and average time to degree (shorter is not necessarily better-read diploma mill, longer is not necessarily worse-read stringent assessment) were ranked #17, #19 and #20 respectively. In other words, the evaluators knew in their heads what the criterions were for a successful program.</p>
<p>But when you look at the R-based coefficients (a ploy by NRC to sneak in the highly criticized reputational ranking when they realized that their new S-measure resulted in vastly different ranking from their 1995 exercise), it’s apparent the evaluators could not get pass their deeply ingrained subjective perception of the prestige/reputation held by certain departments, resulting in a ranking of variables that is absolutely invalid! As the ranking of variable coefficients for R5 and R95 are slightly different for linguistics programs, I will list both:</p>
<p>Rank of variable/Variable (category)/R5 value</p>
<h1>1 / % of student with Work Space (CAT3) / 0.065</h1>
<h1>2 / Honors and awards per faculty (CAT1) / 0.032</h1>
<h1>3 / Publications per faculty (CAT1) / 0.030</h1>
<h1>4 / # Ph.D. over five years (CAT3) / 0.026</h1>
<h1>5 / % with portable fellowships (CAT2) / 0.026</h1>
<h1>6 / Median GRE scores (CAT2) / 0.012</h1>
<h1>7 / Health insurance covered? (CAT3) / 0.008</h1>
<h1>8 / % of racial diversity-Student (CAT2) / 0.007</h1>
<h1>9 / # Student support activities (CAT3) / 0.003</h1>
<h1>10 / % with Research Grants (CAT1) / -0.009</h1>
<h1>11 / Avg. time to degree (CAT3) / -0.016</h1>
<h1>12 / % of racial diversity-Faculty (CAT1) / -0.045</h1>
<h1>13 / Citations per faculty (CAT1) / -0.046</h1>
<h1>14 / % of academic placement after degree (CAT3) / -0.047</h1>
<h1>15 / % completion of Ph.D. (CAT3) / -0.060</h1>
<h1>16 / % of gender diversity-Faculty (CAT1) / -0.070</h1>
<h1>17 / % international students (CAT2) / -0.075</h1>
<h1>18 / % of gender diversity-Student (CAT2) / -0.078</h1>
<h1>19 / % Full financial support (CAT3) / -0.117</h1>
<h1>20 / % of Interdisciplinary work (CAT1) / -0.119</h1>
<p>Rank of variable/Variable (category)/R95 value</p>
<h1>1 / % of student with Work Space (CAT3) / 0.126</h1>
<h1>2 / Publications per faculty (CAT1) / 0.105</h1>
<h1>3 / Median GRE scores (CAT2) / 0.100</h1>
<h1>4 / % with Research Grants (CAT1) / 0.097</h1>
<h1>5 / Honors and awards per faculty (CAT1) / 0.083</h1>
<h1>6 / Health insurance covered? (CAT3) / 0.079</h1>
<h1>7 / % of racial diversity-Student (CAT2) / 0.079</h1>
<h1>8 / # Student support activities (CAT3) / 0.078</h1>
<h1>9 / # Ph.D. over five years (CAT3) / 0.075</h1>
<h1>10 / % with portable fellowships (CAT2) / 0.057</h1>
<h1>11 / Citations per faculty (CAT1) / 0.050</h1>
<h1>12 / Avg. time to degree (CAT3) / 0.049</h1>
<h1>13 / % of racial diversity-Faculty (CAT1) / 0.024</h1>
<h1>14 / % international students (CAT2) / 0.018</h1>
<h1>15 / % completion of Ph.D. (CAT3) / 0.012</h1>
<h1>16 / % of academic placement after degree (CAT3) / 0.008</h1>
<h1>17 / % of gender diversity-Faculty (CAT1) / -0.013</h1>
<h1>18 / % of Interdisciplinary work (CAT1) / -0.028</h1>
<h1>19 / % of gender diversity-Student (CAT2) / -0.034</h1>
<h1>20 / % Full financial support (CAT3) / -0.054</h1>
<p>On top of the R-based list of importance of variables (for both R5 & R95) is……percentage of students with private work space!!! Other ludicrous top variables include percentage of student with health insurance covered (#7 for R5, #6 for R95). Furthermore, important variables that relates directly to strength of program such as number of citation per faculty (#13 on R5, #11 on R95), percentage of Ph.D. students placed into academic jobs after graduation (#14 on R5, #16 on R95) and faculty involvement in interdisciplinary work (#20 on R5, #18 on R95) were paradoxically ranked very low in terms of importance! To compound its absurdity, if I understood the use of coefficients accurately, a negative coefficient actually punishes a program for having a good score on the particular variable!!! So programs with faculties highly involved in interdisciplinary work are actually discredited under the R-based measure in linguistics program ranking?!? The logic of the R-based coefficients for linguistic programs seems to say: it’s more important that the department has money for its students to lead comfortable lives, to hell with the importance and significance of our faculties’ work!</p>
<p>In sum, my conclusion of the NRC ranking is: (1) Though the NRC is given in 2 measures and in ranges, we can still discriminate a ranking if we look carefully at the weighted variables to see if they are valid, and (2) If you want to know who has the money to build big offices and conduct extravagant luncheons to crow about how great they are, look at the R-measure; if you want to know who really has the goods, go for the S-measure (for linguistic programs at least). For me, I’m happily ignoring the #12-51 placement of my department, and absolutely basking in #2 of the S-measure.</p>
<p>limnieng, great summary. I think you are the first person to have actually read the and dissect the methodology on here. I’m not sure about that negative coefficient, but it is unfortunate that the NRC even reported statistics with such a methodology. They should have either decided reputation surveys were valid, or they were not. Unfortunately the R-based methodology reeks of molding data to fit the desired result. The R-based weights you illustrated are just damning.</p>
<p>DunninLA: “I continue to be puzzled about what he heck happened to Cornell. They fail to excel really in ANY of the five Broad Fields outside of Agricultural Sciences, and even there Wisconsin cleans their clock and they become Penn St.'s peer … weird. How can Brown score higher than Cornell in Engineering (admittedly my ranking weights placement in R1/S1 and R2/S2 proportionally higher than placements into R3/S3, but still ? (scratching head emoticon inserted here). Could it be that the type of faculty who can set up wherever they choose just don’t want to live in Ithaca, NY?”</p>
<p>Wow… I just skimmed through this thread and am utterly struck by the complete idiocy of your comment and the misleading nature of the rankings your post–not least because I am a student of Cornell. Did you get rejected by Cornell three times or something? I know it happened to several people around me, but at least they were not as bitter as you.</p>
<p>The rankings you naively and brainlessly tallied up only reflect the top three schools in each field of study. Every heard of the top 5, top 10 or top 20? These are also excellent programs as well. Why does Brown come out on top of your engineering ranking? Because they scored 1/3 and 1/1 on Mechanical engineering R- and S-rankings, respectively (an oddity indeed, because they beat MIT, Stanford, Berkeley, etc. on this count). If you bother to look a little closer at the other subfield rankings as well, you will notice that Brown does not break in to top 10 in any one of the other engineering programs they have (of which they admittedly have only few). Cornell, on the other hand, has three programs for which they are top 5 (ME, BE, OR), and two others in top 15 (ChemE and EE)–all counting the better of the two rankings. Just try to use your head a little. As a person given the task of advising a student wishing to compare the overall merits of the Brown and Cornell engineering, would you tell him that Brown’s is better simply because it has one spectacular program in ME? Would you not then be overlooking other important disciplines of the engineering science, such as EE, BE, ChemE, CivE, etc.?</p>
<p>Look, if you really have a lot of time to spare on your hand and do service to this community, a less nonsensical way to concoct a comparative ranking would be to add up both the upper and lower bounds of each program’s respective rankings (as Brian Leiter does:[Leiter</a> Reports: A Philosophy Blog: A Quick Guide to the New National Research Council (NRC) Rankings](<a href=“Leiter Reports: A Philosophy Blog: A Quick Guide to the New National Research Council (NRC) Rankings”>Leiter Reports: A Philosophy Blog: A Quick Guide to the New National Research Council (NRC) Rankings)). Next you can come up with a way to add up the numerical values and sort them in an ascending order to create an overall assessment of a school’s strength vis-a-vis its peers.</p>
<p>blu – </p>
<p>By a different methodology, Cornell in the 1995 publication came out in the top 8, so that is why I was surprised Cornell is so rarely placed in R Group 1, R Group 2, R Group 3, and the same for S groupings, in this year’s publication. It’s like Cornell fell off the map. no, I was not rejected at Cornell. A long time ago I attended a HYPSM for undergrad.</p>
<p>As to what truly WHOLE UNIVERSITY useful rankings can come from this publication, not many really… which is what I was playing with. Unless you’re a provost or Chancellor of a University, who cares how the WHOLE University does across 59 subfields or 6 Broad Fields. All that matters to an aspiring Ph.D. student is the ONE Professor they will be working under in their ONE department, and to a lesser degree the other resources in the Department. An ineffective (for mentoring) Professor at a #1 ranked program in … say Applied LInguistics … would be less desirable than a highly effective mentoring Professor at a #10 ranked school.</p>
<p>The point I take from your post is that armchair enthusiasts of higher education like myself should be aware that current Ph.D. students in programs that did not show well in this NRC study will react strongly and emotionally and it might be best to simply avoid calling out Departments that underperformed their peers.</p>
<p>To be clear about which data I used – I did not access the NRC study itself. I simply accessed the link provided by UCBChemEguy in post #12, <a href=“http://insidehighered.com/content/download/367660/4465647[/url]”>http://insidehighered.com/content/download/367660/4465647</a> then tallied the number of times each University appeard in R1, R2, R3, S1, S2, S3. As you see multiple Universities can occupy a Grouping. It is not first place, second place, third place.</p>
<p>This is just fun filler anyway. Soon, there will be dozens of commentaries and slicing and dicing of the NRC data by people or groups who carefully set up their methodologies and reasoning.</p>
<p>This thread with a couple of exceptions is composed of posts from laymen who have waited several years for this release and are having fun with it, the same way baseball enthusiasts have fun with baseball statistics.</p>
<p>
</p>
<p>limnieng:</p>
<p>Thank you so much for your enlightening analysis–I now understand the fundamental difference between the approaches. And I fully agree that the S-factors are quite sensible and the R-factors not so. But I think your aside (quoted above) might be a bit too cynical. It seems to be the study (as revealed by your analysis) could be taken as a reductio ad absurdum of the reputational approach. In this light, you could view the study as an implicit discrediting of reputation-based methods rather than as an attempt to retain them. At least one could foresee that it could have that result even if it was not intended.</p>
<p>Descartesz:</p>
<p>I was simply echoing the analysis of Prof. Stephen Stigler from the U. of Chicago in his report ([A</a> short summary of the 2010 NRC report | The University of Chicago](<a href=“http://news.uchicago.edu/btn/nrc.summary.php]A”>http://news.uchicago.edu/btn/nrc.summary.php)), that the R-based ranking was a way to sneak in reputational survey.</p>
<p>I did not ask him how he came to that conclusion though.</p>
<p>Granted, DunninLA, that you were playing with numbers a bit just to amuse yourself and viewers. I apologize for my strong reaction, but if you go back and read your previous post, a student/faculty at any targeted university would be offended by such remarks.</p>
<p>It’s funny that you think that Cornell underperformed, because another blogger (admittedly very partisan) proclaimed Cornell #9 in the overall 2010 NRC rankings, even leaving out agricultural sciences ([MetaEzra</a> – In Composite 2010 NRC Rankings, Cornell Ranks 9th](<a href=“http://www.metaezra.com/archive/2010/09/in_composite_2010_nrc_rankings.shtml]MetaEzra”>MetaEzra -- In Composite 2010 NRC Rankings, Cornell Ranks 9th)). Not that I agree with this blogger: in fact, I think his concocted ranking is as much misleading as the top 3 rankings you posted.</p>
<p>There are many ways to put specious spin on the NRC data, many of them probably ending up as subjective and groundless as some of the other rankings that float out there. The take-home message should be that students and employers all beware of such claims of relative superiority based on superficially-compiled overall rankings (see for instance all the university news fanfares that celebrate their “high marks,” under the search “NRC rankings” in Good News). The NRC data are thus perhaps best used for their originally intended purposes alone: that is, “to help universities improve the quality of their programs through benchmarking and to provide prospective students and the public with accessible information on doctoral programs nationwide.” The study committee avoided overall rankings for a reason and, as a critical Inside Higher Ed piece reports, not even a single committee member “endorsed the actual rankings” ([News:</a> You’re Not No. 1 - Inside Higher Ed](<a href=“http://www.insidehighered.com/news/2010/09/29/rankings]News:”>http://www.insidehighered.com/news/2010/09/29/rankings)). The data do however provide very pertinent information on % of funded first-year students, average time to degree, dropout rate, etc., which should be important considerations for all aspiring grad students. But each take what you will with disciplinary rankings.</p>
<p>Here is the 2nd part of post #37, which listed the schools ranked in the first three Broad Fields.</p>
<p>This is a WEIGHTED ranking. R1 and S1 placements get 3 points, while R2 and S2 placements get 2 points. </p>
<p>Humanities</p>
<p>Rank / School / R1 / R2 / R3 / S1 / S2 / S3 / Weighted Sum </p>
<p>1 Harvard 6 / 3 / 0 / 3 / 1 / 1 / 36
2 Princeton 4 / 4 / 2 / 3 / 1 / 0 / 33
3 UCB (alone) 3 / 5 / 1 / 1 / 4 / 0 / 31
4 Stanford 3 / 2 / 0 / 3 / 4 / 0 / 30
5 Yale 3 / 3 / 1 / 3 / 0 / 1 / 26
6 Columbia 4 / 2 / 0 / 1 / 2 / 2 / 25
7 Duke 2 / 1 / 0 / 2 / 2 / 1 / 19
8 Chicago 2 / 2 / 2 / 2 / 0 / 1 / 19
9 NYU 3 / 1 / 1 / 2 / 0 / 0 / 18
10 UCLA 1 / 2 / 0 / 2 / 0 / 0 / 13
10 Indiana 1 / 1 / 2 / 2 / 0 / 0 / 13
12 Brown 1 / 1 / 0 / 2 / 0 / 1 / 12
13 Penn 2 / 0 / 0 / 0 / 0 / 2 / 8
14 N. Carolina 1 / 0 / 1 / 1 / 0 / 0 / 7
14 Maryland 1 / 2 / 0 / 0 / 0 / 0 / 7
16 Rutgers 1 / 0 / 0 / 1 / 0 / 0 / 6
16 Minnesota 1 / 0 / 1 / 0 / 1 / 0 / 6
18 Michigan 1 / 0 / 0 / 0 / 1 / 0 / 5
18 Virginia 1 / 1 / 0 / 0 / 0 / 0 / 5
18 Penn St. 0 / 1 / 0 / 1 / 0 / 0 / 5
21 Hopkins 1 / 0 / 0 / 0 / 0 / 1 / 4
21 MIT 0 / 0 / 1 / 0 / 1 / 1 / 4
21 Cornell 0 / 2 / 0 / 0 / 0 / 0 / 4
21 Tufts 0 / 1 / 0 / 0 / 1 / 0 / 4
21 Washington 0 / 0 / 1 / 0 / 1 / 1 / 4
26 Notre Dame 1 / 0 / 0 / 0 / 0 / 0 / 3
26 Texas 1 / 0 / 0 / 0 / 0 / 0 / 3
26 UC Davis 0 / 0 / 0 / 1 / 0 / 0 / 3
26 Vanderbilt 0 / 0 / 0 / 1 / 0 / 0 / 3
26 Connecticut 0 / 0 / 0 / 1 / 0 / 0 / 3
26 UCSB 0 / 1 / 0 / 0 / 0 / 1 / 3
32 Wisconsin 0 / 1 / 0 / 0 / 0 / 0 / 2
32 WashU 0 / 1 / 0 / 0 / 0 / 0 / 2
32 Emory 0 / 1 / 0 / 0 / 0 / 0 / 2
32 tOhio St. 0 / 1 / 0 / 0 / 0 / 0 / 2
32 Northwestern 0 / 0 / 0 / 0 / 1 / 0 / 2
37 SUNY-Buff 0 / 0 / 1 / 0 / 0 / 0 / 1
37 CUNY 0 / 0 / 1 / 0 / 0 / 0 / 1
37 Rochester 0 / 0 / 1 / 0 / 0 / 0 / 1
37 Cincinnati 0 / 0 / 0 / 0 / 0 / 1 / 1</p>
<p>Math and Physical Sciences </p>
<p>Rank / School / R1 / R2 / R3 / S1 / S2 / S3 / Weighted Sum </p>
<p>1 Princeton 4 / 0 / 0 / 5 / 0 / 0 / 27
1 Harvard 1 / 3 / 1 / 3 / 4 / 0 / 27
3 UCB (alone) 4 / 1 / 1 / 2 / 1 / 3 / 26
4 Caltech 2 / 2 / 1 / 2 / 1 / 1 / 20
5 MIT 2 / 1 / 2 / 0 / 3 / 1 / 17
6 Stanford 2 / 0 / 0 / 2 / 0 / 2 / 14
7 UCLA 1 / 0 / 1 / 1 / 1 / 0 / 9
8 Washington 1 / 0 / 0 / 0 / 2 / 0 / 7
9 Columbia 1 / 0 / 0 / 1 / 0 / 0 / 6
9 Colo. St. 1 / 0 / 0 / 1 / 0 / 0 / 6
9 NYU 0 / 1 / 1 / 1 / 0 / 0 / 6
12 UC San Diego 1 / 0 / 0 / 0 / 1 / 0 / 5
12 UCSB 0 / 0 / 0 / 1 / 1 / 0 / 5
14 Texas 1 / 0 / 0 / 0 / 0 / 0 / 3
14 Illinois 0 / 1 / 1 / 0 / 0 / 0 / 3
14 Wisconsin 0 / 0 / 1 / 0 / 1 / 0 / 3
14 Brown 0 / 1 / 0 / 0 / 0 / 1 / 3
14 Northwestern 0 / 1 / 0 / 0 / 0 / 1 / 3
19 Michigan 0 / 0 / 1 / 0 / 0 / 1 / 2
19 Penn St. 0 / 0 / 0 / 0 / 1 / 0 / 2
19 Cornell 0 / 0 / 1 / 0 / 0 / 1 / 2
19 Hopkins 0 / 1 / 0 / 0 / 0 / 0 / 2
19 Iowa St. 0 / 1 / 0 / 0 / 0 / 0 / 2
19 UC Irvine 0 / 0 / 0 / 0 / 1 / 0 / 2
25 Chicago 0 / 0 / 0 / 0 / 0 / 1 / 1
25 Penn 0 / 0 / 0 / 0 / 0 / 1 / 1
25 Arizona 0 / 0 / 1 / 0 / 0 / 0 / 1
25 N. Carolina St. 0 / 0 / 1 / 0 / 0 / 0 / 1
25 Carnegie Mellon 0 / 0 / 0 / 0 / 0 / 1 / 1
25 Hawaii 0 / 0 / 0 / 0 / 0 / 1 / 1</p>
<p>Social and Behavioral Sciences</p>
<p>Rank / School / R1 / R2 / R3 / S1 / S2 / S3 / Weighted Sum </p>
<p>1 Harvard 5 / 0 / 2 / 4 / 1 / 1 / 32
2 Stanford 2 / 2 / 0 / 2 / 1 / 1 / 19
3 Princeton 2 / 0 / 0 / 3 / 1 / 0 / 17
4 UCB (alone) 1 / 2 / 2 / 1 / 0 / 0 / 12
4 Maryland 0 / 3 / 1 / 0 / 2 / 1 / 12
6 Hopkins 1 / 1 / 0 / 2 / 0 / 0 / 11
7 Michigan 1 / 1 / 1 / 1 / 0 / 1 / 10
7 Wisconsin 0 / 2 / 1 / 1 / 0 / 2 / 10
9 Chicago 2 / 1 / 0 / 0 / 0 / 1 / 9
10 Penn 0 / 2 / 0 / 1 / 0 / 1 / 8
10 UCLA 0 / 2 / 0 / 0 / 2 / 0 / 8
12 MIT 1 / 1 / 0 / 0 / 1 / 0 / 7
12 UCSB 1 / 1 / 0 / 0 / 1 / 0 / 7
12 Indiana 1 / 1 / 0 / 0 / 1 / 0 / 7
12 Penn St. 0 / 1 / 0 / 1 / 0 / 2 / 7
16 Boston U 1 / 0 / 0 / 1 / 0 / 0 / 6
17 Columbia 1 / 1 / 0 / 0 / 0 / 0 / 5
17 Texas 1 / 1 / 0 / 0 / 0 / 0 / 5
19 tOhio St. 1 / 0 / 0 / 0 / 0 / 1 / 4
19 Cornell 0 / 1 / 0 / 0 / 0 / 2 / 4
19 UC Davis 0 / 1 / 0 / 0 / 1 / 0 / 4
19 Colorado 0 / 1 / 0 / 0 / 1 / 0 / 4
23 Renssellaer 1 / 0 / 0 / 0 / 0 / 0 / 3
23 Rutgers 1 / 0 / 0 / 0 / 0 / 0 / 3
23 Miami 1 / 0 / 0 / 0 / 0 / 0 / 3
23 USC 1 / 0 / 0 / 0 / 0 / 0 / 3
23 Duke 0 / 0 / 0 / 1 / 0 / 0 / 3
28 Illinois 0 / 1 / 0 / 0 / 0 / 0 / 2
28 Clemson 0 / 1 / 0 / 0 / 0 / 0 / 2
28 Arizona 0 / 1 / 0 / 0 / 0 / 0 / 2
28 Howard 0 / 1 / 0 / 0 / 0 / 0 / 2
28 Florida 0 / 1 / 0 / 0 / 0 / 0 / 2
28 Oregon St. 0 / 0 / 0 / 0 / 1 / 0 / 2
28 Carnegie M 0 / 0 / 0 / 0 / 1 / 0 / 2
35 Connecticut 0 / 0 / 1 / 0 / 0 / 0 / 1
35 Ohio (Mia) 0 / 0 / 1 / 0 / 0 / 0 / 1
35 Kent St. 0 / 0 / 1 / 0 / 0 / 0 / 1
35 Syracuse 0 / 0 / 1 / 0 / 0 / 0 / 1
35 Connecticut 0 / 0 / 1 / 0 / 0 / 0 / 1
35 N. Carolina 0 / 0 / 1 / 0 / 0 / 0 / 1
35 S. Florida 0 / 0 / 1 / 0 / 0 / 0 / 1
35 UC San Diego 0 / 0 / 1 / 0 / 0 / 0 / 1
35 NYU 0 / 0 / 0 / 0 / 0 / 1 / 1
35 UMASS 0 / 0 / 0 / 0 / 0 / 1 / 1
35 Kansas St. 0 / 0 / 0 / 0 / 0 / 1 / 1</p>
<p>OK, now the real fun.</p>
<p>Here are the 33 Universities that are ranked in this 2010 NRC study in at LEAST THREE of the six Broad Fields of research. Those ranked in only two or one are not shown.</p>
<p>How do we value the placements in each of the six areas in order to answer the question: what is the highest ranking US Research University across all disciplines? That is admittedly a silly question. I don’t know anyone with a need to get 59 Ph.D. degrees and a constraint that they must all be at the same University. </p>
<p>But as I said, just for fun, here are three different ways of answering that question:</p>
<p>1) Which University has the lowest average ranking order across all six Broad Fields in which they are ranked (non-zero if you will).</p>
<p>2) same question, but removing the data for the somewhat specialized area of Agricultural Sciences in which most research universities do not participate.</p>
<p>3) Let’s allow each university to take its best showing for 3 of the 6 Broad Fields. How do they rank then?</p>
<pre><code>Ranking Position in Each of the 6 Broad Fields of Research
Broad Field Ag / Bio / Eng / Hum / MathSci / SocBeh /
</code></pre>
<p>School / Rank in AG / Rank in Bio / Rank in Eng / Rank in Hum / Rank in Math-PSci / Rank in Soc-Behav. / Ave. non-zero rank</p>
<p>AVERABE OF ALL NON-ZERO SUBRANKS</p>
<p>1 UCB + UCSF 6 / 1 / 2 / 3 / 3 / 4 / 3.2
1 Stanford nr / 2 / 2 / 4 / 6 / 2 / 3.2
3 Harvard nr / 4 / 13 / 1 / 1 / 1 / 4.0
3 UCB (alone) 6 / 6 / 2 / 3 / 3 / 4 / 4.0
5 Princeton nr / 13 / 6 / 2 / 1 / 3 / 5.0
6 Yale 13 / 3 / 10 / 5 / nr / nr / 7.8
7 Caltech nr / 18 / 2 / nr / 4 / nr / 8.0
8 MIT nr / 5 / 1 / 21 / 5 / 12 / 8.8
9 UCLA nr / 11 / nr / 10 / nr / 10 / 10.3
10 Hopkins nr / 6 / nr / 21 / 19 / 6 / 10.4
11 Michigan nr / 9 / 11 / 18 / 19 / 7 / 12.8
11 Brown nr / nr / 13 / 12 / 14 / nr / 13.0
11 Penn St. 3 / nr / nr / 18 / 19 / 12 / 13.0
14 Columbia nr / 21 / nr / 6 / 9 / 17 / 13.3
15 UCSB nr / nr / 5 / 26 / 12 / 12 / 13.8
16 Illinois 4 / nr / 11 / nr / 14 / 28 / 14.3
17 Penn nr / 10 / nr / 13 / 25 / 10 / 14.5
18 Wisconsin 1 / nr / 21 / 32 / 14 / 7 / 15.0
19 UC Davis 5 / 18 / nr / 26 / nr / 17 / 16.5
20 Duke nr / 21 / nr / 7 / nr / 23 / 17.0
20 Texas nr / nr / 8 / 26 / nr / 17 / 17.0
20 Washington 8 / 27 / 21 / 21 / 8 / nr / 17.0
23 NYU nr / nr / nr / 9 / 9 / 35 / 17.7
24 UCSD nr / 15 / 9 / nr / 12 / 35 / 17.8
24 Cornell 2 / 30 / 16 / 21 / 19 / 19 / 17.8
26 Chicago nr / 30 / nr / 8 / 25 / 9 / 18.0
27 Minnesota 7 / 35 / nr / 16 / nr / nr / 19.3
28 Rutgers nr / 35 / nr / 16 / nr / 23 / 24.7
29 Ohio St. 11 / 41 / nr / 32 / nr / 19 / 25.8
30 Florida nr / 35 / 16 / nr / nr / 28 / 26.3
30 N. Carolina nr / 15 / nr / 14 / 25 / 25 / 26.3
32 Northwestern nr / 41 / 21 / 32 / 14 / nr / 27.0
33 UMASS 12 / 21 / 21 / nr / nr / 23 / 38.5</p>
<p>School / Rank in AG / Rank in Bio / Rank in Eng / Rank in Hum / Rank in Math-PSci / Rank in Soc-Behav. / Ave. non-zero rank</p>
<p>AVE of ALL SUBRANKS EXCEPT AGRICULTURAL SCIENCES</p>
<p>1 UCB + UCSF 6 / 1 / 2 / 3 / 3 / 4 / 2.6
2 Stanford nr / 2 / 2 / 4 / 6 / 2 / 3.2
3 UCB (alone) 6 / 6 / 2 / 3 / 3 / 4 / 3.6
3 Yale 13 / 3 / 10 / 5 / nr / nr / 3.6
5 Harvard nr / 4 / 13 / 1 / 1 / 1 / 4.0
6 Caltech nr / 18 / 2 / nr / 4 / nr / 4.8
7 Princeton nr / 13 / 6 / 2 / 1 / 3 / 5.0
8 UCLA nr / 11 / nr / 10 / nr / 10 / 6.2
9 Brown nr / nr / 13 / 12 / 14 / nr / 7.8
10 MIT nr / 5 / 1 / 21 / 5 / 12 / 8.8
11 Penn St. 3 / nr / nr / 18 / 19 / 12 / 9.8
12 Duke nr / 21 / nr / 7 / nr / 23 / 10.2
12 Texas nr / nr / 8 / 26 / nr / 17 / 10.2
12 Minnesota 7 / 35 / nr / 16 / nr / nr / 10.2
15 Hopkins nr / 6 / nr / 21 / 19 / 6 / 10.4
16 Illinois 4 / nr / 11 / nr / 14 / 28 / 10.6
16 Columbia nr / 21 / nr / 6 / 9 / 17 / 10.6
16 NYU nr / nr / nr / 9 / 9 / 35 / 10.6
19 UCSB nr / nr / 5 / 26 / 12 / 12 / 11.0
20 Penn nr / 10 / nr / 13 / 25 / 10 / 11.6
21 UC Davis 5 / 18 / nr / 26 / nr / 17 / 12.2
22 Michigan nr / 9 / 11 / 18 / 19 / 7 / 12.8
23 UMASS 12 / 21 / 21 / nr / nr / 23 / 13.0
24 UCSD nr / 15 / 9 / nr / 12 / 35 / 14.2
25 Chicago nr / 30 / nr / 8 / 25 / 9 / 14.4
26 Wisconsin 1 / nr / 21 / 32 / 14 / 7 / 14.8
27 Rutgers nr / 35 / nr / 16 / nr / 23 / 14.8
28 Washington 8 / 27 / 21 / 21 / 8 / nr / 15.4
29 N. Carolina nr / 15 / nr / 14 / 25 / 25 / 15.8
30 Florida nr / 35 / 16 / nr / nr / 28 / 15.8
31 Ohio St. 11 / 41 / nr / 32 / nr / 19 / 18.4
32 Cornell 2 / 30 / 16 / 21 / 19 / 19 / 21.0
33 Northwestern nr / 41 / 21 / 32 / 14 / nr / 21.6</p>
<pre><code> School / Rank in AG / Rank in Bio / Rank in Eng / Rank in Hum / Rank in Math-PSci / Rank in Soc-Behav. / Ave. non-zero rank
</code></pre>
<p>AVE of the Top 3 BROAD FIELDS</p>
<p>1 Harvard nr / 4 / 13 / 1 / 1 / 1 / 1.0
2 UCB + UCSF 6 / 1 / 2 / 3 / 3 / 4 / 2.0
2 Stanford nr / 2 / 2 / 4 / 6 / 2 / 2.0
2 Princeton nr / 13 / 6 / 2 / 1 / 3 / 2.0
5 UCB (alone) 6 / 6 / 2 / 3 / 3 / 4 / 2.7
6 MIT nr / 5 / 1 / 21 / 5 / 12 / 3.7
7 Yale 13 / 3 / 10 / 5 / nr / nr / 6.0
8 Wisconsin 1 / nr / 21 / 32 / 14 / 7 / 7.3
9 Caltech nr / 18 / 2 / nr / 4 / nr / 8.0
10 Michigan nr / 9 / 11 / 18 / 19 / 7 / 9.0
11 Illinois 4 / nr / 11 / nr / 14 / 28 / 9.7
11 UCSB nr / nr / 5 / 26 / 12 / 12 / 9.7
13 UCLA nr / 11 / nr / 10 / nr / 10 / 10.3
13 Hopkins nr / 6 / nr / 21 / 19 / 6 / 10.3
15 Columbia nr / 21 / nr / 6 / 9 / 17 / 10.7
16 Penn St. 3 / nr / nr / 18 / 19 / 12 / 11.0
16 Penn nr / 10 / nr / 13 / 25 / 10 / 11.0
18 UCSD nr / 15 / 9 / nr / 12 / 35 / 12.0
19 Cornell 2 / 30 / 16 / 21 / 19 / 19 / 12.3
19 Washington 8 / 27 / 21 / 21 / 8 / nr / 12.3
21 Brown nr / nr / 13 / 12 / 14 / nr / 13.0
22 UC Davis 5 / 18 / nr / 26 / nr / 17 / 13.3
23 Chicago nr / 30 / nr / 8 / 25 / 9 / 14.0
24 Duke nr / 21 / nr / 7 / nr / 23 / 17.0
24 Texas nr / nr / 8 / 26 / nr / 17 / 17.0
26 NYU nr / nr / nr / 9 / 9 / 35 / 17.7
27 UMASS 12 / 21 / 21 / nr / nr / 23 / 18.0
27 N. Carolina nr / 15 / nr / 14 / 25 / 25 / 18.0
29 Minnesota 7 / 35 / nr / 16 / nr / nr / 19.3
30 Northwestern nr / 41 / 21 / 32 / 14 / nr / 22.3
31 Ohio St. 11 / 41 / nr / 32 / nr / 19 / 23.7
32 Rutgers nr / 35 / nr / 16 / nr / 23 / 24.7
33 Florida nr / 35 / 16 / nr / nr / 28 / 26.3</p>
<p>I ran out of editing time to say the rankings shown in the post above are a compilation of the rankings in each of the SIX BROAD FIELDS that are shown in my posts #37 and #51 above.</p>
<p>These aggregate rankings are sensible-ish. But I can say with confidence that the electrical engineering rankings are nonsense. Somehow MIT and Berkeley actually ended up outside the top 10. I bet that not even the faculty at the universities that magically displaced MIT and Berkeley would acknowledge that (Harvard engineering? It almost seems like a parody).</p>
<p>FWIW, from The Cornell Chronicle:</p>
<p>" …almost half the participating Cornell University Graduate School and Weill Cornell Graduate School of Medical Sciences graduate fields were included within the top 10 range of rankings on an overall measure, and more than 75 percent are included in the top 20 range."</p>
<p>[Cornell</a> University Graduate School – NRC Assessment of Research Doctorates](<a href=“http://www.gradschool.cornell.edu/index.php?p=165]Cornell”>http://www.gradschool.cornell.edu/index.php?p=165)</p>
<p>Here is another question that is begged by the discussion on the difference betwen the R rankings and the S rankings.</p>
<p>I wondered which schools then did better by the R ranking methodology, and which by the S ranking methodology.</p>
<p>Here are the same 32 schools that are highly ranked (possibly in the top 3 of a sub-field) in at least THREE of the six Broad Fields.</p>
<p>Again, I counted the number of times each school appeard in R1, R2, R3, S1, S2, S3, and then WEIGHTED SUM each school by 3<em>R1 + 2</em>R1 + R3 + 3<em>S1 + 2</em> S2 + S3.</p>
<p>However, what if we Sum R and S scores separately? What does it mean if a school scores much higher by the R methodolgy than by the S? Or vice versa?</p>
<p>Here are the 32 schools listed again, with Total R+S Weighted SUM, then separately R weighted SUM, and S weighted SUM. Not sure what it means…</p>
<pre><code>Total Score Rank / School / R1 / R2 / R3 / S1 / S2 / S3 / Total Weighted Score / R Score ONLY / S Score ONLY
</code></pre>
<p>1 UCB + UCSF 16 / 14 / 8 / 9 / 9 / 8 / 137 / 84 / 53
2 Harvard 16 / 8 / 5 / 13 / 9 / 4 / 130 / 69 / 61
2 Stanford 11 / 10 / 2 / 15 / 9 / 4 / 122 / 55 / 67
2 UCB (alone) 14 / 12 / 8 / 8 / 7 / 7 / 119 / 74 / 45
5 Princeton 10 / 7 / 3 / 16 / 2 / 1 / 100 / 47 / 53
6 MIT 10 / 6 / 3 / 6 / 5 / 5 / 78 / 45 / 33
7 Yale 7 / 4 / 4 / 9 / 3 / 3 / 69 / 33 / 36
8 Wisconsin 6 / 4 / 5 / 4 / 3 / 4 / 53 / 31 / 22
9 Caltech 4 / 4 / 2 / 8 / 2 / 2 / 52 / 22 / 30
10 UCLA 3 / 5 / 3 / 4 / 4 / 0 / 42 / 22 / 20
10 Columbia 6 / 4 / 0 / 2 / 3 / 4 / 42 / 26 / 16
12 Penn St. 3 / 3 / 1 / 5 / 2 / 5 / 40 / 16 / 24
13 Chicago 4 / 4 / 3 / 3 / 1 / 4 / 38 / 23 / 15
14 Michigan 3 / 3 / 6 / 1 / 4 / 5 / 37 / 21 / 16
14 Hopkins 2 / 5 / 2 / 4 / 3 / 1 / 37 / 18 / 19
16 Illinois 2 / 6 / 3 / 3 / 0 / 2 / 32 / 21 / 11
16 Penn 3 / 5 / 0 / 3 / 0 / 4 / 32 / 19 / 13
18 Cornell 1 / 7 / 3 / 1 / 2 / 4 / 31 / 20 / 11
19 UCSB 3 / 2 / 1 / 3 / 2 / 1 / 28 / 14 / 14
19 Duke 3 / 1 / 1 / 3 / 2 / 2 / 27 / 12 / 15
21 NYU 3 / 2 / 3 / 3 / 0 / 1 / 26 / 16 / 10
22 UC Davis 3 / 2 / 1 / 2 / 1 / 1 / 23 / 14 / 9
22 N. Carolina 2 / 1 / 2 / 4 / 0 / 1 / 23 / 10 / 13
24 Washington 2 / 1 / 3 / 0 / 4 / 2 / 21 / 11 / 10
24 Brown 2 / 2 / 0 / 3 / 0 / 2 / 21 / 10 / 11
26 UCSD 2 / 0 / 2 / 2 / 3 / 0 / 20 / 8 / 12
26 Texas 5 / 1 / 0 / 0 / 1 / 1 / 20 / 17 / 3
28 Minnesota 2 / 2 / 2 / 0 / 2 / 1 / 17 / 12 / 5
29 Florida 0 / 4 / 0 / 0 / 0 / 8 / 16 / 8 / 8
30 UMASS 1 / 1 / 0 / 2 / 0 / 2 / 13 / 5 / 8
30 Rutgers 2 / 2 / 0 / 1 / 0 / 0 / 13 / 10 / 3
32 Northwestern 0 / 1 / 2 / 0 / 3 / 2 / 12 / 4 / 8
32 Ohio St. 1 / 2 / 1 / 0 / 1 / 2 / 12 / 8 / 4</p>
<p>Clearly Berkeley, MIT, Columbia, Wisconsin, Cornell and Chicago stand out as benefitting relatively more from the R methodology, while Stanford and Penn St. benefit relatively more from S methodology.</p>
<p>Schools ranked better by S method:
Stanford, Princeton, Caltech, Penn State.</p>
<p>Schools ranked better by R method:
Harvard, Berkeley, MIT, Columbia, Chicago, Cornell</p>
<p>Harvard, Berkeley, Stanford, Princeton, Yale, MIT and Caltech are expected to do extremely well on rankings like this. But what happened to Michigan? I expected it to do well – in the top 10 at least. It should should have performed better than Wisconsin or UCLA at least.</p>
<p>RML – by one of the many ways one can rank order, Michigan is in the #10 spot.</p>
<p>If you look at post #52 above, you will see that Michigan does in fact occupy the #10 spot – notice I doubled counted UC Berkeley, with and without assuming UCSF is UCB’s de facto medical research facility – in the first grouping – the average of all non-zero rank positions in the six Broad Fields.</p>
<p>Re S and R:</p>
<p>As I understand it, R is an attempt to capture and systematize what people in the world actually think, and eyeballing the differences in areas where I am somewhat familiar the S rankings come off as weird and the R rankings as not so weird. I think the two sets of (fuzzy) rankings are useful, both for seeing where they are consistent and for raising questions about what is really being valued and why when they diverge.</p>