The Wisdom of US News Peer Assessment Rating

College Search & Selection
61

<p>

</p>

<p>Don’t you think I … knew that? And, isn’t it exactly my point that the narrative and the commentaries do NOT follow the model? </p>

<p>Again, I have no real beef with the model and the attempt to
validate" the PA through this exercise since the laws of averages do their job. The issue remains entirely with the stated “conclusion”

The real conclusion should have read along the lines of “Despite my best efforts to maximize every element that could elevate Cal’s PA and not including less competitive numbers, I cannot come close Cal’s numbers, but I will call it close to almost perfect.”</p>

<p>Should we not ask why the four SAT scores were not used? After all, on the list of individual correlations between the input variables and the actual PA ratings, weren’t the SAT 4 data points among the 5 highest factors? Again, it is a fact that Berkeley has one of the lowest SAT composite scores among similarly ranked schools, and probably the lowest among schools that have a similar or slightly lower PA. </p>

<p>

</p>

<p>The answer is quite obvious. </p>

<p>However, in the end, it is CH’ prerogative to compose a ranking that suits his needs and present is as a plausible validation of the Peer Assessment. It’s also our right to question how the combination of the factors listed reflect the REAL questions posed to the top academics. In particular, how the above list reflects the “faculty dedication to teaching” remains a burning and unanswered question. As we know, USNews has changed the exact wording of its methodology … sometimes includind the dedication line, sometimes not. Here’s the latest: </p>

<p>

</p>

<p>Perhaps our esteemed analyst might consider measuring the level of dedication to teaching **undergraduates **by the faculty at Berkeley, measuring the number of classes per year, the extent of absences, the ratio of classes and students taught between undergraduate and graduate, the ratio of the small army of teaching fellows (or whatever the name is today) versus faculty, the average size of the classes taught by the tenured faculty, etc. And the list goes on!</p>

62

<p>

You could use National Academies membership. You can get the data off the membership section of the websites.</p>

<p>

Haha! Knock yourself out trying to compile that info.<br>
FWIW, Berkeley is no different from other research universities (public and private) with their use of graduate student instructors - they teach discussion and lab sections. However, some people on this site erroneously think the GSIs ( :smiley: ) at Berkeley are gawd-awful and can’t speak a lick of English.</p>

63

<p>Rank, Actual PA Score, Predicted PA Score, School, National Academy of Science Members:
1 4.9 4.8 Harvard University 151
2 4.9 4.9 Massachusetts Institute of Technology 109
3 4.9 4.9 Stanford University 131
4 4.8 4.9 Princeton University 73
5 4.8 4.8 Yale University 58
6 4.7 4.4 University of California-Berkeley 127
7 4.6 4.7 California Institute of Technology 72
8 4.6 3.9 University of Chicago 40
9 4.5 4.5 Columbia University in the City of New York 41
10 4.5 4.3 Cornell University 38
11 4.5 4.2 Johns Hopkins University 21
12 4.5 4.4 University of Pennsylvania 32
13 4.4 4.5 Duke University 18
14 4.4 4.3 University of Michigan-Ann Arbor 20
15 4.3 4.5 Brown University 11
16 4.3 4.4 Dartmouth College 1
17 4.3 4.2 Northwestern University 15
18 4.3 4.1 University of Virginia-Main Campus 4
19 4.2 4.3 University of California-Los Angeles 31
20 4.1 4.2 Carnegie Mellon University 7
21 4.1 3.7 University of North Carolina at Chapel Hill 10
22 4.1 3.9 University of Wisconsin-Madison 43
23 4.1 4.3 Washington University in St Louis 16
24 4.0 3.9 Georgetown University 0
25 4.0 3.6 Georgia Institute of Technology-Main Campus 1
26 4.0 4.1 Rice University 3
27 4.0 3.9 The University of Texas at Austin 15
28 4.0 4.2 University of Illinois at Urbana-Champaign 26
29 4.0 4.1 Vanderbilt University 4
30 3.9 3.9 Emory University 2
31 3.9 4.0 University of Notre Dame 0
32 3.9 4.0 University of Southern California 10
33 3.9 3.5 University of Washington-Seattle Campus 44
34 3.8 3.8 New York University 27
35 3.8 3.4 University of California-Davis 20
36 3.8 3.9 University of California-San Diego 67
37 3.7 3.6 College of William and Mary 0
38 3.7 3.5 Indiana University-Bloomington 10
39 3.7 3.9 Pennsylvania State University-Main Campus 14
40 3.7 3.5 Purdue University-Main Campus 2
41 3.6 3.5 Ohio State University-Main Campus 9
42 3.6 3.9 Tufts University 1
43 3.6 3.8 University of Florida 11
44 3.6 3.7 University of Maryland-College Park 14
45 3.6 3.5 University of Minnesota-Twin Cities 11
46 3.5 3.8 Boston College 0
47 3.5 3.6 Brandeis University 8
48 3.5 3.4 Rensselaer Polytechnic Institute 1
49 3.5 3.7 Texas A & M University 6
50 3.5 3.2 University of Arizona 14
51 3.5 3.5 University of California-Irvine 22
52 3.5 3.5 University of California-Santa Barbara 28
53 3.5 3.2 University of Iowa 4
54 3.5 3.4 Wake Forest University 0
55 3.4 3.4 Boston University 7
56 3.4 3.5 Case Western Reserve University 2
57 3.4 3.2 George Washington University 0
58 3.4 3.6 Michigan State University 7
59 3.4 . University of Colorado at Boulder 14
60 3.4 3.3 University of Georgia 6
61 3.4 3.5 University of Pittsburgh-Pittsburgh Campus 3
62 3.4 3.5 University of Rochester 8
63 3.3 3.4 Rutgers University-New Brunswick 19
64 3.3 3.1 Syracuse University 1
65 3.3 3.4 Tulane University of Louisiana 1
66 3.3 2.9 University of Oregon 6
67 3.3 3.5 Virginia Polytechnic Institute and State University 3
68 3.2 3.1 Arizona State University 9
69 3.2 3.4 Lehigh University 2
70 3.2 3.2 Miami University-Oxford 0
71 3.2 2.9 Stony Brook University 9
72 3.2 3.1 University of Massachusetts Amherst 3
73 3.2 3.2 University of Miami 1
74 3.2 3.1 University of Missouri-Columbia 4
75 3.1 2.9 Baylor University 0
76 3.1 3.2 Clemson University 0</p>

64

<p>^ Regression data for actual PA to # NAS members:</p>

<p>intercept = 3.5896
0.0113 NAS Members
R squared = 0.485</p>

<p>So, not a great relationship. But, if I look at just the USNWR top 30, R squared improves to 0.68.</p>

65

<p>Xiggi, </p>

<p>“Originally Posted by xiggi
Perhaps our esteemed analyst might consider measuring the level of dedication to teaching undergraduates by the faculty at Berkeley, measuring the number of classes per year, the extent of absences, the ratio of classes and students taught between undergraduate and graduate, the ratio of the small army of teaching fellows (or whatever the name is today) versus faculty, the average size of the classes taught by the tenured faculty, etc. And the list goes on!”</p>

<p>So Xiggi, since you brought these issues up, would you please provide the facts to back up your sentiments in the preceding paragraph.</p>

<p>And maybe, you can compare these facts to Stanford? Harvard? Do you have that information?</p>

<p>And how many classes and which ones did you go to at Berkeley?
Have you ever taken a chemistry class at Berkleley?</p>

<p>I heard a beginning chemistry class at Berkeley has 1 professor and maybe 8 TAs and it is amazing.</p>

<p>It’s like a Broadway production. Know anything about it?</p>

<p>And I’m curious…how many students go to lectures these days?</p>

<p>If I go to Stanford and there is a lecture class of 200 students, how many actually go to class? I’m sure it varies, but in your personal experience, how many go to class?</p>

<p>Aren’t many students getting their information outside of class now?</p>

66

<p>Dstark, you may add this story to your Chemistry example, as well as some about Christina Maslach --among others:</p>

<p>

</p>

<p>[University</a> of California - UC Newsroom | Astronomer Alex Filippenko named Professor of the Year](<a href=“http://www.universityofcalifornia.edu/news/article/8685]University”>http://www.universityofcalifornia.edu/news/article/8685)</p>

<p>Of course, you should also consider checking research about the academic ratchet theory and the discretionary time. A good start is Massy and Zemsky or Gordon C. Winston. </p>

<p>As far as students not attending lectures, based on personal experience, I’d say that it is a myth. However, with several thousands colleges and universities, different schools and different class organizations might provide “incentives” to skip lectures altogether. There is no doubt that there are students who find a way (or have the aptitude) to do well in school without attending class or opening a book until the end of the quarter or semester a la Bill Gates.</p>

67

<p>UCBChemEGrad-
Can you tell me where to find the NAS members by college? I looked under National Academy of Sciences membership and did not find the listing. I’d like to add that data to my model to see what happens.</p>

68

<p>Xiggi, </p>

<p>“The “academic ratchet theory,” states that faculty increase their discretionary time by reducing institutional responsibilities. Data from four liberal arts and two research universities indicate that class-size utility is asymmetric around teaching method-dependent norms and that research university faculty prefer smaller teaching loads more fervently than liberal arts faculty.”</p>

<p>This is what you are talking about?</p>

69

<p>NAS Members -> Membership Directory -> Directory Search -> Institution (fill in name and search)</p>

<p>National Academy of Engineering
Membership Directory -> View Membership By: Primary Work Institution</p>

<p>List of Nobel laureate by university affiliation:
[List</a> of Nobel laureates by university affiliation - Wikipedia, the free encyclopedia](<a href=“http://en.wikipedia.org/wiki/Nobel_Prize_laureates_by_university_affiliation]List”>List of Nobel laureates by university affiliation - Wikipedia)</p>

70

<p>NAS and NAE numbers are obviously biased towards science and engineering. These numbers exclude social sciences and humanities, disadvantaging schools with core strengths in these fields. Sorry, guys, I will never buy an argument that says excellence in science and engineering defines overall academic strength. That’s just too narrow and parochial a view. Also, the density of superstars is not necessarily reflective of overall faculty strength, even in sciences and engineering. You could have a faculty with relatively few superstars but great strength at the median, and in my book that could be a stronger faculty than one with more superstars but a lower faculty median. Hard to measure, though. </p>

<p>At the end of the day we’re not likely to have more than expert peer judgment to make these determinations. But I’m generally pretty comfortable with relying on expert peer judgment, subjective though it may be. (Just because it’s subjective doesn’t mean it’s necessarily biased or wrong). But in general I think faculty-to-faculty peer evaluations by discipline are going to be much more revealing than institution-to-institution peer review as in the US News PA rating, which is going to be fraught with incomplete information and institutional jockeying for position.</p>

71

<p>

I agree. However, the overall PA seems to correlate quite well with departmental PA’s.</p>

72

<p>Collegehelp,</p>

<p>GoBlue’s path is correct.
[National</a> Academy of Sciences:](<a href=“http://www.nasonline.org/site/Dir?sid=1011&view=basic&pg=srch]National”>http://www.nasonline.org/site/Dir?sid=1011&view=basic&pg=srch)</p>

<p>Unfortunately, you have to type the name of the university as they have it inputed, or else no results return.</p>

<p>The National Academy of Engineering membership is easier…it gives an alphabetical list of all institutions and number of members:
[Members</a> By Parent Institution](<a href=“http://www.nae.edu/nae/naepub.nsf/Members+By+Parent+InstitutionA?openview]Members”>http://www.nae.edu/nae/naepub.nsf/Members+By+Parent+InstitutionA?openview)</p>

73

<p>

And the winner is:</p>

<p>

</p>

<p>:eek:</p>

<p>(There are only 538 US members in total.)</p>

74

<p>I entered the number of NSF members into the data and ran my software program again. The software came up with a 12-factor model of PA that increased the R-square (the percent of PA explained by the data) from 92.5% to 94%. The correlation between the actual US News PA ratings and the new predicted PA ratings was .97.</p>

<p>Most important, the addition of NSF members into the equation corrected the discrepancies for U of Chicago and Berkeley.</p>

<p>The 12 factors include:
SAT CR 75th
yield
Research $ percent
public or private
SAT math 75th squared
graduation percent squared
number of freshmen squared
SAT CR 75th natural log
endowment per FTE natural log
number of freshmen natural log
number of NSF members
number of NSF members squared</p>

<p>actual PA rank, actual PA, new predicted PA using NSF members, school</p>

<p>1 4.9 5.0 Harvard University
2 4.9 4.8 Massachusetts Institute of Technology
3 4.9 4.8 Stanford University
4 4.8 5.0 Princeton University
5 4.8 5.0 Yale University
6 4.7 4.6 University of California-Berkeley
7 4.6 4.5 California Institute of Technology
8 4.6 4.4 University of Chicago
9 4.5 4.4 Columbia University in the City of New York
10 4.5 4.4 Cornell University
11 4.5 4.2 Johns Hopkins University
12 4.5 4.4 University of Pennsylvania
13 4.4 4.4 Duke University
14 4.4 4.3 University of Michigan-Ann Arbor
15 4.3 4.3 Brown University
16 4.3 4.2 Dartmouth College
17 4.3 4.3 Northwestern University
18 4.3 4.1 University of Virginia-Main Campus
19 4.2 4.2 University of California-Los Angeles
20 4.1 4.0 Carnegie Mellon University
21 4.1 3.7 University of North Carolina at Chapel Hill
22 4.1 4.1 University of Wisconsin-Madison
23 4.1 4.4 Washington University in St Louis
24 4.0 3.7 Georgetown University
25 4.0 3.6 Georgia Institute of Technology-Main Campus
26 4.0 4.0 Rice University
27 4.0 3.8 The University of Texas at Austin
28 4.0 4.2 University of Illinois at Urbana-Champaign
29 4.0 4.0 Vanderbilt University
30 3.9 3.8 Emory University
31 3.9 4.0 University of Notre Dame
32 3.9 3.9 University of Southern California
33 3.9 3.8 University of Washington-Seattle Campus
34 3.8 4.0 New York University
35 3.8 3.5 University of California-Davis
36 3.8 4.2 University of California-San Diego
37 3.7 3.6 College of William and Mary
38 3.7 3.4 Indiana University-Bloomington
39 3.7 3.8 Pennsylvania State University-Main Campus
40 3.7 3.4 Purdue University-Main Campus
41 3.6 3.5 Ohio State University-Main Campus
42 3.6 3.9 Tufts University
43 3.6 3.8 University of Florida
44 3.6 3.7 University of Maryland-College Park
45 3.6 3.5 University of Minnesota-Twin Cities
46 3.5 3.7 Boston College
47 3.5 3.6 Brandeis University
48 3.5 3.4 Rensselaer Polytechnic Institute
49 3.5 3.5 Texas A & M University
50 3.5 3.2 University of Arizona
51 3.5 3.7 University of California-Irvine
52 3.5 3.6 University of California-Santa Barbara
53 3.5 3.3 University of Iowa
54 3.5 3.4 Wake Forest University
55 3.4 3.5 Boston University
56 3.4 3.5 Case Western Reserve University
57 3.4 3.2 George Washington University
58 3.4 3.5 Michigan State University
59 3.4 . University of Colorado at Boulder
60 3.4 3.2 University of Georgia
61 3.4 3.4 University of Pittsburgh-Pittsburgh Campus
62 3.4 3.5 University of Rochester
63 3.3 3.5 Rutgers University-New Brunswick
64 3.3 3.1 Syracuse University
65 3.3 3.3 Tulane University of Louisiana
66 3.3 2.9 University of Oregon
67 3.3 3.4 Virginia Polytechnic Institute and State University
68 3.2 3.1 Arizona State University
69 3.2 3.3 Lehigh University
70 3.2 3.2 Miami University-Oxford
71 3.2 2.9 Stony Brook University
72 3.2 3.1 University of Massachusetts Amherst
73 3.2 3.2 University of Miami
74 3.2 3.1 University of Missouri-Columbia
75 3.1 2.9 Baylor University
76 3.1 3.2 Clemson University</p>

75

<p>Above are National Academy of Sciences members, not NSF members. I misspoke.</p>

<p>NAS members were correlated .43 with the US News Peer Assessment rating.</p>

76

<p>While we can add more data and more criteria, this is a soundbite from the “real world”"</p>

<p>According to J. Bradford DeLong, Professor of Economics at U.C Berkeley, a Research Associate of the NBER, a Visiting Scholar at the Federal Reserve Bank of San Francisco, and Chair of Berkeley’s Political Economy major:</p>

<p>

</p>

<p>From the horsies’ mouth!</p>

77

<p>Just thought I’d offer a little test here. Try and identify which of the following have a 4.5 PA score and which of the following have a 3.9 score.</p>

<p>College : Math 25 - Math 75 , Accept Rate , Yield , 6-Yr Grad Rte , Endowment per Capita</p>

<p>School A : 660 - 770 , 24% , 33% , 91% , $127,932
School B : 660 - 760 , 33% , 56% , 95% , $530,640
School C : 660 - 740 , 27% , 30% , 88% , $435,402
School D : 690 - 780 , 17% , 34% , 92% , $399,791
School E : 660 - 740 , 33% , 39% , 91% , $297,459
School F : 650 - 740 , 25% , 35% , 85% , $219,855
School G : 670 - 780 , 25% , 33% , 91% , $879,268
School H : 660 - 770 , 21% , 47% , 92% , $271,970</p>

78

<p>Ouch for UC Berkeley undergrads</p>

79

<p>Xiggi, </p>

<p>Since Berkeley is a public institution, there is a wealth of anecdotal information available and you can selectively use it to “support” your case.</p>

<p>Christine Romer moved to Washington to act as Obama’s economic advisor.
David Romer moved to be with his wife.</p>

<p>I’m sure Stanford suffered the same plight when Condoleeza Rice left to be National Security Advisor/ Secretary of State.</p>

<p>Hawkette,</p>

<p>Peer Assessment is measuring something else other than student SAT scores, graduation rates, and financial data. Why have numerous data points measuring the same criteria (even though collegehelp’s analysis shows there is some relationship to the objective data)?
Again, peer assessment is measuring “distinguished academic programs”.</p>

80

<p>

</p>

<p>UCB, anecdotal information it is indeed! </p>

<p>And, as far as I know, I did not pretend it to be anything else than a comment made by ONE professor at Cal. I did, however, take the time to look at the faculty pages in the Econ department and poked around to see if it was supported by evidence. I checked </p>

<p>[Economics</a> Faculty Profiles](<a href=“http://elsa.berkeley.edu/econ/faculty/facmaster_a.shtml]Economics”>http://elsa.berkeley.edu/econ/faculty/facmaster_a.shtml)
and
[Economics</a> Courses at Cal](<a href=“http://elsa.berkeley.edu/econ/courses.shtml]Economics”>http://elsa.berkeley.edu/econ/courses.shtml)</p>

<p>I also checked the page of [Romer</a>, Christina D.](<a href=“http://elsa.berkeley.edu/econ/faculty/romer_c.shtml]Romer”>http://elsa.berkeley.edu/econ/faculty/romer_c.shtml). Her Current Status is listed as Teaching. Quite an accomplishment to be able to juggle that with the demands of being the 25th Chair of the Council of Economic Advisors and serving Obama. </p>

<p>If you have anecdotes that contradict the account of Prof. DeLong, by all means do not hesitate to post them. If you know of Econ undergraduate classes that are taught by professors covered by the data posted by CH, by all means let me know. If you do not believe that his account correctly depicts what happens in the Econ Department at Cal, by all means ignore my comments! </p>

<p>In the meantime, does it not matter that professors are absent AND not replaced? Does it not matter, that the UG seem to be impacted the most? Does the issue of discretionary time not matter in a debate where dedication to teaching UNDERGRADUATES is at the center?</p>

<p>After all, aren’t we interested in unveiling the truth? Be it what it is!</p>

<p>Edit: Comments about Romer - Xposted</p>