<p>I am amazed at how well the subjective US News Peer Assessment rating captures hard data about college quality. I did a statistical analysis of the PA for National Universities to see how well it correlated with hard facts. It correlated very well. The correlation was .96 between the PA and a set of factors taken from IPEDS data (US Department of Education). The maximum is 1. A multiple correlation of .96 is extremely high. It means that 92.5% of the explanation for the Peer Assessment rating can be found in fewer than 10 pieces of data:</p>
<p>SAT math
admissions percent
yield
size of freshman class
public or private
graduation percent
percent of core expenditures spent on research
endowment per FTE (student)</p>
<p>multiple linear regression analysis results:
F=225, p<.0001
Rsquare = .9249
Multiple R = .9617</p>
<p>I used the above variables plus some transformed versions of them (squares and natural logs).</p>
<p>I based this analysis on 200 National Universities in US News. I selected only those with both math and CR scores above 450.</p>
<p>Faculty award and distinction is probably captured by the research expenditure data that I used. But actually PA can be predicted very well without the research expenditure figures. I can account for 89% of the PA with purely undergrad education data: SAT, size of freshmen class, graduation percent.</p>
<p>Why does public or private matter? I am not sure, but it is a very tiny piece of the equation and adds very little to the overall model for predicting PA scores.</p>
<p>lockn-
I ran a succession of models using 1, 2, 3,…10 variables. The factor that predicts the most by itself is graduation percent squared.</p>
<p>hawkette-
I selected factors to look at based on intuition. The computer program selected the weights based on mathematical algorithms. The weights were selected to yield the best possible prediction model of PA…whichever weights worked best.</p>
<p>I still standby Cal’s and Michigan’s PA score… I think PA is totally accurate for what it is supposed to be measuring: most distinguished academic programs.</p>
<p>CH,
Can you please explain where you got the number for “percent of core expenditures spent on research” and provide those specifics to the forum? </p>
<p>Can you also comment on the relationship between that number and undergraduate education, particularly as it relates to fields which don’t require large capital expenditures to do research?</p>
<p>Also, which SAT Math number are you using? 25th? 75th? Mid-point?</p>
<p>And is your endowment per capita number based on undergrads only or the entire undergrad and grad student population?</p>
<p>And would you please provide the specific category weights that the mathematical program produced in order to optimize its results and correlation?</p>
<p>It doesn’t relate to a field. It isn’t meant to be broken down like that.</p>
<p>We can go on forever and ask: How does the “graduation rate” relate to fields where the undergrad student population is very small? What does the “size of the entering freshman class” have to do with fields that has less than 40 ppl?</p>
<p>These hard facts are not meant to be broken down and not meant to be reviewed on a finite department-by-department basis. </p>
<p>It is meant to be all encompassing and look at the university as a whole as opposed to being broken down field by field. It would more relevant if it was a department rankings… but it isn’t :)</p>
And therein lies the rub. Each undergrad applied math program has roughly the same purpose for potential undergrads: learning applied math. The methods may differ, but the programs can nonetheless be compared. The same is not true of universities as a whole. How can we hope to compare one college that might have excellent psychology programs to one with an emphasis in engineering?</p>
<p>The “percentage of core expenditures spent on research” number inherently gives more weight to schools that conduct greater amount of research in general. You have to conduct a separate department-by-department survey in order to compare institutions on a program level basis. There is an inherent bias towards schools with research oriented missions and with emphasis on engineering/science/medicine type fields that will accrue more research expenditure funding than fields in humanities.</p>
<p>If I may elaborate…
I initially selected about 15 variables from IPEDS data including the number of bachelors degrees produced, and several other expenditure variables related to instructional expenses, public service, and academic support services but some of them were not used by the computer software. They didn’t add enough useful information. The computer software program actually calculates all possible models and then reports the best 1-variable model, the best 2-variable model, and so on. Almost all of the work and decision-making is done by the computer software.</p>
<p>“Can you please explain where you got the number for “percent of core expenditures spent on research” and provide those specifics to the forum?”</p>
<p>This is a number that is calculated by IPEDS and is available to download from the Peer Analysis System under “Commonly used variables” </p>
<p>“Can you also comment on the relationship between that number and undergraduate education, particularly as it relates to fields which don’t require large capital expenditures to do research?”</p>
<p>The research expenditures include money spent to support scholarship in all fields. Exposing undergrads to active scholarship and research is a good thing for undergrads. It is a valuable part of their education. Research expenditures were not a big part of the equation, by the way.</p>
<p>“Also, which SAT Math number are you using? 25th? 75th? Mid-point?”</p>
<p>I entered 4 SAT numbers: math and CR 25th and 75th. But not all of them were used by the computer software. Math 75th had the biggest predictive power of PA.</p>
<p>“And is your endowment per capita number based on undergrads only or the entire undergrad and grad student population?”</p>
<p>This is an IPEDS-calculated number. I think it is all students grad plus undergrad but can’t recall the definition in IPEDS.</p>
<p>“And would you please provide the specific category weights that the mathematical program produced in order to optimize its results and correlation?”</p>
<p>intercept 11.43798
SAT math 75th -.02507
admit % .00989
yield -.00572
freshmen class .00012111
public or private -.24812
SAT math 75th squared .00002187
grad pct squared .00011670
research $ % squared .00013254
endowment per FTE natural log .04340
admit % natural log -.71281</p>
<p>hawkette-
No, the variables are in different orders of magnitude. If you multiply each variable value by its weight and then add the intercept you should get a number that approximates the PA score for a particular college (5-point scale). SATs scores are big so the weight is small but the net contribution to the PA score might be substantial.</p>