<p>This</a> thread got me interested in the correlation between SAT scores and other college-related statistics. So, I decided to put them to the test. All data is publicly available from IPEDS.</p>
<p>I used a sample of all research universities classified as High or Very High research intensity on the Carnegie classification that also had SAT Math 75%ile scores > 640. This sample has 111 members.</p>
<p>The statistics of interest were full-time retention rate, 6-year grad rate, and research $ per FTE. The last figure is quite variable, and I removed MIT and Caltech from the list because they were very high outliers. I combined these data by averaging the retention and grad rates and adding 1/1000th of the research/FTE stat.</p>
<p>I found the following correlations:
CR 75%: 0.760
CR 25%: 0.773
MA 25%: 0.783
MA 75%: 0.789</p>
<p>Thoughts?</p>
<p>SAT scores are excellent predictors of success in research universities. </p>
<p>It is interesting that SAT scores are related to research expenditures. I don’t think I had ever seen that before. Schools that are magnets for research dollars are also magnets for great students.</p>
<p>Schools that excel in one area seem to excel in many areas. If you have great students, you have great faculty and great research.</p>
<p>I’ve always wondered which SAT scores were most relevant when considering tech schools. A comparison of 18 schools with > 50% STEM majors and > 640 M75% scores yields the following correlations for grad rate / retention rate only:</p>
<p>M75: 0.826
M25: 0.791
C75: 0.745
C25: 0.814</p>
<p>These figures are quite interesting. It appears that tech schools with the strongest top level math students and the highest bottom floor of critical reading ability both retain and graduate their students. The correlation for C25 & M75 averaged is a whopping 0.84.</p>
<p>It makes you wonder why any college would eliminate the SAT requirement!!</p>
<p>Could you repeat this for LACs? Maybe without the research dollars factor…</p>
<p>“It makes you wonder why any college would eliminate the SAT requirement!!”</p>
<p>Because you guys are so much better at predicting collegiate success than full-time professional administrators. I suppose doing well on the ACT exam wouldn’t qualify?</p>
<p>We ARE better than full-time professional administrators…
Administrators often don’t even consult their own faculty about how to analyze.
Yes, the ACT would qualify. Perhaps noimagination can do an ACT analysis?</p>
<p>Here are some ACT correlations:</p>
<p>same selection criteria as noimagination except I included MIT and Caltech</p>
<p>0.76 ACT 25th X retention rate after 1 year
0.75 ACT 75th X retention rate after 1 year
0.71 ACT 25th X graduation rate after 6 years
0.71 ACT 75th X graduation rate after 6 years
0.60 ACT 25th X research expenditures per FTE
0.58 ACT 75th X research expenditures per FTE
0.70 ACT 25th X instructional expenditures per FTE
0.71 ACT 75th X instructional expenditures per FTE</p>
<p>Okay, here are a few of the things I’ll try to crunch out for you:
- ACT analysis (with comparison to SAT as a predictor) - EDIT: collegehelp has my back covered
- LACs (if I carefully choose the sample, I see no reason not to include the research figures)
- Residual analysis (which schools do more with the quality of students they have?)</p>
<p>That last one is particularly interesting to me. It stands to reason that smart students will do well anywhere, but that doesn’t mean some schools don’t do an exceptional job with the students they have.</p>
<p>This is interesting. So much for the traditional CC wisdom that an applicant is so much more than their scores.</p>
<p>For the 88 Carnegie Bac-Arts/Sciences Colleges w/SAT M75%ile > 640, measuring only retention rate + grad rate (no research stat):</p>
<p>M75: 0.747
M25: 0.795
C75: 0.663
C25: 0.700</p>
<p>The math stats appear to be far more useful when looking at LACs…</p>
<p>Having gone to MIT I was surrounded by students with almost perfect SAT scores, yet those kids ranged from truly brilliant to truly incompetent. Anecdotally, I tend to side with barrons that SAT scores are not a very good predictor.</p>
<p>^^^But you only went to MIT! ;-)</p>
<p>It suggests a couple of things-- 1) Schools that out or under perform expectations when looking at SAT in those areas are places to look toward to learn from. 2) All of this information being used by USNews and other groups is adding very little to their analysis suggesting that they’re not really measuring much. It may be enough (that’s one way to look at it), or it may be there is much more to get at about a college that’s interesting that is nowhere close to being measured because we’re just stacking the same information on top of each other.</p>
<p>^ yeah, but back when I went is was actually a good school ;-)</p>
<p>@barrons: I’m not trying to predict first-semester grades, so I don’t see the relevance.</p>
<p>Anyway, I ran the residuals test. My sample was the 94 schools with High or Very High research activity and >640 M75 SAT and between 10% and 35% (inc) receiving federal grant aid. I included the last factor because the small number of schools outside those benchmarks warp the regression line in a fashion that I feel rewards discrimination against low-income students. The remaining sample has 0>-.38 correlation between grant aid and grad/retention, so this was not a significant issue in the final product. </p>
<p>MIT was removed because they were an outlier with research figures off the charts and distorted the results, but they still deserve some recognition for outstanding performance.</p>
<p>Here are the top 20 schools (in order) based on the residual size versus a regression line based on SAT M 75%ile data, which had almost 0.8 correlation:
Yeshiva University
Johns Hopkins University
Pennsylvania State University-Main Campus
Stanford University
University of California-Davis
Syracuse University
University of Pittsburgh-Pittsburgh Campus
Georgetown University
University of Georgia
University of California-San Diego
University of Rochester
University of California-Santa Barbara
University of California-Irvine
Duke University
University of Washington-Seattle Campus
Texas A & M University
University of North Carolina at Chapel Hill
Marquette University
Fordham University
Michigan State University
I’m not going to pick on any schools for underperforming, but those that did so significantly seemed to fit one of two types:
- Primarily ACT schools with SAT scores from OOS students that were significantly higher than comparable ACT scores from resident students. I may redo the test at a later point to correct for this.
- Tech schools with rigorous academics and perhaps less-than-stellar social opportunities that have good scores but abysmal retention and grad rates.</p>
<p>Thoughts?</p>
<p>rogracer-
What do you mean when you say that some MIT students were truly incompetent? Socially incompetent? I find it hard to believe that any MIT students are incompetence although some may have underperformed relative to their ability. </p>
<p>Furthermore, we are not trying to predict the behavior of individual students directly. We are predicting the “behavior” of freshman classes as a whole. There is a difference.</p>
<p>When the correlation is +.8 it means that about 64% (.8 squared) of the explanation can be attributed to SAT scores. There is still variability among schools with very similar SAT scores. (This is the “residual” that noimagination analyzed. Which schools do best with the students they have?)</p>
<p>^ IMO, it’s the residuals that really matter. It doesn’t take a 2400 SAT (oh, I’m funny) to see that higher scoring students are more likely to attend and graduate from schools with significant research prestige. That’s basically the status quo in most people’s minds. I also suspect that smarter students would be fairly likely to graduate at any school. The residuals allow us to see that certain schools simply perform far better than one would expect solely based on their test scores.</p>
<p>Don’t just take my word for it. You should all run some tests yourselves based on whatever statistics you like and see what you get.</p>
<p>1)Should this theoretically be much different than US News’s predicted vs. actual graduation rates?
2) Would you get an even higher prediction of graduation rates by including % class rank, and some sort of relative wealth measure, as additional dependent variables?</p>
<p>That there is a substantial correlation seems to me to be a rather inutitively obvious finding. The residuals are actually more interesting.</p>
<p>IMO, extrapolation of these findings to particular situations must be made with care and insight. An individual has only his/her own SAT scores and capabilities, not the aggregate scores of a particular institution. If an institution’s scores as a whole are high, and its graduatiion rate as a whole is high, that does not mean you personally have a higher chance of graduating from there than from elsewhere. Your chance will be largely based on your own capabilities, in each case. </p>
<p>A perhaps more interesting analysis would be graduation rates of individuals with particular score ranges as they compare to the institution’s median ranges. One might find that chances of graduation go down when particular scores are far below the institution’s medians, vs. when the same scores are closer to the institution’s medians. But data is not available for this.</p>
<p>Edit: this was cross-posted with #19. Time out for basketball.</p>
<p>The residuals are still subject to interpretation. Does an institution have a higher-than-expected graduation rate because of lower grading standards or because of better teaching, or some other reason? Caltech usually has a lower than expected grad rate. The question is why.</p>
<p>The residual analysis is related to the US News over- and under-performance.</p>
<p>Sometimes individual students with lower SATs still do well because of good work habits. That is true. But, when the average grad rate is higher for an entire freshman class it means than individual students have a better chance of graduating although it is hard to predict which ones, specifically, will graduate.</p>