<p>Even better would be to compare SAT scores of entering freshmen with GRE/LSAT/MCAT scores of graduating seniors, with appropriate normalization. Is such data available, should someone want to crunch it? No measure is perfect, but this could be an available metric.</p>
<p>
</p>
<p>Is there any serious suggestion that it’s better for a college’s academic level to choose the lowest-scoring applicants rather than the highest-scoring applicants? It’s instructive to compare high school grades to SAT scores as a criterion for college entrance (and thus as a proxy for college academic quality). High grade averages in high school classes are also more readily obtained by richer rather than poorer families, </p>
<p>[Making</a> the grade: the SAT versus the GPA](<a href=“http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/archive/2001/10/29/ED141825.DTL]Making”>http://www.sfgate.com/cgi-bin/article.cgi?file=/chronicle/archive/2001/10/29/ED141825.DTL) </p>
<p>but there doesn’t seem to be any national advocacy group posting a website advocating that high schools abolish course grades.</p>
<p>
</p>
<p>So could SAT scores…I could pay money for SAT coaching.</p>
<p>An Amherst admissions advisor said that if they just selected students based on SAT scores, they’d have a very wealthy student population.</p>
<p>[Online</a> NewsHour: A Look at Amherst College’s Admissions Process – June 22, 2004](<a href=“http://www.pbs.org/newshour/bb/education/jan-june04/merrow_6-22.html]Online”>http://www.pbs.org/newshour/bb/education/jan-june04/merrow_6-22.html)</p>
<p>
</p>
<p>
</p>
<p>I would like to see someone post a bivariate plot (“scattergram”) of applicant incomes in relation to applicant SAT scores, and another bivariate plot of the same group of applicants with their incomes in relation to their high school grade averages. (There are lots of issues about adjusting high school grade averages to be comparable from one high school to the next, and indeed that is what keeps admission tests in the mix when colleges make admission decisions.) </p>
<p>Returning to the stated concern of the participant who opened this thread, I would like to ask everyone to take a look at the Washington Monthly college ratings, </p>
<p>[“The</a> Washington Monthly College Rankings” by The Editors](<a href=“http://www.washingtonmonthly.com/features/2006/0609.collegechart.html]"The”>http://www.washingtonmonthly.com/features/2006/0609.collegechart.html) </p>
<p>because they are based on largely objective numerical criteria, and they explicitly take into account student socioeconomic factors.</p>
<p>That would be fascinating, but almost impossible to get good data. Only people interested in med school take the MCATs, so we would not have an idea of what would have been the performance of those who do not take it. Some of the decision not to apply is based on gpa, so those with low gpa, even if they would have liked to go to med school, don’t take the test. So two places might have different rates of MCAT-taking among students of similar ability. The one with more selection among the undergrads (i.e. lower gpa students drop out of premed before taking MCAT) might show a higher mean MCAT, although in fact the students are equal.</p>
<p>Same problems might apply to GRE and LSAT. Few people take them just for the fun of it, and those who do probably do not do the usual studying beforehand.</p>
<p>There is excellent data showing that, at the student level, SAT’s are correlated with these other tests. Many years ago there was a detailed study that found a very high correlation between SAT and MCAT for a cohort of students. So much so that the authors suggested there was little more information to be gained by giving them the MCAT. </p>
<p>I suppose the biggest problem with using such a correlation to evaluate the educational quality of colleges is that it would imply that the education students received would be reflected in scores on these tests. The LSAT is apparently very good for predicting grades in law school and the MCAT very good for predicting grades in medical school. However, that does not mean that either is a particularly good measure of what one learned in college. The subject tests of the GREs are probably better tests of certain kinds of learning that took place in college, but perhaps not at all what the professors would have considered most important. I am sure Caltech professors are most interested in their graduates depth of understanding of scientific principles, ability to analyze data, and design experiments. These are extremely difficult to test in a GRE format. Some highly selective graduate schools do not even use the GRE, they just are not interested in the results.</p>
<p>Token. I mentioned Caltech, MIT, and Harvey Mudd because those are usually the three top colleges overall in percent of students who get PhD’s, and their grads are not accurately characterized as doing so because they could not get jobs as engineers and computer scientists. The data hardly supports the contention that these students are getting research degrees because they cannot get jobs. It does not support the contention that they would have been able to get jobs had they gone to a major university (Caltech? MIT?). </p>
<p>They are getting research degrees because they are interested in the fields, and many of them want research jobs for which a doctorate degree is an entry requirement.</p>
<p>High rates of PhD attainment imply a talented student body and an academic orientation. Many other colleges have equally talented students, but students who are not as interested in the sorts of things that people with PhDs do. </p>
<p>High rates of professional degree attainment imply a talented student body and a preprofessional orientation. Many other colleges have equally talented students, but students who are not as interested in the sorts of things that people with professional degrees do. </p>
<p>Students who find themselves interested in the same things as students at a high-PhD-rate college might like that college, and get more out of it than if they went to an, equally rigorous, high-professional-degree-rate college. Students who feel at home at a high-professional-degree-rate college might be better off there than at one of the pre graduate places.</p>
<p>Side note. At the elite colleges, which include essentially all the places we have been discussing, MOST students go on to graduate education. So if they are only doing this because they cannot get jobs, it is at least as prevalent at the preprofessional schools. If you look into it, you will find that graduates of HYPSM who do NOT get advanced degrees are in the minority. On the other hand, at colleges lower in prestige and admission difficulty, for example, those lower on the USNews list, advanced degree attainment is much less common. Does this mean these students graduate with better job prospects???</p>
<p>
</p>
<p>It might mean those students feel they have to get into the full-time labor force, because they come from lower-income families. </p>
<p><a href=“http://www.jackkentcookefoundation.org/jkcf_web/Documents/Achievement%20Trap.pdf[/url]”>http://www.jackkentcookefoundation.org/jkcf_web/Documents/Achievement%20Trap.pdf</a></p>
<p>Most of those in the lower 25% are not majoring in rocket science either. After the first year you might never see them again.</p>
<p>
None of this matters if the sample is big enough. We just need enough samples of SAT/MCAT scores (imagine 10,000 scores from each of 100 colleges) and compare the normalized spreads (MCAT minus SAT). Same for SAT/GRE and SAT/LSAT. Thus we can get some objective numerical idea of what the schools did to the students during attendance, on average. It’s always an apple-to-apples comparison: What did the various undergrad schools do for med school applicants? For law school applicants? For those who took the GRE? That’s all we learn, but we learn something.</p>
<p>Vossron: Large sample size does not solve bias. You could do a complete sample of all the Republican likely voters, millions of people, and you would still underestimate national support for Clinton. You would have a very precise estimate of support for her among Republicans, but not of her support overall.</p>
<p>If the sample of MCAT takers is not representative of students as a whole at each college, then it does not matter how many colleges you test. The MCAT-taker sample is limited in telling you what happens to students overall at that college. If colleges are homogeneous in the probability that a student at a given level of ability will pursue premed far enough to take the MCAT, then the sample still will not tell you about all the people who did not take the test, but it will capture the variation in preparation among the colleges. However, there is no reason to assume that colleges are homogeneous on this measure, and it would be very difficult to check.</p>
<p>Token: So are you converting the argument that “people get PhD’s because they cannot find work” into they “get PhD’s because they can afford to stay out of the job market”? </p>
<p>Certainly there are financial sacrifices in attending graduate school. However, absent evidence that the small LAC’s that have high PhD rates also have wealthier student bodies than many of the elite universities with lower PhD rates, not clear how the family income consideration explains the differences in PhD rates.</p>
<p>afan: You restated my caveats well, thanks. Capturing the variation in preparation among the colleges is the exact purpose. Just as the PhD percentages are one datum, the MCAT, LSAT and GRE to SAT spreads are just additional data points. Given enough data points, we <em>might</em> be able to see evidence of academic quality. You’re right, we’ll never completely eliminate bias, but we can collect and analyze as much data as possible, in order to get closer to an answer. And such an answer is just one datum high school students should consider.</p>
<p>What do all of you think about the Washington Monthly ratings? </p>
<p>[“The</a> Washington Monthly College Rankings” by The Editors](<a href=“http://www.washingtonmonthly.com/features/2006/0609.collegechart.html]"The”>http://www.washingtonmonthly.com/features/2006/0609.collegechart.html) </p>
<p>They try to take into account some of those inequalities of inputs (student characteristics) among different colleges to gauge added value from attending each college.</p>
<p>
</p>
<p>Have not seen that, but there is plenty of tabular data showing strong positive relationships between parental education and GPA and between parental income and GPA. The correlation of income with gpa is lower than the correlation of income with SAT. Authors speculate that this is because lower income students and higher income students attend different high schools with different grading standards. There is also similar data about the correlation between income and SAT 2 scores. The correlation is only slightly less than that between income and SAT 1. </p>
<p>Family income is also correlated with college gpa. However, SAT scores are better predictors of college GPA than are parent education or income.</p>
<p>
</p>
<p>Arbitrary standards for “serving the country”. Who is to say that serving in the Peace Corps (overseas) does more for than country than serving in Americorps (in the US)? Or than contributing to efficient allocation of financial resources by being an investment banker in NY? Why isn’t lobbying to end the war as valuable to the country as joining the military? Or for that matter, what about joining a police or fire department, or the FBI?</p>
<p>The criteria seem so carelessly chosen that the rankings appear meaningless.</p>
<p>Re Washington Monthly, are they ratings or a ranking? Is it more than yet another one-size-fits-all ranking which, I think, is nonsense to begin with?</p>
<p>So is the consensus here that “Quantitative Academic Quality Rankings” (to use the phrase in the thread title) are unlikely to convince anyone who has already made up his mind where to go to college?</p>
<p>Hmmm. If they’ve already made up their minds? Can we avoid that can of worms and consider those still trying to decide?</p>
<p>Well, does one size fit all, or doesn’t it? Which “fit” factors trump the best basis for decision you can propose for the general question?</p>
<p>There are different factors for each student. For those who value academic quality, the various factors proposed here can all be used. Others may value a winning football team. Let’s construct a new web site with a ranked list for every factor we can imagine. Students can assign weights to each factor and then push the button to get a this-size-fits-this-student ranked list. :)</p>
<p>The PhD rates are probably very good indicators of the nature of the educational experience at a given college, particularly those that have unusually high rates. </p>
<p>Which type of experience is best depends on the orientation and goals of the student and will vary from individual to individual. Therefore, as with other attempts to rank colleges, it fails at the step where one has to assume that the criterion is equally important to all prospective students.</p>
<p>Someone who thinks like a future PhD will probably find the experience at one of the places with high PhD rates very gratifying. Another student, equally talented and ambitious, but not headed toward a research career, might find the same place miserable.</p>
<p>A high PhD rate does not mean that the academic experience is uniformly better than a place with a low rate, but it probably means that it is different. </p>
<p>It might reflect a science and engineering focus among the students, but many of the high PhD LAC’s send students to PhD programs across the range of academic fields.</p>
<p>It is not a consequence of the college being small. </p>
<p>It is not a reflection of poor job prospects for the graduates.</p>
<p>
</p>
<p>Okay, I thought there was some general agreement on this point, but I was just checking.</p>