As University of Chicago has recently removed SAT/ACT scores from their admissions, are they just transferring more weight to GPA and extracurricular s. Will this continue to happen with other universities?
Returning to the question of what exactly the SAT/ACT are testing… Does anyone know of research which shows the effect of different levels of preparation, holding all other factors equal, ranging from taking the tests cold up to taking them after intensive tutoring in specially designed courses?
For me that’s the all-important question about these test scores. If all tests were taken cold, then I would concede that something real is being measured, perhaps something approaching general intelligence. We know that’s not what is happening. The question is what effect the different levels of tutoring are having.
Many commenters above (myself included) have given anecdotal evidence, and many of us have made bare assertions. I previously cited my son-in-law, who was briefly employed as a tutor. He was dead certain that he could jack up a mediocre score to a stellar one by teaching any reasonably intelligent kid what might be called meta-techniques for identification of incorrect and correct answers - techniques, it goes without saying, that violate the spirit of the test itself and falsify the results. He didn’t claim that truly below average kids could be raised to the highest levels, nor did he say that truly gifted kids needed intensive tutoring. But if one assumes that most kids thinking of applying to the University of Chicago are average or somewhat above average in natural ability, there would be - if he is correct - a truly significant upward deformation of the totality of all their scores based on who took these fancy courses and who did not.
That’s just one individual’s testimony. Surely research has been done on this. But suppose that his thesis is correct. Suppose further what I believe to be the case - that significant numbers of kids are getting that kind of instruction. It follows as night follows day that the test scores will no longer measure accurately what they were meant to measure - natural ability. I would go further and say that it renders the tests meaningless, not only because of the unfair socioeconomic impact but because the scores themselves are no longer measuring what they purport to measure.
I am ready to be proved wrong by actual research. Anybody know of any?
UChicago hasn’t removed SAT/ACT scores from consideration. Applicants may still submit test scores if they wish to do so, and those scores will be considered alongside rest of the application. They aren’t ignoring these scores.
Here’s an excerpt from the Maroon article about UChicago’s new test-optional policy:
https://www.chicagomaroon.com/article/2018/6/14/university-chicago-goes-test-optional/
Seems like the University found test scores to be a poor indicator of college performance.
Here’s Dean Nondorf’s full interview with Chronicle of Higher Education:
https://www.chronicle.com/article/An-Ultra-Selective-University/243678?cid=wsinglestory_hp_3
@marlowe1 - prep has been going on for at least a generation now, ever since people figured out that the College Board was lying when they said you couldn’t study for these tests. While ymmv, prep is clearly beneficial to individuals on average. But you are correct that the test loses a good deal - if not most - of its value as an indicator of innate intelligence. However, currently neither the SAT nor the ACT is designed to test “intelligence” so much as “college readiness” - many states, for instance, have adopted the ACT as part of the requirement for graduation from high school, and recently the SAT has gone into this field as well. That’s where the two agencies are duking it out and its a lot of fun to read about but isn’t really relevant to the college admissions process.
So all this does support your view that there might not be much objective relevance to either test for college admissions anymore, especially since adcoms have gotten so good at putting classes together using subjective criteria. Furthermore, college prep curricula have become more standardized across schools, so transcripts probably are a much more accurate indicator than they used to be in assessing proper fit with something like UChicago. Haven’t read the Nondorf interview in Chronicle for Higher Ed. yet but it sounds like he might be making the same sort of statement.
Finally, it can’t be underscored how ridiculously distorted these scores are if the college superscores. No small surprise if a super scored SAT of 1500 or ACT of 34 isn’t all that predictive of success. They might think of looking at the kids who just submitted one overall score to see how they are doing academically at UChicago. Perhaps it’s the case that a high composite score from a single test correlates much better with success than the equivalent score that results from multiple retake and superscoring. After all, as pointed out earlier, a superscore means your “real” score is (perhaps notably) lower. While it may be difficult to glean how much of a score reflects some raw intelligence, it’s my understanding that facility with test taking tends to be correlated with it.
“Does testing tell us who’s going to be the best art historian? The answer is No.”
Have no doubt of this. But someone should ask Nondorf whether testing tells them who’s going to be the best mathematician, physicist, biologist, economist, or stats guru. Because last time I checked the number of kids in those majors kinda dominated stuff like Art History.
And ya know what? Why are we outsiders presuming to judge Chi? As if we know more than their leaders, boards, alum committees, etc.
Why the assumption std tests top all other measures? It’s a complex variety of factors. If you want decisions to be made solely on scores, find a college that agrees, where their findings mirror your opinions.
There has been plenty of research generally on these questions, typically finding minimal gains from preparation. Unfortunately, much of it relates to older versions of the tests, which were expressly higher in g-loading. Nevertheless, there is good reason to believe that the tests are still highly correlated with intelligence, as we observe comparable mean differences among races and between sexes, and, perhaps more importantly, comparable variance differences in the score distributions by race and sex, as in more typical IQ tests.
Here is an example of the research (2007):
The instant study concludes:
http://nepc.colorado.edu/files/Briggs_Theeffectofadmissionstestpreparation.pdf
As the authors anticipate, I don’t expect most of the public to accept this, just as they refuse to accept that tests are not measuring income or other aspects of SES.
Testing prepping industry is big business and a very profitable one. How do you expect people working in the industry to say that their service would not make much difference? And the media and politicians also want the public to believe that high ACT/SAT scores are the result of test prep which means that only socially privileged could attain them.
If UC really believed that (which they clearly don’t) AOs would have figured out a way to subsidize free test preps which they knew would be either ineffective or filled with kids of wrong demographics.
My hunch is that UC’s leadership in recent years in protecting campus free speech and getting rid off of trigger warnings has alienated many prospective URM and low income students. I speculate that UC AOs must have seen some drop in applications from those demographics and are desperate to try all measures to lure them back.
Another persistent myth about test preparation is that blacks and Hispanics do not have access to it. Here is an article describing a recent study showing that the black prep rate was found to be 16%. Hispanics were 11%, and whites were 10%:
http://www.insidehighered.com/news/2012/01/19/study-finds-east-asian-americans-gain-most-sat-courses.
Also, see here:
http://muse.jhu.edu/article/421673
And here:
https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1573-7861.2012.01326.x
No one needs to ask this because Nondorf has not stated that they are abandoning testing completely, only that they are not requiring it. Of course a student interested in studying math is going to have to show evidence of math ability. That doesn’t have to be from the math section of the ACT. If they have nothing to prove their aptitude, I assume they aren’t going to be admitted. @satchel I know you were joking about the barista comment, but I bristle at the idea that only STEM kids can end up doing something worthwhile. How many presidents have been engineers? How ,many great artists? How many CEO’s? The world is full of people who can have a huge impact on the world, and enormously successful lives without getting far beyond college algebra. Chicago is not an engineering school. They want to fill all of their departments with the best. Why shouldn’t they be able to look for those kids in novel places?
As far as testing goes, I feel that many here are conflating the usefulness of prep with the usefulness of paid prep services. There is no doubt in my mind, at least from my little sample of two, that prep can make a difference. Its especially true on the ACT which is a measure of knowledge picked up in school. The rules of grammar are not intuitive. Knowing or not knowing where to place a comma is not a measure of innate intelligence. The same is true for the pre-calculus on the test. That is not something you are born knowing. Its something you study, either on your own or with a class. It is amazing how many “tricks” can be used to score highly on the science section. Many kids in our area had no paid preparation, although the school does offer a very low cost option. However, everyone was aware that the tests were important and that they should prepare. I imagine that’s not true everywhere.
MODERATOR’S NOTE:
Let’s move off the topic of test prep companies, please.
Colleges don’t want to rely on GPA or standarized testing because it helps get more applications, lowers acceptance rates and raises place on rankings. With no objective way to judge their admission practices, they can make it more “holistic” and fuzzy. Its a good way to circumvent transparensy.
There are many good ways to evaluate a student. Take a look at Bard’s immediate decision plan. A student prepares for and participates in a seminar, then interviews with an admissions officer. They receive a decision the next day. This provides the school with direct evidence of the student’s aptitude, ability, personality and potential. No proxies needed. Of course, it might be difficult for low income students to travel to participate in such events, but schools could set up regional seminar sites all over the country. With their Billions in endowment, I see now reason why the elite schools couldn’t do something like this. Go into the communities, meet the kids and find the talent. Ask each high school to recommend likely candidates and reach out to those students in person. Then evaluate them on the things that really matter to each college.
Well, these days colleges hardly spend anytime on a single application, you are suggesting an ideal scenerio not a practical one. UPenn is known to spend 10 munutes on every application, clearly fit isn’t high on prioritiy list of college adcoms trying to go through an overwhelmjng number of applications before deadline arrives.
And don’t forget that many first readers are just recent college kids; we have a neighbor whose daughter is doing that full time so she can get tuition remission for professional school in a year or two (forgot what the requirement is). This is a top 10 ranked national university (USNWR).
I don’t have a problem with that - she went there undergrad and is for sure very smart and knows at least parts of the university - but the fact that young people with limited experience are involved like this implies that there is a handbook of procedures, and many of these amorphous “fit” things have been reduced to an algorithm. There are also scores floating all around these admissions offices, which could be tested statistically for biases.
Betcha Penn spends more than 10 minutes after first cut. And not counting the final decision discussion.
But the rumors sure swirl and spread.
Don’t be so sure most adcoms or extra readers are even in their twenties. The young ones get a lot of oversights, ime. But it seems dismissing the process is more popular.
“But it seems dismissing the process is more popular.”
If the college you’re familiar with or any of these other top colleges would like to be more open about the mechanics of their process then that would certainly clear up some of the guesswork. As none have chosen to do so, people are left to piece together clues based on the statements of employees, their direct experience, news articles and common sense. Being dismissive and suggesting people just trust that all is well, that there are swarms of wise, kindhearted admissions fairies who have limitless time to pore over the minutia of every application to find the Very Best Students, resulting in a process that needs no questioning is asking for people to suspend their disbelief in a process that is opaque at best and seems loaded or arbitrary at worst.
Milee, there is plenty of info to go on. But, no, a top holistic rarely spells it out in the sort of detail one might encounter in high school. Few talk about their rubrics. In fact, in life, whether it’s a contest or a job, it’s rarely spelled out to the level of formula.
Kids applying to a single digit college, who feel their stats, rigor and ECs put them in that league, should certainly be able to dig a bit, try to put two and two together. The competition is that fierce. Why trust fate?
But I’ve said this before, including recently: I get far more auto blowback for saying to dig in-- and far fewer taking up the challenge. Who’s being dismissive? I don’t say “trust.” I say, “work it. Don’t assume, don’t trust the first info you find or some media article or some other poster who ay not know.” But, nooo.
There are ways to even google.
Idk if its just me or did others notice majority’s blind trust in magical abilities of college adcoms.
Another popular myth is test prep and multiple attempts being the reason for every student’s high scores.