Let’s have an intelligent discussion on the role of standardized testing in elite college admissions.
“peoplespolicyproject(dot.org)/2018/03/20/the-sat-can-level-the-playing-field-in-education/”
The SAT Can Level the Playing Field in Education
The SAT does not enjoy a good reputation among progressives. Arguments against the use of the test, as well as its analog, the ACT, abound. Both are widely derided as tools of elitism, rejected as culturally biased, and denounced for dehumanizing test takers.
I understand the intuitive feeling that we should not reduce human potential to a test score. And the major testing companies (and nonprofit organizations like the Educational Testing Service, which basically function like companies) are not particularly sympathetic entities. But if you believe in equality and a more level playing field in college admissions, you should defend the SAT.
Unequal SAT Results Mirror an Unequal Society
Some caveats are in order. First, it’s important to acknowledge that yes, SAT results reflect inequalities in race and social class. Black and Hispanic students and poor students do not perform as well on these tests as their white and affluent counterparts, mirroring state-mandated standardized test results and grading distributions. But this reflects a symptom of larger inequality, not a biased test.
Poorer students and racial minorities suffer in these testing outcomes because of socioeconomic inequalities that dog our entire educational system. Test development includes a process called differential item functioning. In this diagnostic, students from different demographic backgrounds are matched by ability and take the same test items. Once overall ability is controlled for, black students do just as well as white on the same items, as do poor students and wealthier ones.
Racial and class inequalities in the SAT certainly are troubling—but only insofar as they show our persistently unequal society.
Critics of standardized tests often complain that affluent students have greater access to test prep materials and coaching. This is indeed a concern, but the research here is clear: coaching services produce far smaller gains than those advertised by the big test prep companies, which routinely claim triple-digit improvements.
A 2006 meta-analysis found that students retaking the SAT after coaching resulted in, on average, an increase of about 50 points on a 1600 scale. That’s not an insignificant number. However, as the researchers point out, we can expect some of that gain to occur simply through increased familiarity with the test and, for lower-scoring students — the type most likely to retake the test — regression to the mean. More recent research found that, after using statistical controls to compare similar students, the combined effect of coaching on a 1600 point scale was about 20 points.
It’s also worth noting that affluent parents can hire a wide variety of coaches and tutors for their children to improve their grades, performance in sports, or ability in extracurricular activities, all of which impact their chances of college admission. Coaching is thus far from a problem with testing alone. Yes, the game is rigged, but it’s rigged from the top to bottom — not just with the SAT.
The Problem with “Holistic Assessments” and Grades
Let’s look at the alternative metrics for student performance: “soft” criteria and high school grade point average (GPA).
Detractors of entrance exams often argue for more “holistic” methods of evaluating students than tests, pushing for greater emphasis on student activities, college essays, and letters of recommendation. They argue that these things allow them to select students that are more than just grades and test scores and build a diverse student body. As Jennifer Finney Boylan put it in a piece decrying the SAT, the only way to fairly choose between applicants is “to look at the complex portrait of their lives.”
But this reasoning goes directly against the stated goal of equality. It should be obvious: affluent parents have far greater ability to provide opportunities for extracurricular (and frequently out-of-school) activities than less affluent parents do.
The student who is captain of the sailing team, president of the robotics club, and who spent a summer building houses in the Global South will likely look more “holistically” valuable than a poorer student who has not had the resources to do similar activities. Who is more likely to be a star violin player or to have completed a summer internship at a fancy magazine: a poor student or an affluent one? College essays are more easily improved through coaching than test scores, and teachers at expensive private schools likely feel more pressure to write effusive letters of recommendation than their peers in public schools.
Favoring the “soft” aspects of a college application is straightforwardly beneficial to the more privileged at the expense of the less.
There’s another reason for people who believe in equality to champion standardized achievement tests: they serve as one of the only bulwarks against the pernicious effects of high school grade inflation.
Everyone has heard of grade inflation in college. Depending on your point of view, it’s either a sign of the continued degradation of educational standards or a mostly-inconsequential statistical curio. But far fewer people seem aware that grade inflation exists in high school. In fact, high school grades are inflating at a rapid pace — but unequally, in a way that should disturb us.
Research by Michael Hurwitz and Jason Lee found that, from 1998 to 2016, the average high school GPA rose from 3.27 to 3.38. That may not sound like much, but distributed over millions of students, it’s a large increase. What’s more, the phenomenon is concentrated at the top. Paul Sackett and Nathan Kuncel, working with a data set from over 300 colleges and a million students, found that the modal GPA — that is, the GPA that occurred most often in the data set — was 4.0. The data set only includes college students, screening out many of the lowest GPA. Still, it demonstrates the extent to which students entering college are now bunched at the top of the grade distribution. Even relative measures like class rank fail to really distinguish grades, as high schools vary widely in how grades are weighted and do not always make this information public to colleges. Some high schools now graduate over 100 students named valedictorians.
Why should progressives care about high school grade inflation? Because it perpetuates traditional inequalities.
In research published in the recent book Measuring Success, Hurwitz and Lee find that whiter schools, more affluent schools, and private schools are all seeing far more grade inflation than higher minority, poorer, and public schools. In fact, public schools have seen little grade inflation; the problem is rampant in private schools, where grades are inflating at three times the rate of public.
You might assume this simply means that schools seeing higher grade inflation are doing a better job of educating students. But when Hurwitz and Lee control for ability by including SAT scores in the model, they still find grades inflating at disparate rates along predictable demographic lines. The potential unfair advantage this may confer in college admissions is clear.
The likely causes of these developments seem obvious. Private schools have a direct financial incentive to please parents in a way that public schools do not. White and affluent parents likely enjoy greater social capital and thus greater ability to pressure teachers, a privilege poor and minority parents largely lack. We could argue that the solution is simply to stop grade inflation. But it’s difficult to imagine a policy fix that could work across the country, and the incentives for teachers of wealthy and white students will only become more obvious as the race to the top of the educational ladder gets more intense.
The SAT and ACT aren’t perfect. But much of the folk wisdom about them and their deficiencies is wrong, and though critics mean well, they actually risk deepening inequality by attacking these tests.
To be clear, I am not advocating an admissions system that uses only the SAT. You only need to look at the various social problems associated with China’s Gaokao exam to see the dangers of placing too much emphasis on any one metric. And I should point out that I have always been a strong supporter of race-based affirmative action programs, which remain necessary to combat traditional inequalities. What I am saying is that the casual assumption that deemphasizing or eliminating the SAT will result in a more equitable system doesn’t withstand scrutiny.
Unlike their rich peers, students who labor under racial and economic disadvantage have very few ways to distinguish themselves from the rest of the pack. A stellar SAT score is potentially one of the most powerful. We should take care not to rob them of that tool in a misguided push for equality.
The wealthy are definitely advantaged in all stages of life, and certainly in the college admission process. Not just for better access to K-12, test prep, ECs, coaching, but also being full pay is a huge advantage.
I think the TO discussion for this upcoming admission cycle is different than a general conversation because so many students were unable to sit for the test and it’s unclear if they’ll have the chance before applications are submitted.
I agree with you in principal that standardize tests are helpful in the age of rampant grade inflation and inequities in our HS curriculums/opportunities by having a metric that is the same across the country.
I disagree that HS grade inflation isn’t common knowledge. That’s why school reports go with transcripts to provide some context.
TO favors full pay kids with subpar test scores. Children of alumni also with “less than” scores could benefit. URMs might benefit, but not nearly as much. The colleges will benefit as they will see an increase in applications, thus making them appear more selective than they are.
I suppose it is possible future waitlist kids might benefit because colleges appear more selective, thus kids will feel pressure to submit more applications than necessary, thus yields go down making it more likely colleges have to go their waitlists.
Students who actually do submit respectable-to-high scores could benefit, unless USNWR changes its methodology.
Not everyone preps for the SAT. My kid only did a handful of tests from a SAT prep book at home in the few weeks leading up to her test… Meanwhile most students at her school took summer long prep classes. Her score is high enough for any college she would like to apply to. My point is that standardized tests do allow for some students to shine that come from disadvantaged backgrounds (not saying my D is disadvantaged but her prep style could easily have been replicated by someone who is).
@RockyPA
Same story here. I’ve seen it first hand where poor kids used free test prep and scored high. One low-income student, friend of the family, earned National Merit Semifinal status using a free test prep book found in her guidance counselor’s office. Using that book - and she did use it all summer - helped her. Her high SAT allowed her to apply to and earn admittance to an Ivy. She came from a poor, rural school district. Without that test score, there was no chance for her. With it, she had a chance.
Test Optional also allows colleges who, in the past have given guaranteed merit scholarships based on test score and GPA, to potentially save money by quietly giving less merit money than in past years.
Miami University just announced, “Providing a test score will not be a requirement for merit scholarship or honors program consideration.” Prior to this admissions cycle a hypothetical 33 ACT and 3.75 guaranteed a half to full tuition scholarship. However, by removing test scores as a requirement of merit they implicitly give themselves the ability to give a lot less merit money this year.
I agree, and I would add that middle class families are the ones it hurts.
It hurts everyone by making college admissions even more subjective. We have a whole set of other colleges - community colleges and regional universities, to serve those who did not perform well in K-12 and/or attended subpar schools. The actual problem is the poor state of inner city schools, and some rural schools. That problem could be solved by school choice in urban/suburban areas, and greater technological investment in rural areas , which often lack broadband.
Standardized testing is the great equalizer. Other countries rely almost entirely upon entrance exams, which give everyone an equal chance.
Ultimately this is a political problem that can’t be solved until the impediments for better K-12 schools are removed. Band-aids don’t help and just perpetuate inequality.
From the college’s perspective, TO is a solution to a scarcity problem in elite college admissions. Take a look at the data College Board publishes titled “Total and Section Score User Group Percentile Ranks by Gender and Race/Ethnicity”. Take a close look at the percentile ranks across from different score cutoffs by group. What a dismal report. Now suppose you were an admissions officer setting admissions strategy given this playing field. If the school has a soft quota for certain groups, then it must also have lower cutoffs for those groups. That’s the landscape that the College Board reveals. But, the published SAT scores are important to rankings and the perception of quality. So, at the other end of the spectrum, TO heightens the competition among the Average Excellent students for the unhooked slots. Overall, TO is a curtain that conceals how the admissions sausage is actually made.
Based on my own experience long time ago regarding SAT, it’s a BS to think that you cannot prepare for SAT and obtain great scores just by studying on your own without going through online or physical prep courses. I had 3.0 GPA in high school (but took AP classes) because I was a lazy, recent immigrant kid from an Asian country who didn’t speak fluent English (also from a dysfunctional family), and knowing I needed great SAT score, I bought a Barron’s SAT book and studied 2 hours every day during one summer. And the SAT score I obtained was among top 3 highest in my high school, and it was higher than the average SAT needed to apply to Harvard. I still managed to get great SAT score (99.9% as I remember) even though I was no genius, as demonstrated by my college GPA of 2.9 and a professional degree GPA of the same 2.9. Basically, I was the type of student who got bored after I quickly learned the basic concepts. Got many Ds and Cs throughout my academic career. Therefore, I know that it’s actually better to study on your own if you want to get great SAT score. Because of my own experience, I had my kid study for PSAT on his own even though I could easily afford any prep courses, and he managed to become the National Merit Finalist. To me, PSAT/SAT is nothing more than showing you can self study and is not a dummy and can do well in higher level stuff IF YOU ARE MOTIVATED. As it turns out, I was not that motivated to learn in academic setting. But in the real world, I was REALLY motivated. IMO the ability to self study is way more important than taking notes and doing well on any type of tests.
Of course there is going to be outliers like what you did but I think the point is that when you look at 100,000s of high school students certain social-economic groups are going to be at a disadvantage when trying to get high scores on standardized tests.
Here’s an example of how money plays a factor. When D20 was in 8th grade, she and about 25 other middle school kids in her class took an SAT prep class for a month taught by two of her teachers (one Math, one English). The cost was around $500. After prepping, they took the real SAT just for the practice (i.e. there scores didn’t count. This was just done for “fun” and started them on understanding how to prep for and take timed standardized tests and demonstrates how money plays into test taking.
In high school, most these same kids starting prepping for and taking the SAT/ACT their sophomore year. They all had private tutors and/or took expensive test prep classes over the summer. In addition, most could afford to take these tests multiple times. This allowed them to get a higher single sitting score or be able to super score certain sections.
My D20 took the ACT three times Over 9 months until she got her optimal score of a 34. Had we only been able to afford one test without a private tutor, she would have been sitting on a 30, which would have been too low for her top college choices.
This is just one example of how money can influence getting high test scores on standardized tests.
I won’t even discuss how our family being able to afford a house In a highly ranked public school district allowed D to excel in middle school and high school and all of the advantages that that education afforded her. That’s a whole other topic.
No doubt that prepping helps for the SAT or ACT. A private tutor, like a private coach or music teacher, can help a student stay focused. So do we eliminate athletic scholarships, musical performance, and athletic ability from being considered for admissions decisions, knowing that the privately coached kids have this advantage?
Of all the parts of holistic admissions, standardized tests are the most objective and straightforward. It is possible, as several people have pointed out, to study for an ACT or SAT by yourself with low cost materials and get high scores. There are lots of these kids; they are not outliers.
So who does TO help? Wealthy applicants with less than stellar scores.
Who does TO hurt? Rural low income kids from less competitive high schools, and less wealthy kids from all backgrounds who, in a test mandatory world, could show that they are able to successfully compete with wealthier kids on an objective assessment.
When the NFL brings in college athletes from different sports divisions and conferences and gives them all the same “tests” (speed, jumping, dash times, power lifts, etc.) we think this is a fair way to compare football players from vastly different leagues. It gives NFL talent coaches “another piece of the puzzle in their overall evaluation process” (sound familiar?). How else can the NFL fairly evaluate all the different prospects who played against much different levels of competition?
The SAT and ACT are like an academic version of the NFL Combine for high school kids. For seniors from “proven” schools, it might not help them that much. For kids from “unknown” high schools, it could help them a lot.
Who does TO help?
- All of the many kids who attend schools with grade inflation. Pro: Kid looks better on paper than they actually are when compared to their peers nationally. Con: Kids have no idea how to obtain top grades as grades are averaged, raised or kids are given extra work to “earn” an A to replace a low score(s).
- It helps kids in all socio-economic levels who have low test scores and either don't want to work on it, or work on it and raise them only slightly. These kids are often not in the same category as top scorers. They might end up at a competitive school where they are lost or have large gaps.
- It helps kids who believe themselves to be competitive for the top schools but likely aren't. Sets them up for sink or swim.
- Great for kids who have excellent extracurriculars and personality and would bring a lot of socialization to a college but might not be the top students. There are kids like this at every school ( except maybe MIT and Caltech). There is always a bottom %.
- Good for kids who are grinders and work like crazy to get top grades but might not have great scores.
@websensation I am an outlier in that category too. Born into a very low SES, studied and ended up full scholarship at an Ivy. My kids are the opposite. All have top scores on standardized tests ( I had a low PSAT score in high school mainly due to lack of math access which I filled with study).
@socaldad2002 Funny how you seem ok with dismissing websensation’s experience with low scores which were raised with study but believe that your own kids experience is valid because they were high SES. Seems odd to me.
In both cases, you have a kid with low scores who studied and raised them. Seems like a strong correlation to me. Everything points to kids who work on it being able to raise scores ( some by a little some by a lot).
As a practical matter, test optional means that applicants whose SAT/ACT scores are in the lower part of the college’s range will not submit SAT/ACT scores. Of the group of non-submitters, those with the lowest SAT/ACT scores would gain a relative advantage over those with the low-but-not-that-low SAT/ACT scores.
Those with good SAT/ACT scores relative to the college’s range will still submit them and gain whatever admission advantage that having good SAT/ACT scores will give.
If the college is test blind, then the obvious benefactors are discrepent applicants whose SAT/ACT scores are worse than the other aspects of their applications, while the obvious losers are discrepent applicants whose SAT/ACT scores are better than the other aspects of their applications.
https://files.eric.ed.gov/fulltext/ED563419.pdf describes some research about discrepent applicants in terms of SAT scores and HS GPA (from 2010, using the three part SAT). Black, Hispanic, female, lower parental income, and lower parental education (less than bachelor’s degree) applicants are more likely to be discrepent with higher HS GPA and lower SAT scores. Male, higher parental income, and graduate degree parental education applicants are more likely to be discrepent with higher SAT scores than HS GPA.
Of course, any change that may improve chances for the less advantaged may only be temporary until parents with money deploy the money to help their kids do better in whatever the current measures of merit are.
“even though I was no genius”
I wouldn’t sell yourself short on this, that’s one of the definitions of genius, being able to do well like that, just walking into a test without any prep and doing well. The lack of attention to subjects you didn’t like is also one where people like Einstein, Jobs, Ramanujan also showed.
“I am an outlier in that category too.”
There are many, many, outliers (99%ile) but still outliers, so it’s possible socaldad’s experience may be more typical. In the good old days of the early 80s, many of us took a class, this was probably the beginning of formal test prep, and took it twice, like I did. We were middle class families back then.
Yes, it appears that anyone who actually prepares has a chance to do better. No surprise there. Some, who may in fact, have greater potential might improve a lot. This is not dependent on income but on willingness to do the work ( though some seem to argue that Daddy/Mommy hiring someone to lead the way discredits the actual results.
I think some people are overestimating how predictive test scores are beyond the rest of the application. Studies typically show SAT scores only offer a small predictive incremental benefit beyond a measure of GPA and course rigor. Some example studies are below. In all cases, GPA+SAT was better than GPA alone, but the incremental benefit was explaining only an extra 2-3% of variance in first year GPA, and the vast majority of variance in FY GPA was not explained by any combination of GPA + SAT stats alone.
However, selective colleges use far more criteria than considered in the studies below in their admission decision. It’s not just a choice of either admitting by UW GPA alone or UW GPA + SAT. They also consider things like course rigor + transcript review (whether courses with higher/lower grades are rigorous and highly relevant to planned field of study), essays, LORs, GC comments, awards, ECs, HS background/history, personal factors, interview, various subjective factors, … etc. I expect the incremental benefit of SAT beyond the combination of all of these factors would be far smaller than the benefit beyond UW GPA alone.
https://journals.sagepub.com/doi/full/10.1177/2332858416670601
10k Kids in CUNY System FY GPA
SAT explains 14% of variance in first year GPA
HS NYS Regents test explains 16% of variance in first year GPA
HS GPA explains 25% of variance in first year GPA
HS GPA + SAT explains 28% of variance in first year GPA
10k Kids in Kentucky Public Colleges FY GPA
SAT explains 16% of variance in first year GPA
HS KCCT test explains 17% of variance in first year GPA
HS GPA explains 32% of variance in first year GPA
HS GPA + SAT explains 34% of variance in first year GPA
https://www.edpolicyinca.org/sites/default/files/SBAC-SAT%20Paper.pdf
[QUOTE=""]
50k Kids in California Public System: First Year GPA
SAT I explains 13% of variance in GPA
GPA explains 20% of variance in GPA
GPA + SAT I explains 23% of variance in GPA
[/QUOTE]
The studies above look at first year GPA, which is the standard metric for which college admission tests are evaluated. However, selective colleges generally aren’t focused on trying to admit the class who will have the highest possible first year GPA prior to effects of a curve. Some selective colleges go so far as to make classes pass fail during the first semester and/or first year, giving students from different academic backgrounds a chance to get used to the different level of rigor and/or giving students a chance to try different fields with fear of getting a less than A grade. Selective colleges are generally far more concerned about graduation rate than first year GPA, and predictive ability of graduation rate is far lower than first year GPA. This weak prediction remains after controlling for field of study.
For example, the study at https://files.eric.ed.gov/fulltext/ED502858.pdf found the combination of GPA + SAT + SAT II + Demographics + Family Income + … could only explain ~10% of variance in 4 year graduation rate among UC students, with variation between 8% to 12%, depending on field of study. ~90% of variance in graduation rate depended on other factors. I expect that the predictive ability for graduation rate at the most selective colleges would be even smaller since few kids at HYPSM… do not graduate because the are failing out. There are generally other reasons. For example, I’d expect that failing to graduate for financial + family reasons is more common than failing to graduate for academic reasons.
Studies of colleges who have gone test optional generally show little difference in both GPA and graduation between test submitters and non-submitters. An example is the Bates 25 years of test optional study at https://www.bates.edu/admission/optional-testing/ . Some specific numbers are below:
Test Submitters – 3.16 mean GPA, 89% graduation rate
Non-Submitters – 3.13 mean GPA, 88% graduation rate
Rather than GPA or graduation rate, a more relevant difference in the study above is rate of students is major choice. There were a good number of submitters and non-submitters among all majors, but submitters were overrepresented in some majors and underrepresented in others. For example, 79% of math majors submitted scores compared to 67% of English majors and 54% of psychology majors. It’s unclear how much of that effect is due to Bates applicants who are interested in math and English being more likely to be submitters since the SAT emphasizes math and English over other fields; and how much of the effect is due to students switching out of math and English to psychology.
One of the only studies I’ve seen that looks in to the latter is the Duke study at http://public.econ.duke.edu/~psarcidi/grades_4.0.pdf . After full controls, it found that the only statistically significant predictors for switching out of a natural sciences, engineering, or economics major were being female , the admission rating of HS curriculum, and harshness of grading among specific Duke classes. SAT score did not reach statistical significance after full controls, although it did appear to offer a small protective benefit against harness of grading. Being female was by far the strongest predictor of switching out, with full controls. The Bates study described earlier found that women were more likely to be non-submitters than men , so this gender factor is probably also relevant in the major distribution at Bates.
1 Like