New pet peeve: test optional at top schools

The logical fallacies are twofold:

  1. The one I mentioned is that you made the claim that since your interlocutor had suggested eliminating one criterion in the interest of equity, that they were necessarily in favor of eliminating all criteria regardless of the effects on equity. That is a classic application of the slippery slope fallacy.
  2. You presented the unsupported claim that all criteria are barriers to equity, without providing any actual evidence for that. You did this in the service of the slippery slope argument, so I don’t see it as entirely separate, but technically it is a separate logical fallacy, specifically begging the question.
2 Likes

I wonder whether the low correlation of 0.04 of high school GPA with parents’ income in your first table holds for the high school transcript as used in your second table, which includes strength of schedule and number of APs.

Conversely, I wonder if the high school transcripts in your second table were quite as predictive if they did not include strength of rigour and number of APs.

Unless we know whether those are really comparable data points, I find the juxtaposition potentially misleading.

1 Like

Great segment on TO on “This American Life” (NPR) today.

In sum, you want to see what happens when you take away standardized testing? Look at UT Austin. They have been taking the top 10% (now 6%) of high school classes, regardless of scores or SES for years.

Yes, those from bad high schools struggle to catch up. But after a semester or two, with some support, they do.

If the goal is to find “qualified” students (defined as those who thrive and succeed), schools can do that without scores.

To the extent scores add predictive value on success, it is probably because they correlate high SES, which correlates with better high schools.

5 Likes

I think GPA is a bit difficult because an A in a low ranked public school is not the same rigor as a top 20 prep school. There’s also a lot of difference in course offerings (for example my school only offers AP courses junior/senior year and there are only 7 you can take and 3 are night courses). I do think that SAT/ACT isn’t entirely representative of a student’s academic level but I think both need to be reported in combination because it offers a scale of reference

3 Likes

They were different studies. The UC study looked at UW GPA. The Ithaca study considered transcript including GPA + strength of schedule + # of AP credits. The latter is consistently far more correlated with measures of college success that the former. It’s unclear from the Ithaca study how correlated their GPA + strength of schedule + AP metric is with income, but it is clear that the students who were admitted test optional under Ithaca’s system that considers these factors had much larger portion of underrepresented group than test submitters. Specific numbers are below:

Ithaca Applicants
Test Submitter – 10% Pell, 26% URM
Test Optional – 17% Pell, 40% URM

Ithaca Admits
Test Submitter Admits-- 15% Pell, 22% URM
Test Optional Admits – 29% Pell, 35% URM

Ithaca Enrolled
Test Submitter Enrolls-- 18% Pell, 19% URM
Test Optional Enrolls – 30% Pell, 31% URM

2 Likes

You’re making your own assumptions. The other poster effectively stated that better representation by underrepresented groups is the goal and she’s for TO because it helps achieve that goal. So I ask her if she would be for getting rid of some other criteria which would help get closer to that goal.

No, I didn’t say she would necessarily be in favor. I asked her whether she would be in favor.

Don’t you agree that many other admission criteria also disadvantage students from underrepresented groups?

Some of these comments are getting a bit snarky. Might I remind members of the forum rules: “Our forum is expected to be a friendly and welcoming place, and one in which members can post without their motives, intelligence, or other personal characteristics being questioned by others."

Additionally, “College Confidential forums exist to discuss college admission and other topics of interest. It is not a place for contentious debate. If you find yourself repeating talking points, it might be time to step away and do something else… If a thread starts to get heated, it might be closed or heavily moderated.”

http://talk.qa.collegeconfidential.com/guidelines

Large parts of this thread have devolved into a conversation better suited for PM than informative for the community at large.

Final time posting this. If the circular conversation continues, I will close the thread without comment.

For the moment, I’ll ignore the expectation that decision makers consider costs/consequences of actions, even if the specific costs/consequences are not explicitly repeated in every post. Due to the Harvard lawsuit, we have more information about admit rate by income for different groups at Harvard and how that admit rate and can better estimate how admit rate would change if parts of the application were made optional at Harvard than at other “top schools,” so I’ll use Harvard as an example.

The Harvard internal study at http://samv91khoyt2i553a2t1s05i-wpengine.netdna-ssl.com/wp-content/uploads/2018/06/Doc-421-157-May-30-2013-Report.pdf lists the admit rate by income level, which is summarized below. I only listed non-URMs to minimize variables.

Harvard Admit Rate for Class of 2009-16
White Students – Low income* has 12% admit rate, Others have 10% admit rate
Asian Students – Low income* has 10% admit rate, Others have 7% admit rate
International Students – Low income* has 8% admit rate, Others have 6% admit rate
*Harvard considers less than US ~median income as “low income”

Note that low income applicants at Harvard have a higher average admit rate than applicants who are not low income. So if the admission for the specified classes was done by a purely lottery system without considering the application at all, then the average admit rate for low income kids would go down. Admission by pure lottery that does not consider applicant does not guarantee an increase in representation of lower income kids. Instead you’ll have more effective results by considering how specific admission criteria/consideration/hooks are correlated with income. I am ignoring feedback effects or the purposes of this example, which no doubt would influences results.

However, making everything optional does not mean a purely lottery admission system. A pure lottery would be making the admission fully blind, not fully optional. Making everything optional means that applicants can pick and choose which criteria to submit to the college. The college presumably will not admit any applicant who chooses to submit nothing, but applicants can choose to submit only the criteria in which they excel and not submit criteria in which they do poorer than expected based on the rest of the application. For example, the kid who has great grades+scores, but was obnoxious and insulting in his classes can choose to not submit LORs. Or the kid who has great grades+scores, but doesn’t want to take the time to write take the time to write essays for individual colleges can choose to not submit essays. It’s not a given that making an admission criteria optional like this means % low income kids will go up. Rather than indiscriminately making everything optional, if the goal is to increase lower income kids, you need to consider which groups are likely to do poorly on the optional criteria and do well on the remaining criteria that will be considered.

For example, suppose Harvard went GPA optional instead of test optional this year. By “GPA optional”, I mean the college can see the course rigor, number of AP classes, and such; but does not see grades – only pass or fail. The winners in a GPA-optional system would be kids who have great scores and other parts of the application, but their GPA is lower than expected based on their high scores and great other parts of application. I’d expect Harvard applicants who have great scores, but weaker GPA than expected by their great scores to be primarily high income applicants rather than low income students. So I’d expect if Harvard went GPA optional instead of test optional this year, there would be an increase in representation of high income kids and decrease in representation of low income kids. If the goal is to increase low income kids, I expect you want to make GPA required and make the optional parts areas where low income kids are likely to do worse than expected based on GPA, like scores.

1 Like

The Harvard data may be skewed because it may have given low income applicants special consideration. I also find it ironic that some would argue from both sides:

  • Those of you who favor TO also seem to believe that GPAs are more inflated in more affluent school districts. If you believe that, wouldn’t you also believe that removing grades from consideration would help less affluent applicants?

  • LoRs from most GCs and teachers at public schools, especially ones in poor school district, are known to be much less helpful than those from expensive private schools. If LoRs were eliminated, wouldn’t the advantage enjoyed by aflluent private school students disappear?

  • Aflluent families can hire consultants/specialists to help craft application essays. If we remove those required essays, wouldn’t that advantage they enjoy also disappear?

I don’t think it matters if grades are inflated or not. Driven kids are those who do well GPA wise anywhere they are. There is a goal set for an A and they make it. The bar is set lower (usually) in less affluent school districts, but that doesn’t mean the students there are less capable than those in other schools. It’s not difficult for the driven kids to fill in educational gaps if given a chance.

I have a feeling adcoms can tell which kids are coming from expensive private schools and which ones aren’t so are able to read the LORs accordingly.

One could say the same about SAT/ACT scores - as it’s true. Adcoms would know the 1300 coming from a smaller school is a great score there (vs a school where a 1550 is a top score), but a 1300 would also need to go into the stats and few tippy top schools want many with lower scores (hurts image), even if they know those students are capable of doing well. If they don’t need to see the 1300, that eliminates having to include it in their admission stats.

The same thing applies for the art student (or many other majors) whose success really doesn’t depend upon a high SAT regardless of where they come from.

2 Likes

If someone claimed that GPAs are more inflated in affluent districts, I missed the post. The Gershenson study is usually referenced for this claim, which I believe found a median GPA of 3.0 in more affluent districts vs 2.6 in less affluent. That is a significant difference, does not necessarily mean one is more inflated than the other.

My claim is instead that every study I’ve ever seen that reviewed the relative correlation with income between HSGPA and SAT score found SAT scores were far more correlated with income than HSGPA. This includes studies sponsored by the College Board. For example, the study at https://www.edpolicyinca.org/sites/default/files/R_Kurlaender_Mar-2019.pdf reviews how many SES disadvantaged students are in the top 10% of the UC applicant pool under several admissions models including only HSGPA, only SAT, and both HSGPA + SAT. A summary is below. The portion of top UC applicants who are SES disadvantaged is a noteworthy 29% for an admission system using HSGPA alone, but only 8% for SAT alone.

Top 10% of UC Applicant Pool
Only HS GPA Considered – 29% of top UC applicants are SES disadvantaged
Only SAT Considered – 8% of top UC applicants are SES disadvantaged
Both GPA+SAT Considered – 10% of top UC applicants are SES disadvantaged

As discussed, whether removing grades helps more or less affluent kids depends on what other criteria you are using in place of GPA to make admission decisions, which you did not specify. As suggested by the study above and others that have been mentioned in this thread, if you use emphasize SAT in place of GPA, it is likely to help higher income kids, not lower income kids.

If I use LDC as a proxy for affluence, the Harvard lawsuit sample suggests that teacher LORs are the analyzed criteria with the 2nd smallest difference between more and less affluent applicants, after grades. This may relate to being easier to stand out as a big fish in an open admissions public HS than a selective private HS. For example, the Harvard Reader Guidelines gives the following reference for assigning a high LOR rating. The quoted statements used in the reader guidelines all compare the applicant to other students within the HS, so it follows that it may be easier to achieve those statements in a HS that is not highly selective.

Harvard Reader Guidelines: School Support

  1. The best of a career,” “one of the best in many years,” truly over the top.
  2. Very strong support. “One of the best” or “the best this year.”

I haven’t seen a review of how well essays correlate with income. This makes it a matter of speculation. The important criteria for lower SES representation is not whether there is a non-zero correlation with income. It is instead whether the correlation is higher or lower than the remaining criteria that will be used in its place to make decisions. If the remaining criteria emphasizes SAT score, I suspect removing essays will help high income students rather than low income students.

2 Likes

This is widely vectored on CC, but I wonder how much truth there is to it.

And I mean, sure, let’s go ahead and stipulate that a T20 prep school may well be different. But there are about 24k high schools in the country, so you’re talking about less than a tenth of a percent of all high schools there. So let’s make this meaningful, shall we?

I would suggest that the claim quoted above has not been demonstrated to be correct. However, even if it is, I issue a serious challenge: Demonstrate that an A from a high school in the bottom quartile (however defined, though quantitatively/measurably, please!) is meaningfully different in terms of rigor from an A in the same course from a high school in the top quartile (defined similarly).

My prediction: It can’t be done.

I don’t, and have said so explicitly.

I do agree that grades are higher in more affluent school districts, but higher does not necessarily mean inflated.

1 Like

I don’t see how grades could be helpful to compare kids from different schools. They are only useful comparing kids from the same schools. Every school is so different.

That’s what AOs say they do with them, from everything I’ve read. They compare coursework and grades to classmates.

That creates a problem in a TO world, if a sole application comes from a high school that an admissions office doesn’t know well. Why would an AO take a chance on an unknown school when they have gobs of kids from schools they know? I can see an AO looking for a score to validate a high gpa from an unknown high school. Without one, a kid from an unknown school has little chance. TO still benefits those from private schools because the colleges know what they are getting.

There’re plenty of such comments/claims just in the threads like this (and likely also in this very thread).

Good private schools with resources are known to direct their students to apply to different elite colleges to maximize their chances of admissions. It’s also been known that this practice allows these schools to write more glowing LoRs for each student (so the word “best” can be used in more instances).

Me neither. Could this mean potentially it’s even more correlated with income than even the test scores?

I agree with what you said, which is why I think standardized testing should be paired alongside it. Because with GPA on its own, for that exact reason you can’t truly determine the rigor between schools. With SAT/ACT on their own, it’s not entirely accurate because of variables such as extra time, being bad at test taking, unavailability of testing centers in low funded areas, or potentially flukes/sheer luck of what’s on the test. I think both should be compared side by side because they at least offer some sort of scale of reference.

Letters of recommendation should be eliminated:

1 Like

Yet the studies that have posted in this thread all show no significant difference in college GPA or college graduation rate between test submitter and non-submitter applicants at selective colleges. The Ithaca study posted earlier this weekend showed a negligible difference in prediction of college GPA between transcript (GPA + rigor +AP credits) vs transcript + scores. The scores only added 1% variance explained beyond transcript.

If standardized testing is really critical to validate transcript, wouldn’t we instead see a significant difference in outcome between students who have that score validation and students who do not? Perhaps admission officers at selective test optional colleges have other ways to validate besides just looking at GPA in isolation without considering anything about the classes that were taken, high school profile, and non-transcript parts of the application?

3 Likes

I don’t quite follow how this works, since I argued that claims that there are meaningful differences in rigor between high schools (assuming the same courses, and leaving aside the far extremes of the distribution) are being asserted without evidence, and so should be questioned.

Basically, if my suspicion is correct, you can truly determine the difference in rigor between schools, though only in a trivial sense: It’s optimal to ignore it, because it doesn’t provide anything meaningful.

(I suspect part of this may not be a lack of difference between high schools so much as it’s the undefined nature of what “rigor” is, and the lack of certainty on whether “more rigor”, however defined, in high school is actually useful for postsecondary success.)