Yes, where can we find this study? That’s a rather broad indictment of GCs from @MITChris.
It’s linked in our blog post.
- unequal access to AP coursework across schools: https://edtrust.org/wp-content/uploads/2014/09/Inequities-in-Advanced-Coursework-Whats-Driving-Them-and-What-Leaders-Can-Do-January-2019.pdf
- audit studies of recommendations for AP coursework within schools: https://www.smith.edu/sites/default/files/media/Francis_Counselors_BEJEAP_0.pdf
- evidence that universal testing increases the # of low income and/or URM students in advanced classes within districts compared to recommendations from teachers/counselors: Universal screening increases the representation of low-income and minority students in gifted education - PMC
@DadSays — not at all trying to issue a broadside against all GCs, and I don’t think the research would support a broad indictment, either (although, as you can see in the literature review of the Francis paper, there is quite a bit of research in this area). The point is that bias — especially/including implicit bias from well-intentioned people — is common in every aspect of the application, not just the SAT/ACT, and we see our job as being disentangling all of that, and doing our best to adjust/project (and to be aware of / adjust for our own biases in the moment as well).
13 Likes
Thank you, @MITChris, for posting these links. I’d like to say I’m shocked that this stuff goes on; but, these days, not much surprises me.
Then again, it’s pretty shocking to see it so graphically like these really eye-opening articles.
I just want to emphasize again that — when you say “this stuff goes on” — I really don’t think this is intentional. I think that bias is complex and structural and multivalent, and I’m not accusing anyone of anything. I’m just trying to point out that this stuff is really complex and often counterintuitive.
(edit 
— got it, just wasn’t sure how to read the tone, internet is hard. There are other papers about essays, LORs, etc linked at annotation 10 We are reinstating our SAT/ACT requirement for future admissions cycles | MIT Admissions)
2 Likes
Intentional or not, this stuff goes on. I just had no idea it was like this: that’s my fundamental point.
ETA: the article from MIT that @MITChris added in his post above is amazing. I would encourage all to read it, as well as the footnotes. I had only read the topline article and thread headline. I really had no idea about the complexities involved in making this decision. I had simply thought it was a top school reinstating another hoop to jump through without fully appreciating what AOs are REALLY looking at. It is really refreshing to see MIT being so fully transparent about the reasons for its decision.
3 Likes
I have posted before, but will again as there are so many misconceptions about disadvantaged students on this thread. I went to an inner city high school where more than 50% dropped out. Less than 10% went to a 4 year college and essentially all of them went to a UC or CSU - back when acceptance rates were normal there. No one dreamed of private schools, let alone MIT. I took AP Calc AB my senior year, so no score to look at (5 end of the day which was irrelevant as MIT doesn’t accept AB). The only other AP offered was Biology which I also took senior year, so no score on application. I’d never even heard of any of these math or science competitions so utterly irrelevant. My EC consisted of working, and working some more. But I had a decent SAT - pre-1995 reset of 710 - and MIT took a chance that a kid with potential could make it. I will forever respect their process as they truly look for kids who didn’t have all the advantages - does it suck that literally every single legacy family I know even double legacy with donations’ kids didn’t get in (and we’re talking about a TON of kids) - yes. And neither of my kids applied, as it wasn’t a fit for them. But most of you have not the slightest idea of what it’s like to be a kid at a disadvantaged school and should really just not opine on your opinion on their success or not. Scores helped me and I fully understand why MIT needs them to find the kid at a school where more kids drop out than go to college.
17 Likes
Thank you, Darcy. I think there has been pretty close to 100% support (not entirely, but pretty close) among our alumni base for this decision, both because they tend to trust the office (maybe that’s survivorship bias), and because they know what the MIT education is like).
With our blog post (glad you liked it, @mynameiswhatever), we’re trying to persuade people that this is the right decision for us, and why (without saying it would be for other places, where we can’t do research). That desire mostly comes from our broader commitment to the public interest, i.e. trying to be as transparent and educational as we can about our own work (About MIT Admissions | MIT Admissions).
7 Likes
Thank you Chris for answering all these questions and being transparent.
I just read your blog post Diversity or Merit from 2011 and it should be required reading for all CC posters.
I’m sure you’re trying to do right by the student, but if reference #2 above is what you’re citing to support your statement about GCs, I would urge you to look more closely at the methodology. The “GCs” used in that study were ‘solicited conference attendees’ asked to ‘voluntarily participate in the study for the chance to win one of six $100 gift cards’. The authors add (proudly?) that they restricted ‘analysis to the 152 participants who indicated they have had some school counseling experience at the high school’. I don’t know if this actually got published in a peer-reviewed journal but I sure hope not.
Reference #3 (the PNAS paper) is interesting and should be followed up, but restricted to a single school district. PNAS is a respected journal.
Reference #1 is marketing material.
Appreciate the careful reading. No, it was not driven by that paper alone (although it was eventually peer-reviewed and published in The B.E. Journal of Economic Analysis & Policy; Do School Counselors Exhibit Bias in Recommending Students for Advanced Coursework?)). The core insight that all aspects of the application process — not just the SAT — are subject to both explicit and implicit bias is robust and well-supported across the literature, which is why — as I said earlier — we have to try to interpretively account for them in our process.
2 Likes
I appreciate providing such extensive references. That’s extremely rare among this type of college website publication. Website publications on other websites often read like marketing material. It’s great that MIT puts in extra effort and detail. The links contain much useful information that is interesting to discuss, but the conclusion from the papers are often nuanced without a simple overriding conclusion. Specific reference links are below, with comments:
Essay (https://www.science.org/doi/10.1126/sciadv.abi9031)
Finds that essay content is correlated with both income and SAT scores. It does not evaluate whether essay quality is correlated with income. For example, the study found that students who wrote about travel and China were more likely to be high income. It did not evaluate whether the high income students who wrote essays about travel and China wrote higher quality essays or were more likely to be admitted.Grades (https://eportfolios.macaulay.cuny.edu/liufall2013/files/2013/10/New_Perspectives.pdf) – Finds that while both SAT and grades are both correlated with income, SAT was much more correlated with income than both grades and class rank within the evaluated sample.
Advanced Coursework – Discussed in more detail in posts above
LORs (Lost in Translation: Elite College Admission and High School Differences in Letters of Recommendation // CurateND) – A PhD dissertation in which a computer evaluates LORs by measures such as word count and number of words in certain groupings, such as number of “subjectivity” or “positivity” words. The author finds that significant differences exist both between different school types and demographic groups. For example, the author found that typed LORs about females were significantly more likely to use “positivity” words while typed LORs from private HS were significantly less likely to use “positivity” words. Less clear is how the difference in number of “positivity” words influences chance of admission.
“Just About Every Other Factor” (Commentary: What's less fair than the SAT? You might be surprised) – LA Times opinion piece with significant errors. For example, it references the essay study linked above as evidence that high quality essay content was more correlated with income than scores, even though the referenced study does not evaluate quality of essay.
The combination of influences makes it difficult to draw a simple conclusion of how a test optional policy influences student body. Other studies/evaluations of test optional colleges have found that kids who are admitted test optional consistently average lower income and other markers of disadvantage that those that are admitted as test submitter (https://www.nacacnet.org/globalassets/documents/publications/research/defining-access-report-2018.pdf , Wayback Machine, https://www.bates.edu/admission/files/2014/01/25th-Year-SAT-report-Stanford-6.3.11-wch.ppt ) , suggesting that that many disadvantaged kids do manage to be admitted without scores.
However, I understand that MIT is a unique school that has many differences from the average college and may show different patterns and have different needs. I also agree that SAT score can highlight some kids would not be identified otherwise, including some lower income kids. Some specific lower income kids would be admitted under a test considered system who would not be admitted under a test blind system.
1 Like
Thank you for this thoughtful review of those citations (and your compliments on our site). I think we definitely do not want to make blanket statements for the profession, or think anyone has the research to support them. Probably the most interesting feedback I’ve gotten from some TO colleagues privately is: “hey, look, I bet this research is true for MIT, but the way you wrote it has allowed it to be used as a cudgel against all of us in some broader culture war.” If I could go back, I think we’d try to make it even clearer we were just trying to talk about MIT, and my hope is that every college does/is doing the research to make the right choice for them and their students.
3 Likes
CA resident, rural area. It was no problem to get seat for SAT from October 2021 onwards around us.
4 Likes
I appreciate MIT’s transparency and this conversation. Does anyone think other colleges will follow MIT in reinstating a test requirement? I know UNC announced a few weeks ago & Florida schools have kept the requirement.
As a parent with one S23 left to apply, I have a new appreciation for test scores.
4 Likes
this is stupid, but kinda funny. hopefully doesn’t offend anyone. interesting that this topic is well known enough out in the world to make the Onion…
2 Likes
Back when the SAT verbal (as it was called then) section was mostly a vocabulary test, there were common accusations that the many of the words reflected a higher SES bias (e.g. words from sports were claimed to be more likely to be from sports associated with higher SES, or “preppy sports”). So that type of accusation is not exactly new.
They do if the applicants can provide them, it’s just not available to everyone so it’s an additional barrier and can’t be used broadly as mentioned above. The kids in the bay area I know that get into MIT have taken Calc in 10th or 11th and Stats as well so they have those AP scores (hint -they’re 5s), the AMC or AIME as well. These are public schools btw, that being said you can’t assume the bay area is representative, it’s an outlier.
“Of course, to even have an AP calculus score by the time one applies to colleges in 12th grade, one has to have been on the +2 math track (calculus in 11th grade), which is a lot less common than being on the +1 math track that reaches calculus in 12th grade.”
It’s a lot more common than you think and the middle school placements are based on tests, which MITChris said is indicative of the first year at MIT. And you can retake these placement tests in middle school if you want the accelerated track.
“They could do Algebra 1 and Geometry freshman year, Allegra 2 and pre-calc in sophomore year, and then calc AB or BC in Junior year (usually a full year class fall -spring blocks)”.
I don’t think you can two math classes in one year, I guess you could if one was an elective. But that’s not a typical MIT applicant profile I mentioned, it’s a lot cleaner that:
Precalc-Honors in 9th, Calc BC in 10th, Stats in 11th, DE (MV, Linear Algebra, Diff Equ) in 12th.
I think it depends on their original motivations leading to the test optional switch and what has changed since then. For example, the hundreds of colleges that chose to switch to test optional prior to COVID had a variety of different motivations. Those motivations probably are still largely present, so I’d expect their decision to go test optional to be unlikely to change. I’d make a similar comment about colleges that were considering switching to test optional, and the pandemic gave them an additional push in that direction.
The colleges that are most likely to switch back to test required are colleges that had mot been considering test optional and instead primarily switched due to many students being unable to safely take the test, with the COVID-19 pandemic. When those colleges believe that the COVID-19 pandemic is well enough under control that students can safely take the test, then some will switch back to test required.
It’s “some” rather than all because there are also additional factors that may have changed. For example, some colleges may like the increase in applications from test optional kids and find it undesirable to lose that; some colleges may fear negative publicity from such a change; some colleges may change their mind after experiencing the forced trial; etc.
Calc BC in 10th, Stats in 11th, DE (MV, Linear Algebra, Diff Equ) in 12th.
(Apologies as I can’t remember how to quote reply @theloniusmonk and this question may be more appropriate in another thread, http://talk.qa.collegeconfidential.com/t/high-school-math-acceleration-thread/)
For @MITChris for the above math track, would such an applicant have a better chance by doing DE math in both 11th and 12th, rather than AP stats in 11th and DE only in 12th? Or would there not be a significant difference between AP stats in 11th/DE in 12th vs DE in both 11th and 12th?
I meant that I’d expect math classes at MIT to generally emphasize longer and more complex calculus+ questions, and generally not emphasize answering simple, multiple choice algebra/geometry questions at a rapid pace.
I agree that showing evidence of good preparation for success in this type of complex calculus+ class is not equally available to everyone; and I’m sure MIT considers the context of the student’s background and options available to the student when evaluating the applicant.
For example, I was accepted to MIT many years ago from a basic, public HS. My family wasn’t wealthy and my school wasn’t highly resourced. I believe it was a typical public HS, from a typical SES type of family.
Like most HSs, my HS did not offer a large number of AP classes. It offered calculus, computer science, and English. I took the first 2 prior to my senior year. Like most public HSs, mine did not have a standard math path that led to calculus before senior year. Instead I was accelerated enough to take calculus before senior year due to a unique situation suggested by a HS math teacher and discussed with GC that involved independent study instead of taking pre-calculus HS math classes . After finishing the math offered at my HS, I was a half-time student at a nearby college while in HS. I took post-calculus math, as well as other courses, and received A’s. My ECs were mostly math-related, including Math League and math competitions. My HS and I achieved some regional recognition in math competitions, but nothing beyond regional.
So in this example, non SAT/ACT evidence of being adequately prepared for post-calculus math classes includes contributions from the following, as well as contributions from nearly every component of the application. Many components of the application can contribute information, but I also realize that most of the things I did will not be possible/available to the typical HS student.
- Took post-calculus math classes at university and received A’s
- Transcript showing pattern of success in math and math-related fields, and taking highest available level or beyond highest available level of classes
- Strong LOR from math professor in post-calculus math class
- Positive comments from GC discussing unique math background and experiences
- Success in math related ECs and math competitions
- Personal comments with additional information about motivations, interests, experiences, future plans, … in essays, statements, and interview
1 Like