i wrote some words on the internet. you may be interested in reading them.

MITChris · October 18, 2011, 10:37pm

hi CC -

as we begin rolling into EA over the next few weeks i'm going to be on CC (even) less. however, i did want to share some things i've been writing over on the mitadmissions blogs. these entries have been a long time coming, and they are constituted of conversations which began on here and crystallized over there.

when i first came to CC in 2009 i'd never been to or heard of it before. i just knew that it was part of my job to post here. over time, i've really developed an affection for this community. it's not an unconditional love. there are still a lot of things about CC that i don't like. but overall, i'm very proud of the people we have here in MITCC. i think that through the efforts of me but mostly mollie, piper, oasis, mikalye, laura, k4r3n, and lots of others (i am so sorry if i forgot you, i'm just writing this before bed) the messages that i've wanted to get out there have gotten out there, and that many of you (prospectives, applicants, admitted/deferred/denied) have all helped share and spread the good word as well.

here is one blog post i wrote which (as i said before) had its beginning on several CC threads:

Diversity</a> or Merit? | MIT Admissions

there is another one going up (about holistic admissions) on thursday, and hopefully another one (about statistics) the week after. i'm scheduling them ahead of time so that they will still go up even after i get buried in applications, but i wanted to create this thread to thank all of you for giving me a space to work out some of these thoughts, and to hopefully help communicate clearly and authoritatively the answers to questions, comments, concerns, and conversations we frequently have in MITCC and elsewhere.

y'all are the best.

CPUscientist3000 · October 18, 2011, 11:43pm

that was a really great read. thanks!

mathboy98 · October 19, 2011, 1:07am

I think the article was a nice read, although this one bit I included above perhaps begs the question: are we introducing a new term (or terms) which we don’t know the meaning to, namely “sufficiently” and “success” , in process of clarifying the meaning of merit? After all, it might be reasonable to guess that those who would be prone to raise objections are defining success differently.

Whenever there is an “and” involved, there is the question of which side is weighted more highly, or in what sense there is an equality between how heavily they are weighted.

It seems as if to resolve these questions once and for all, a stronger claim must be made, namely that diversity and merit are completely not distinct. That the strengths brought by diversity are as fundamental as any other measures of “merit” to building a class which successfully upholds the school mission.

MITChris · October 19, 2011, 12:13pm

mathboy -

thanks for the feedback. always helpful to know when i’m not communicating something clearly enough, and i may go back and edit the post to reflect this.

by “sufficiently” and “success”, we basically mean “will graduate from MIT by more than the skin of their teeth”; namely, that they will be able to do the work and also pursue their area of interest and not just be constantly trying to keep their head above water.

what i was trying to communicate with that “and” was not that there is a weighted more highly question, but that no student is admitted without both things. but yes, your point about diversity itself being a strength is taken, and is the subject of the next post.

k4r3n2 · October 19, 2011, 12:44pm

I like entries like this. I think that it’s really hard to convince people that admissions decisions are reasonable, or that they’re actually truly for the best, or that girls really don’t have an easier time getting in because of their gender, but I definitely commend you for trying :)

And anyway, regardless of what the rest of the world believes, I know from experience that MIT is an awesome school full of awesome, diverse, and interesting people, and that the admissions office does a fantastic job each year picking the final class.

PiperXP · October 19, 2011, 1:12pm

I feel like people are still taking “diversity” to mean purely in racial or gendered diversity. Sure, a school of all males would be boring, maybe I was admitted because I was both qualified and contributed to the gender balance.

… on the other hand, I suspect far fewer pilots apply to MIT than females or males - I would say that lent to my “diversity” quality far more than my race or gender would. I suspect that of a female friend who is an orienteer, or a black friend who is a rockstar at puzzles (was on the winning Mystery Hunt team last year), etc. Musicians, athletes, academic superstars, people with bizarre inventions in their garage - there’s so much more to “diversity” than race and gender.

Basically, I think it might be time for another round of “explain that admissions is complicated, no really, you can’t simplify this into two minority categories”.

mathboy98 · October 19, 2011, 1:37pm

Exactly, I agree completely. If that were the definition of diversity being employed, I would be objecting to the philosophy of upholding diversity as well. Diversity should involve paying attention to distinct sorts of intellectual merit and other qualities that aid in the desired notion of success. It is a case where those involved are using the term “diversity” correctly, but the masses misinterpret it to take on the limited meaning it’s commonly associated with.

@ Chris - thanks for clarifying things objectively as always. Lack of clarification/transparency has easily been my biggest criticism of how students and educators have approached thinking about these kinds of things.

As you can guess, the reason I brought up that diversity must be quite interwoven with merit is that, all games aside, MIT is a super-selective school, and we all can probably agree it’s ridiculous to say the goal is to admit people who will merely stay afloat. The merit to attend MIT must surely involve thriving rather than merely surviving.

What I really liked about your post was that it emphasized how you can effectively learn volumes from any good university library. As someone who personally has experience heavily over-using libraries, I know this to be true. I agree that the central difference between admitting someone to a university and admitting someone who can learn is in considering what is distinct about the opportunities afforded by this specific university, and how well these will be used.

MITChris · October 19, 2011, 2:57pm

“Basically, I think it might be time for another round of “explain that admissions is complicated, no really, you can’t simplify this into two minority categories”.”

yep, this post is coming up.

mathboy - agreed on all points.

MITChris · October 19, 2011, 3:37pm

actually, you know what, maybe i’ll post the draft entries of my next two blog posts here, and y’all can help me rework them before they go live on the site. this feedback is really helpful, and i’ll just reverse the order of operations.

here’s the one scheduled to go live on thursday:

As I was writing my entry on Diversity or Merit, I realized that there was an premise undergirding my own opinion that I should probably do some more work to articulate and explain. 

Specifically, this quote: 

*
One of the most important things about college is its role as a socializing institution. College is a place where you meet all sorts of people from all sorts of backgrounds and you learn from each other. It is, properly constituted, an environment which fosters intellectual, ideological, and social cross-pollination. You need diversity - broadly defined - for college to matter.
*

So let’s talk about holisitc admissions and why it matters.

By “holistic admissions”, we mean, essentially, that our admissions process takes into account many different factors. Specifically, we use “holistic” to differentiate against processes which only consider so-called “objective academic data” (and I am air-quoting so hard I might sprain a knuckle here) in their admissions process.

Sometimes, holistic academic processes are criticized as being misguided. “Academic merit”, it is suggested, should be the sole dispositive factor controlling the admissions process.

Suppose, for a moment, we didn’t have a holistic process. Suppose that we wanted to build an application system around “merit”, defined, as it always seems to be, by test scores and grades. (I’d contest this definition, but will concede it here for the sake of argument).

We could build an working admissions system around this premise. A very simple version might look something like this: create an index by multiplying SAT score and GPA. Rank-order the list by the highest product of the two, admit the top 1,000 students automatically and deny the rest.

Would this really be a better admissions process? Well, I guess I could go home pretty early to get a full night’s rest after turning on the Admissions Machine the day after applications are due. Then again, I’d probably not sleep very well if this is all I did. And the students we admitted wouldn’t feel as home here. Because our fundamental work is to build a community, and this admissions machine would not produce a community. It would produce a bunch of test scores and GPAs crammed into classrooms and dorms.

Even worse, such a system would produce a very thin population, because the slice of students who top the SATs are a comparatively homogenous bunch. Remember, the best predictor of SAT score is parental income, and the best predictor of parental income is their parent’s income. If scores were all we cared about, then our campus would be overwhelmed by socioeconomically similar people. That’s limiting. It’s limiting because of the scope of issues and problems which people have experienced, and it’s limiting because of the solutions which are likely to come to the minds of people. Less abstractly: it lends itself to group think and experential echo chambers. At that point, it’s not a college as I defined it above: it’s a Great Gatsby garden party. 

I think that most people intuitively understand this. I think that most people intuitively agree with the idea that the best function of an admissions office is to constitute a class of sufficiently prepared individuals who have a diversity of skills, experiences, knowledge, background, who can all learn from each other. That there is something important about considering things besides test scores and grades - things like teacher letters, essays, and backgrounds - in a holistic process.

Now, in a holistic - and fundamentally subjective process - not everyone is going to agree with every decision we make. Some people would strike a slightly different balance in terms of class composition. Students who were admitted tend to think the schools that admitted them made the right decision; students who were not admitted quite understandably disagree.

Point is: when people bring up hypothetical comparison cases, saying “well this person got in, and this person didn’t, and that’s not a good decision”, it’s really just arguing over the margins. Folks aren’t really disagreeing with the overall model of holistic admissions. They just don’t like the output any particular iteration of the model might produce. But they agree in principle. Because selective college admissions is about finding the right mix of students who will, as a collective, form the best class that can be admitted to that particular university at that particular time.

In a certain sense, you could say that our job, as an admissions office, is emphatically not to admit the “best students” to MIT, but rather to admit those applicants who will become the best MIT students. We are selecting the right mix of ingredients from which MIT graduates will be produced. This is why, incidentally, David put so much emphasis on making the most of your opportunities. We don’t care as much about what you’ve done so far as much as we care about what you’re going to do at MIT. 

The idea is best understood with a metaphor drawn from my favorite sport of football:

MIT isn’t the end zone.

It’s where you get the ball when you start the next drive.

MITChris · October 19, 2011, 3:39pm

and this, two thursdays from now:

Over the last few weeks I’ve posted entries about diversity vs merit and the holistic admissions process. And while I hope that these entries have helped contribute some insight into the admissions process, one common complaint in the comments on the entries was about a lack of data to accompany and support the claims I had made. As one commenter put it:
*
MIT should release the full set of admissions data stripped of personally identifying information and let the community analyze it, because in the scientific community we trust data and analysis, not assertions.*

So let’s talk about admissions data.

First, I’d like to say that I’m a huge fan of social science and statistics. I read 538 and Football Outsiders every day. When it comes to baseball I’m a converted sabremetrician. I believe in the scientific method, which is to say I believe in data-driven analyses of phenomena, empirical evidence, and testable hypothesis as the best, and sometimes only, route to understanding most things which occur in our universe.

But there is a problem with social science, and that problem is this: sometimes, you don’t have all of the data, either because it is unavailable to you, or because something can’t be adequately captured by existing ontologies. And then, if you try to build a model based on these incomplete data, you are liable to draw erroneous conclusions about what matter.

At its most basic form, it’s post hoc ergo propter hoc - “after this, therefore because of this.” The rooster crows, then the sun rises; all hail the befeathered Sun King. In more complex forms, it’s a very subtle misattribution of traits based on the ontologies used to characterize them, which becomes an epistomelogical exercise: what is measured and how it is measured, whether the trait measured is determinative or descriptive, and so forth.

But let’s back away from the theory for a moment and ground this in some concrete examples.

Here’s another comment from the diversity vs merit post about SAT scores:

This pretty much sums it up:
SAT Math
750-800 15%
700-740 10%
650-690 5%

From what you wrote you’d think being in the 700-740 range and being in the 750-800 range doesn’t have much impact on your chance of admission, but there’s a 50% difference.



The data above is drawn from our statistics page, where, in the interests of transparency, we publish test score ranges we are required to release as part of our common data set.

Now, I and others are on the record as saying that we admit people, not test scores, and that there is really not a difference in our process between someone who scores, say, a 740 on the SAT math and an 800 on the SAT math. So why, as the comment asks, is there such a difference in the admit rate? Clearly we DO prefer higher SAT scores!

Well no, we don’t. What we prefer are things which may coincide with higher SAT scores. For example, a student who receives a gold medal at the IMO is probably more likely to score an 800 on the math SAT than a 740. But if we take an IMO medalist (with an 800) over random applicant X (with a 740), does that mean we preferred an 800 to a 740? NO. It means we preferred the IMO medalist, who happened to get an 800!

The same goes for valedictorians. Almost half of the class of 2015 were valedictorians of their high school. Aha! MIT must consider class rank in our application! Well no, we don’t. And well more than half of the valedictorians who applied were not admitted. However, when you take the applicant pool as a whole, the proportion of students who are just straight crushing high school and have some sort of extra academic distinction that we care about is probably higher than the proportion of students who are merely doing OK in high school but have that sort of distinction.

You see what happens here. It’s correlation masquerading as causation. And it happens with whatever data we do release. The admit rate for students from Wisconsin went up 2%, MIT must really want applicants from Wisconsin! The admit rate for orphan Asian tapdancing robotocists went up this year! MIT must be filling out its electronic dance team!

What’s happening here is a fundamental confusion between process and result. When we say that the admit rate for students with a 750-800 was 15%, we are not describing their chances procedurally. Instead, we are releasing the data which were the result of our process.

Think of it as the difference between life and a fossil. A fossil isn’t the living, breathing plant or animal. It’s the stuff that’s left behind at the end of a process. Or think of it like a shadow. A shadow is not the thing which casts a shadow. It’s where the light isn’t.

That’s how our admissions data work. It shows you where the decision wasn’t. It shows you the shape of our decisions, not their basis. It shows you an accretion of results: the sediment which dropped to the bottom of the delta, and not the moving river where the actual action happened.

And just like you can’t recreate a velociraptor from a fossil, you can’t reanimate an applicant from the shadow of their data. This is why I hate “chance” threads so much. When an applicant says “I have X SAT score and Y GPA, what are my chances to get into MIT” it’s not a question I or anyone else can answer. Because, within certain bounds of sufficient academic preparation, the decision isn’t made on these easily extracted and quantified points of data. The decision is about everything else.

The response to this, of course, is “well, so release the data on everything else!” How? How can we meaningfully quantify how much a teacher supports a student? How can we meaningfully quantify that particularly poignant essay which shows a student’s resolve, or that particularly funny essay that makes us love their personality? Even if we did construct, ex nihilo, categories with which to quantitatively characterize these interactions and experiences, isn’t that just subjectivity wearing an objective mask? I don’t think that “Rate this applicant’s leadership from 1 to 5” is a particularly objective exercise just because we slapped a number in it. Trying to convert inherently subjective interpretations to objective quantities is like wearing fashionable glasses of the wrong prescription: sure, it looks hip, but it just clouds your vision.

The point I’m trying to make is basically this:

For a holistic admission program like ours at MIT, the data we release, and the data you see posted in “chance” threads and elsewhere, are no more than the shadow cast by the living, breathing application. And it’s impossible to quantify and “data-ize” the rest of the application, so we don’t. And for those bits of data that we do have and don’t release, it’s basically because those data would not be any more helpful in quantifying the unquantifiable, but would contribute to unsupported and inaccurate confidence that the data tell the story, and that from the data alone a meaningful relationship can be uncovered. But that’s not how the process works. That’s not what the data do.

I understand that for the initial commentor and others this may be an unsettling explanation. MIT is a place which loves data, which has faith in data being able to explain everything, and any explanation which questions the ability of data to capture all contours of any given phenomena seems suspiciously unscientific.

But Clay Shirky once gave a talk about how memes - jokes, YouTube videos, lolcats, whatever - spread through the Internet, and he said something to the effect that the physics of memes were more like the physics of weather than the physics of a falling object. We understand how things fall pretty well, and we can be pretty accurate in our understanding of when and where and how fast it will drop. But even though we have reams and reams of data about the weather, because of its utter complexity the best way to characterize what will happen the next day is often no better than “partially cloudy with a chance of rain.”

Well, the physics of holistic admissions are akin to Shirky’s idea of “social weather”, only moreso, because people, in social interactions, aren’t obliged to obey the laws of physics. Based on easily apprehendable information, you might know roughly what the temperature will be, and hazard a guess as to whether it will rain. But until all of the ingredients mix together at a precise moment in the process, you don’t know whether the sun will break through the clouds to settle on a single blade of grass until, suddenly, it actually happens.

compact · October 19, 2011, 4:11pm

^ That is an awesome post. It clearly explains that a correlations between two parameters does not mean that one of the parameters is that cause and the other the effect.
A correlation can be present due to some “not so obvious” factor affecting both the observed quantities.

This cleared a lot of my anxiety and frustration over lack of understanding of the admission process. I think it will surely help others.

mathboy98 · October 20, 2011, 12:42am

Is “matter” supposed to be “matters” ? I know, I know, who cares, but someone might like to know.

I do think people should read this over and over again.

iceui2 · October 20, 2011, 3:53am

^Great now everyone will be trying to be IMO medalists.

MITChris · October 20, 2011, 9:09am

mathboy - yep, fixed matter to matters.

mathboy98 · October 20, 2011, 10:48am

Better in my books than trying to change a 740 to an 800, which is essentially a waste of time as an intellectual endeavor. Unless someone really likes the SAT and has fun preparing for it.

Further, as an IMO medal is a much less attainable goal for the “masses”, I think it would be better since they’ll give up pretty soon at that, and try to focus on their actual strengths.

That said, nothing Chris wrote suggests the entirety of applicants should try to attain IMO medals - only that one is more impressive than an 800 on the SAT independent of all other factors.

I always wished the standardized test used most widely at many places were something other than an SAT, basically for the above reasons. I fully sympathize that a 740 vs. an 800 perhaps doesn’t really change anything about suggested performance in and of itself. I also think a 700+ on a fully multiple choice, time-pressured test that’s more about not making mistakes than conceptual reasoning doesn’t provide an adequate baseline indicator of preparation for a top school (which is why other indicators are of course looked at, which is great, but hardly means there isn’t a part of the system that can be significantly improved).

compact · October 24, 2011, 9:14am

something co-related
[xkcd:</a> Correlation](<a href=“http://xkcd.com/552/]xkcd:”>xkcd: Correlation)

mihcal1 · October 25, 2011, 11:02am

So basically, it’s a perfect setup for the [Illusion</a> of Validity](<a href=“Don’t Blink! The Hazards of Confidence - The New York Times”>Don’t Blink! The Hazards of Confidence - The New York Times)

Why is MIT’s admissions process better than random? Say you weeded out the un-qualified (the fewer-than-half of applicants insufficiently prepared to do the work at MIT) and then threw dice to stochastically select among the remaining candidates. Would this produce a lesser class?

MITChris · October 25, 2011, 6:15pm

You know, in reading the Kahneman article:

There are a few things which occur to me. One is that my belief above might be wrong.

Another is that “lifting a log over a wall” might be a less useful tool for evaluating leadership than our battery of letters, interviews, and so forth are at evaluating applicants; that this problem:

is somewhat mitigated by the fact that we have a tremendous amount of information at our disposal, not only which originated in our process but which originated from other people outside of our process like teachers, interviewers, and other external validators (think ISEF, Siemens, IMO, etc etc etc).

I’d also note that

Another is that it may be a mix of the truth; that for some percentage of the class, it may as well be random, but for a (large) subset of the class there are outliers who we need to capture in our process and cannot risk being left to random chance, like a [Ruben</a> Alonzo](<a href=“http://web.mit.edu/newsoffice/2010/truman-scholar.html]Ruben”>Junior Ruben Alonzo named Truman Scholar | MIT News | Massachusetts Institute of Technology) or a [Tessa</a> Green](<a href=“http://web.mit.edu/newsoffice/2011/class-of-2015-0907.html]Tessa”>Fresh faces on campus | MIT News | Massachusetts Institute of Technology); in other words, that there are certain students that we know we need to bring, and then for another subset it falls into this question of randomness and how much better than random we are.

I would also note that Kahneman is somewhat conflicted in his anecdote about stock pickers. There’s a lot of hedging about skilled vs unskilled, and also about “highly efficient markets.” And I’m not sure that the physics of a stock market are all that alike to the physics of “would I enjoy being a roommate with this person? Will this person contribute positively to a campus environment? Will they bring a perspective that is currently rare but valuable at MIT”, which are more the sorts of questions that we might ask.

I think we fall more on the anesthesiologist side of the continuum he creates (as opposed to radiologist). By which I mean: we live with our decisions. We meet them at CPW. We see them on campus. We advise them academically. We employ them in our offices. We go to their graduation. So that, I would hope, provides some of the feedback that Kahneman rightly characterizes as critical to improving decision making.

The last point I would make is that the proof is somewhat in the pudding, by which I mean this: ultimately, it’s not about how our process works. It’s about what our process produces. If you think that MIT has a community you want to be a part of, then our process is producing the right type of community for you. And if MIT doesn’t have that (due to our process), then MIT isn’t the right type of place for you. Now obviously that’s not a very reflective stance to take (what could we do to make our process “better” or “worse”), but it’s the most useful one for applicants, I think. Because ultimately MIT isn’t an ideal. It’s an actual place that you either like or you don’t. And that’s something that is probably a useful way to think about colleges in general: not as what you would like them to be, but what they are and have made themselves to be.

Still, that was a very powerful and important article. Thanks mihcal1. I’ll see if I can incorporate this discussion into a blog post.

MITChris · October 25, 2011, 6:20pm

On that last point I made - it’s an example of inverting the question. Why would a random process be “better” than our guesses? That is an epistemological problem: what is “better”? And how would we measure it?

Perhaps a more pertinent question is: does our process produce a class that we are proud of and that does well at MIT and in the world? And, if so, what could be “better” than that?

itsme123 · October 26, 2011, 6:47am

i think the problem is, when it comes to admissions, mit thinks collectively, and each student individually. i know personally between two students, in my opinion, mit admitted the inferior one, academically and otherwise. there will be a lot of anectodal cases like this, i bet. but mit does what it does.

perhaps mit should admit an “experimental” or control group of students each year (like college board questions, and track the students to see how they turn out). i bet they wil be just fine – do well in the world. things seem to even out in life, especially with time, and so many uncontrollables. of course, this is a preposterous suggestion, but i am not sure how else anyone can justify what is “better”? better implicitly says there is a control group.

Thread		Replies	Views
A few notes on the eve of EA decisions Massachusetts Institute of Technology	3	67	April 17, 2021
Congratulations Massachusetts Institute of Technology	2	35	April 16, 2021
Thanks Massachusetts Institute of Technology	22	69	April 17, 2021
Shoutout to MITChris Massachusetts Institute of Technology	10	42	April 17, 2021
It gets better Massachusetts Institute of Technology	4	32	April 17, 2021

i wrote some words on the internet. you may be interested in reading them.

POPULAR STATES

SEARCH SAT SCORES

SEARCH ACT SCORES

SEARCH GPA’S

i wrote some words on the internet. you may be interested in reading them.

Related topics

POPULAR STATES

SEARCH SAT SCORES

SEARCH ACT SCORES

SEARCH GPA’S

CONNECT WITH US