<p>
[quote]
See, what's often far more indicative is actually having a conversation with someone!
[/quote]
</p>
<p>Story!</p>
<p>I learned this in my organizational econ class. Once, a very long time ago, in an enchanted forest, there was a big longitudinal study of medical school admissions (I believe the medical school was in Texas). There were two cohorts, with applicants randomly assigned to one or the other. One was admitted with a review of the application and a day-long interview by three doctors, the other just with a review of a paper application. The cohorts were tracked and evaluated by (a) patients' average satisfaction with them 10 years down the line (b) their earnings 10 years down the line. Yes, these are "simple" measures, but they were basically the things that the med schools try to maximize when admitting students.</p>
<p>Now we think through what we might expect depending on what we think of interviews. If interviews are good, then we would expect to see the means of those measures of doctor quality to be higher for people admitted through the interview process.</p>
<p>If interviews are white noise -- neither informative nor actively harmful, we would expect to see the interviewed cohort have higher variance, maybe, but no significant difference in means.</p>
<p>If interviews actively harm your ability to pick good doctors, then the mean for the interviewed cohort should be significantly lower.</p>
<p>Guess what the study found? You're right, #3! :)</p>
<p>I've been digging around Google Scholar and it should be easy to find. Searching for "interviews medical school admissions correlation" turns up a lot of studies that find no effect or replicate the result, so at least one should cite the famous original.</p>
<p>The point is, though everyone does interviews, there's not really a shred of scientific evidence that interviews do any good, and quite a few shreds that suggest they make you stupid.</p>
<p>I find this result just as counterintuitive as you do, maybe more so. I can't tell you the number of times that, while reading an application, I've wanted to have the kid in front of me so I could poke him a few times and see if any life comes out. Like our Dear Leader, I have felt that perhaps I could look into his soul and decide what kind of person or at least student he fundamentally is.</p>
<p>Nope.</p>