The vague nature of the evaluation, such as looking for “many other qualities” makes it awkward to confirm whether that criteria is present. The essay is the first component of the application listed in evaluating personal rating, so it is likely a key one. Not considering essays in the lawsuit evaluations further complicates matters.
The lawsuit docs suggest Asian applicants averaged a ~0.13 lower rating than White applicants on the personal rating’s standard 1-4 scale – a relatively small difference, but more than might be expected if just looking at the evaluation of the available material, such as LORs and alumni, with essay unknown. There was significant variation from year-to-year, with Asian applicants averaging a higher rating than White applicants some years, and the reverse other years.
When trying to look at how much race influenced the personal rating after applying controls for the other available sections of the application, experts from both sides found the vast majority variance in ratings was unexplained by a combination of the available application materials, race, and many other characteristics. What strikes me as suspicious, is among the portion of variance that they could explain in personal ratings, race appeared to be a more statistically significant as an influential factor than the other ratings – less likely to be unintentional noise beyond the model. For example, the largest racial regression coefficient for each part of the ratings in Arcidiacono’s expanded model is below after all controls. In short, this is how race appears to influence the rating among applicants with similar stats, majors, gender, ratings in other sections, etc.
Academic: Asian Most Influential: +0.128 (0.031)
ECs: Black Most Influential: -0.283 (0.044)
LOR1: Black Most Influential: -0.103 (0.048)
LOR2: Asian Most Influential: -0.144 (0.028)
Counselor: Hispanic Most Influential: -0.048 ((0.041)
Personal: Black Most Influential: +0.691 (0.053)
Alumni Personal: Black Most Influential: +0.204 (0.041)
Alumni Overall: Missing Race Most Influential: +0.167 (0.039)
It’s possible that the apparent increased contribution of race to the personal rating may relate to a greater degree of uncontrolled variables that are correlated with race. For example, Black applicants may be more likely to have have the overcome “family hardship” part of the personal rating criteria than other races in such a way that is not captured by the SES type controls. If true, then this missing “overcame hardship” would be expected to make it appear that Black applicants have an advantage in personal rating beyond the available controls to some degree. It’s hard to say how things like " the applicant’s humor, sensitivity, grit, leadership, integrity, helpfulness, courage, kindness and many other qualities" would be correlated with race. So in my opinion, it just looks suspicious – not conclusive, and also not easy to distinguish between unintentional racial biases and AA, as the Vox article touches on.