<p>
</p>
<p>Did you actually read the study itself? The IHE article is a science writing article. The actual journal article is at PLoS One, and is linked in the article. The method is explained more thoroughly, and there are statistical results there.</p>
<p>The researchers did use 9 fictional colleges/universities in the study. The word “reputation” was referring to the reputation of an institution as a soft/hard grader.</p>
<p>To test whether the candidate’s success in the admissions process was predicted by the grading norms of their alma maters, we gave participants a simplified admissions decision task with two main pieces of information on each of nine candidates: their GPAs and the distribution from which each GPA came…Participants knew that the nine* fictional** institutions from which the nine applicants came did not differ in quality or selectivity. Grading norms at the different institutions were therefore uninformative with respect to student quality.*</p>
<p>Also, the researchers did not tell the AOs the principles of the study because that was the whole point. They didn’t want to tell the AOs to take into account the grade distributions; they wanted to see if they would do it independently. They told the AOs to simply select the best candidates among all applicants, and then after the experiment, asked them how they made the selections. The officers <em>said</em> that their selections were partially based upon grade distributions, but the analysis belied that thought.</p>
<p>And no, it is not common to publish raw data.</p>