To build on Post #3, the task of ensuring that students enrolled in the classes randomly is complex. There are at least two additional factors that could skew the results:
- Savvy students (who would get good grades anyway) might have enrolled in one professor's class because of his/her good reputation, thereby "confirming" the professor's reputation.
- Foolish students might have enrolled in a class that met at a time of day when they were not fully alert -- like 8:00am or right after lunch.
As a professor myself, I see both of these factors playing a very large role in student outcomes.
Again, building on Post #3, population can make a big difference. At times, I have had classes with a large percentage of : football players, international students, dance majors, pharmacy majors . . . and the class becomes a very different experience.