I pointed out major flaws in the Vanderbilt study here:
In fact, if you actually look at the numbers instead of just read the abstract, you’ll realize that the numbers don’t actually make the point that the author purports to make (at least as strongly).
That’s why people should actually examine the numbers rather than just read a summary.
Here are some findings:
"Another thing is that if you examine the data, the good-private-undergrad advantage really only applies to women. Men who went to a pretty good state school undergrad and then a good private for grad school seem to do about as well as their private-private peers. However, the employment data for women is affected to a large extent by the large numbers that deliberately choose underemployment or dropping out of the workforce to devote more time to family. I wonder if there are cultural forces at work. Are the women who went to state school and then a good private for grad school on average more “traditional” in that respect than the women who went private-private? "
"Among men who go to TierI grad schools, there’s virtually no difference between public and private undergrad (comparing Tiers I and III).
Among women, there’s a big disparity both among those who go to TierI and those who go to TierIII grad schools."
"In any case, I know what the study tries to address, but the data doesn’t support its conclusions that well. You keep ignoring that it’s not just men with MBAs but also MDs where male TierIII undergrads with TierI graduate degrees do better than male TierI undergrads with TierI graduate degrees. In other words, 2 of the 4 types of graduate degrees. In other words, 50%.
And the tiering being done badly does affect the rigor. If you have 30 schools each in 4 tiers of quality, but then put 20 T1 + 5 T2 + 5 T3 in group 1, 5 T1 + 15 T2 + 5 T3 + 5 T4 in group 2, 5 T1 + 5 T2 + 15 T3 + 5 T4 in group 3, and 5 T2 + 5 T3 + 20 T4 in group 4, what would the results of the 4 groups tell you? By using averages, you might draw the mistaken conclusion that a T3 school in group 1 does better than a T2 school in group 4 even though that specific T2 school actually does better than that specific T3 school. That seems to be what you have done with the Albany example.
Also, if you look at the numbers, the family aspect doesn’t change the overall results (in the specific MBA/MD/JD/PhD groupings) that much. It’s almost a non-factor."
"There are only 4 types listed. In 2 of them, TierIII men who attend TierI grad schools do better than TierI men who attend TierI grad schools. In the other 2, they are reversed.
Also, you could hypothesize about why, or you can just use common sense and realize that the terribly flawed tiering invalidates a lot of conclusions. For instance, Temple and NMSU are TierIII, but are you seriously going to tell me that they’re superior to Albany?
Even concerning TierIV, there are categories where they do better. For example among female MDs who go to a TierI med school, TierIV undergrads do better than anyone else. "
"Appendix Table 8 is illuminating.
As for rankings, I like to use the Forbes alumni results subrankings “American Leaders”, PhDs, and Student Awards as well as a WSJ ranking on elite professional school percentage rather than USN."
"I agree. The use of some old obscure Carnegie classification that few people use instead of just Barron’s undergrad selectivity levels or USNews grad school ranks (her way would lower the average level of the public RUs and LACs vs. the private RUs). The division in to public and private (which neither Carnegie nor Barrons nor USNews does).
Labelling the private RUs “TierI”, the LACs “TierII”, and the public RUs “TierIII” (when Barron’s has all 3 types in all 3 tiers).
That’s what makes me suspect that the author has an agenda. Because the straightforward way of tiering would have been to use simply one system (like Barron’s or USN) for undergrad and one system (like USN or ARWU) for grad. "
"Also, the poor way she tiers may mislead people. Obviously, no one would think that Syracuse and SLU are better schools than Dartmouth even though she has Syracuse & SLU as TierI and Dartmouth as TierIV, but some people who aren’t as good at understanding statistics may get the mistaken idea that Syracuse and SLU are better schools than SUNY Binghampton and SUNY Geneseo even though Barron’s actually has both those SUNYs on a higher tier than both Syracuse* and SLU.
*Obviously, Newhouse should be respected, but otherwise, there’s little to suggest that other schools in SU are superior to those 2 SUNYs. "
"Except that this study screws things up with the way it tiers. For instances, it has CWRU and Harvard in the same tier, but an MBA from HBS is not the same as an MBA from Case. So if Weatherhead has a greater proportion of undergrads from state schools than HBS, then it would appear that state school grads who get an MBA from a TierI do worse than better private research uni grads who get an MBA from a TierI . . . but it’s not comparing the same thing! "
“What makes me think that an ulterior motive is at work is that neither the pure Carnegie or Barrons tiering were used as is. In fact, neither of them divide between private and public research universities as the author does! In fact, even though the author says she uses Carnegie and Barrons tiers, she actually makes up her own system(!) For instance, neither Carnegie or Barrons have UVa and WashU in different tiers, yet this author has them in different tiers.”
"BTW, if you look at page 7, you’ll notice that the Barron’s tiers are pure obfuscation. She’s tiering solely using Carnegie and by private/public . . . . which Carnegie doesn’t do (there are 7 public Carnegie I LACs which she excludes from TierII and relegates to TierIV). The whole way she tiers reeks of an agenda. "