“For that matter, it’s possible that some of the Ivies would be more like Colgate or JHU now if it wasn’t for sports (note that the Ivy League is a sports league, which many people seem to forget).”
In their own way, the Ivies and the NESCACs are huge sports schools. While the academic elites are not into big money spectator sports, they devote TONs of resources to their sports programs. The biggest commitment they make is how many of their scarce seats they allocate to athletes in order to field their expansive teams.
Bama has 617 varsity athletes and an enrollment of 28,447. Dartmouth has 903/4,214. Williams has 747/2,070. Very different schools and athletic departments, obviously. But you can argue that Williams is more drunk on sports than Bama is.
Sports+college is pervasive in the U.S. The biggest and most valuable brand in higher ed is The Ivy League. Everyone in the world knows about the Ivies which, as Purple points out, is just an athletic conference.