So anyone who isn't familiar with it is an unsuccessful dolt? Maybe highly educated people are just more well versed in small talk than people with lower levels of education and skilled enough in conversing that they can draw out details from you without you realizing they're doing it.
I don't think knowledge of a particular set of colleges has anything to do with intelligence or success. It's based on experience, opportunity, and access to information. I've learned a lot of the names mentioned here by reading this site but they aren't relevant to me, so why would I know anything more about them than that?