My son is considering Williams and we were wondering what the social life is like for kids (particularly for the guys) who are not on a varsity team. We’ve heard that they tend to dominate social hierarchy almost like fraternities. If my son will only be involved in intramural sports will he have a tough time socially?
Students at Williams find many ways to get to know each other, most notably through the amazing entry system, but also through their classes, extracurricular activities, etc. Keep in mind, Williams is a D3 school, so all athletes at Williams are academically qualified to attend and prioritize academic success more than athletic success.
The athleticism of the Williams student body has its upsides – it cultivates school spirit, and students looks forward to Mountain Day, intramural sports, and taking advantage of the Berkshires. Being on a team is nice because it provides a social group as soon as you step onto campus, but not being on one absolutely does NOT mean you’ll have a difficult time socially. The opposite is true: your son will, in all likelihood, have a wonderful time socially. Hope this helps.
Admittedly I attended some years ago and am female, but I was never aware of any social hierarchy or exclusivity at Williams. Everyone intermingled. Inclusivity starts with the freshman entry, which is designed to be like a microcosm of the college’s population. You interact and become friends with people from all walks of life.
For a male perspective-- My college boyfriend had a great time there, too. He rowed crew, but his closest friends were not on crew. They were people he met in his entry and in classes and through other friends.
Thank you so much for the replies - very helpful in helping my son make his decision.
Both my son and daughter attend and neither were athletes in HS. My son skis/snowboards and was recruited to play club rugby during Club Day because he approached the team table, telling them he wanted to see them play because he knew the rules – and they figured anyone who at least knew the rules could play! 
My daughter hikes. Both would tell you that while teams do form their own social groups, MANY students have nothing to do with either varsity or club sports and have a varied and substantial social life. Entries provide a good deal of the social scene freshman year and whatever extracurricular activities or dept. classes/events your student enjoys provide more. My daughter is active in the music dept. and a cappella, so much of her socializing is with her fellow musicians.