My son is a freshman at Peddie. His two big loves are math and theater. He loves most of his classes and talks non-stop to me about his participation in theater. He is an academic nerd and a sci-fi/ anime geek, and he has plenty of friends who share his interests. He’s going to a musical on Broadway next week for free. He enjoyed coming home for spring break, but he was very excited to get back to school. I tried not to be insulted. 
Both schools have so much to offer. Go to revisit days. You’ll figure it out!
If non-athletic means wants to avoid sports as much as possible, look carefully at the requirements on that front even more than the environment. Being on a freshman team with kids who are hoping to make their way to varsity in the coming years is different from being on a team with a bunch of other people who are simply interested in fulfilling a requirement and maybe learning the basic skills or rules of the game. Some schools have options that are clearly the latter.
Most kids find their tribe regardless, and most accept that there will be requirements they don’t love, but playing sports with people who have expectations for you that do not match your own wants or abilities can be miserable on every front.