I say go with your gut all else being equal. I know many happy students at other schools and I will be the first to say that Yale is not for everyone.
However, I must note the timing of Bull Dog days. This is the last week of classes. My D, who is usually very accommodating to visiting students was stressed about a paper she had due during the same time. Even though her visiting student was her friend from her high school, she admitted she had little interaction with them. Even having to perform for the students with her step team was stressful cause it cut into study time. I just think the students were preoccupied with being done and going home for the summer. I’m sorry they seemed do distant. You’d see a totally different picture move in day in August. Look on YouTube and you’ll get a better sense.