I would choose Wesleyan. 1) much more fun student body IMHO; 2) pleasant campus culture; 3) excellent school; 4) music department is really hard to beat. It’s amazing.
Wesleyan has a hip, fun, interesting, motivated student body on a campus where the culture is to be helpful to each other and the world. Intellectual while also being creative and arty. Top-notch student body.
Swarthmore has a hard-driving, intellectualism-for-the-sake-of-intellectualism culture, and I personally have never met anyone who genuinely enjoyed their time there. Prizes cleverness and for that prize will forsake human relationships. On this forum you will find Swatties – praising the school and the intense rigor of the academics – who say that it may “crush” the student.
Crushing the student, for me, is not necessary for an excellent education.