@otterma suggested St. Olaf, which I was thinking of as I read your post as a perfect place for music and a safety. I know two truly amazing kids that chose it last year. One is very musical, science-oriented, and turned down U Chicago for it.
I also thought of Carleton,& Haverford for your son.
If he loves Swarthmore, ED. Run your numbers but ED is your best bet.