I think you might really enjoy Brandeis. Also check out Clark. Both in Massachusetts.
Similar size to the others you named.
As Brandeis notes, there you “don’t have to choose” between the vibrancy and opportunities of a Tier One research university and the intimacy of a small college with close student-prof relationships. You get both!
Brandeis has excellent and very accessible professors, two of whom recently won a Nobel prize.
Both have a mix of students, many of whom would check your “quirky” box.
Also, neither is fratty. Social milieu is friendly and welcoming. Kids were super friendly to my son at both when we visited.
Brandeis is not known for having a “pretty campus,” though— it has modern buildings, not Gothic or Georgian charm. The Clark campus is attractive enough. Neither competes with the beauty of a Vassar or Middlebury.