However, use the Consortium. There are interesting classes and interesting students there’s and if you’re unhappy with how small Amherst college is, that’s your chance to expand, meet new people, and take classes in subjects Amherst may not have. I assume you also got into Amherst in order to take advantage of what it had to offer and the Consortium is definitely part of it. Taking a class at a women’s college or at UMass would be a worthwhile experience in itself. Take the Consortium class as a fifth class if you worry it’ll be too easy but I don’t see the harm in stepping out of your current comfort zone like what you did for math.
As for math, what you need to be able to take is statistics. See if there’s a gentle introduction at Amherst and plan to take it next semester.
I’m not sure you can easily transfer to a top 25 university, since, beside stellar grades, you have to prove Amherst doesn’t have what you need - and between the high quality, variety of majors, resources, and access to the Consortium, it’s hard to find something that’s missing. And obviously you could transfer to your state flagship if all you wanted was a large university with sports (not to mention there’s a large university in the Consortium.)