I agree with @ameridad that there are quite a few BS that have great programs for the very high-achieving math kid. Exeter is but one!
I also wonder, though, if BS is really a great option for you. Most people pick it for the whole package – community, ability to pursue different interests in one place, quality academics, etc. If you are merely trying to optimize in one subject where you already have pretty good resources, and you’re ho-hum on the rest of it, it may not be a great choice.
If you are still game after the last paragraph, I would suggest you take a look at George. Very strong math (both classroom and competitive) and Quaker, so emphasis on developing and supporting a spiritual life.