Your situation sounds somewhat similar to my son’s, who is applying to schools this year. We are from the midwest, and he is focusing on LACs, in the midwest, east coast, and west coast.
The “top” LACs in the midwest are Carleton and Grinnell. In the next tier are Oberlin and Macalester. Then comes St. Olaf, Kalamazoo, and a bunch in Ohio.
Swarthmore and Haverford are also top LACs, and there are a bunch more in Penn., in all the tiers.
Massachusetts also has a bunch, as does New York and Maine (well, three very good ones).
My suggestion: Find some ranking of LACs – the USNews one is fine. Don’t focus so much on the exact ranking of schools, but consider them more in tiers – e.g., top 10, top 30, top 75. Find a few from each group that look interesting, and research them further. (Some you’ll be able to eliminate right off based on considerations like location.) As you go through them, likely other considerations/preferences will become important – or maybe you already have them – and you can decide which ones to research further and/or include on your list. (You will likely need to include some other schools, like some more sure safeties – both in terms of chances of acceptance and affordability. Maybe public/state schools in your state – or nearby states, if they have reciprocity agreements.)
Also, you’ll need to get a handle on finances. For that, you’ll need to use the NPCs (Net Price Calculators) that each school has. That will give you an estimate of the financial aid (generally need-based only) you’ll get from schools, and you’ll have to determine what’s affordable. “Middle class” can be a fairly broad label, and also schools give different amounts of aid for different income/asset levels. These NPCs aren’t exact, but they should give a reasonable estimate, especially if your financial-related circumstances are not unusual.
You’ll also need to understand the difference between need-based aid and merit aid. The top schools generally only give need-based aid (and often “meet need” – but by their determination of “need”). Lower level schools give merit aid (and generally don’t “meet need”). You may determine that you are likely to get sufficient aid from top schools, or not.
Based on considerations like this, my son applied to a bunch of schools that we divided into four categories: reach, high match, match, and low match. We ended up uncertain whether we would get sufficient aid from top schools, so we included a number of schools where we expected/hoped to get good merit aid.
BTW, one consideration my son had was that he did not want a school experience that was too “intense” – in terms of the focus on studying so much. Schools I’d put in that category are: MIT, Swarthmore, UChicago, Reed, Caltech, and Harvey Mudd. You may or may not feel differently about that.
You’ve been discussing the particulars of math courses/departments. If that’s what you plan to major in, that’s a good thing to look at. My son is planning to major in physics, so we looked at the physics departments at schools. But it sounds like you’ve shifted around a little bit recently, and that may still happen further (nothing wrong with that, and it’s quite common), so you may want to consider other factors/offerings.