This has been mentioned before, but the reason why Americans care more about fit than people in many other countries (not Japan, which also has privates, including LACs, as well as top publics, BTW) is because there is a massive variety of higher education institutions in this country, and that simply isn’t true in other countries. In China, every place worth going to is a public with a set curriculum and where you end up is determined by your gaokao scores. In Germany, you get the choice of one bare-bones public or some other bare-bones public. Geography and prestige may be worth considering, but what else would you consider?
In the US, say that you have UMich, Swarthmore, and Cooper Union as your choices. How would you even compare by prestige? 3 different people may order those 3 schools 3 different ways. And CS/engineering could be extremely rigorous at all 3. Yet being a student at Swarthmore would be an extremely different experience from being a student at UMich which would be an extremely different experience from being a student at Cooper Union. So why shouldn’t fit matter?