Also, why is there the idea that one should avoid living where there may be neighbors who are lower SES?
If one takes that view, then the only choices of places to live are the expensive places. If one only wants to live where others are of greater than or equal SES, then one may be at the margin of affordability, with nothing left for savings.