Michigan has the edge academically, but USC is good too.
In terms of the weather, you’ll learn to stay warm in the winter. Snowball fights are fun and if you’ve never gone skiing (cross-country or downhill) or snowmobiling, both are fun. Ice fishing can be fun too, though mostly you just sit in a tent and do what you like to pass the time. You check your line every 20 or so minutes and go back to the tent. Anyway, fall and spring are wonderful (generally) in the Midwest, and summers are ok. Fall foliage is lovely. There are four seasons versus just one in southern CA.
In terms of fun, Michigan will offer plenty of social opportunities. The advantage USC has is the city of LA and all of its cultural/food/entertainment opportunities. Detroit is fairly close to Ann Arbor and has those “big city” attractions too, but USC is in Los Angeles.
You were right when you said that academic fit is the most important fit area. Second is probably cost.