Are you a first year student? I’ll give you my perspective as someone who transferred in undergrad. You have two competing interests. You need to prioritize your financial health and your future grad school needs. You also want to have a more traditional college experience. I’m somewhat less practical than others and would say that looking for that more traditional experience is important too. I transferred as a third year student from a small school with a full ride to a state flagship where I was instate but didn’t have as much financial aid.
My recommendation is to give yourself one more year. You can spend less money next year in your current situation. UM Ann Arbor will still be there. You can apply as a transfer while you are a sophomore and spend your last two years of college there. In the meantime, try to settle into your current situation knowing you will be there for another year and a half and try to make the most of it. I ended up enjoying my sophomore year at my first school much more. I did eventually find a nice group of people and was kind of sad to leave them.
Even though I was only at my second school for my last two years, I took full advantage of clubs and football games and all that fun stuff. I also only had upper division classes left so I had mostly small classes and got to know my professors well. I ended up finding my people fairly easily even though I was a transfer and I am not a super outgoing person. If you can make the finances work, it is worth a try, but it will be easier to make two years of finances work rather than three.