I have a son in Cornell ILR right now. If you have any questions please private message me.
I don’t know much about UNC, but will give my 2 cents on some of the issues commented above.
Weather : One of the benefits of global warming 
since my son has been there, they have had mild winters and less snow than in past years. He was there last summer too, and they had a drought, no rain in summer, no snow prior winter. When it does snow, it’s pretty but my son has not experienced a really bad winter for last 2 winters. They had tons of snow prior winters, so it’s bound to happen again.
Social : Drinking comment above, my son does not drink at all, and made some nice friends who do not drink. Frats are big, but most are not in frats and many do not drink. This is a reputation that is overblown. All colleges have drinking and all have some kids who don’t drink.
Ivy comment above : Cornell is one of the finest universities in the world and those comments about it’s standing among the ivies is ridiculous. Some of the top research in the world in many many fields is done at Cornell, more than at the smaller ivies. Cornell is heavily represented in my industry where I work, I have twice worked at firms run by Cornell alum. They truly have everything you can think of to study.
Upstate NY & Ithaca : You saw the campus. You either loved it or not. Most who love nature and a small college town would absolutely love it. But some prefer to be nearer to civilization or in your case nearer to home. Family is important. We live near NYC, and I have 2 kids in upstate NY. One is about 3 hours away in a larger city, and the other at Cornell. The one at Cornell definitely does not see his parents as often since Ithaca is so far off the beaten path. The other one is in a larger city with airport, Amtrak station and a shorter drive. He came home FOR THE DAY last weekend. My S at Cornell could never do that. This seems like a big factor to you, and only you can tell.
Academics : Cornell gives a very high workload, in excess of most colleges. In fact the Ivy comments above are related. As an employer I can say I know Cornell grads are hard workers and those who went to HYP are clearly smart, but nobody flunks out of those schools. Lots flunk out of Cornell. It is really really tough. If I hire a Cornell grad, I know they are both smart and a hard worker. ILR students sometimes say ILR stands for “I Like to Read”. My son found he could get good grades if he does the work in most ILR classes (there are some easy ones but not many). Outside ILR, if you took electives in let’s say Math or Science, you will find people working ridiculous hours and still getting bad grades. This is a really intense high pressure school academically in many departments/majors. There is time for fun, but you need to have above avg time management skills, and be OK with the curve and resulting GPA. Also ILR is focused on Labor/HR issues as you know, but they make it really easy to take electives in other colleges, so you can study other areas of business. And kids get recruited regardless of the college they graduated within Cornell.
Cost ; You were not 100% clear on this, but I’ll add that Ithaca is an unusually expensive place to live. There is not enough quality housing for all the students in the city. The housing is overpriced. There is building going on and this involves mostly nice new housing at very high prices. There are lots of dumps to rent but still overpriced. Upstate NY is not an expensive place to live in general, but Ithaca is an exception due to the need to live near campus. If you live further from campus, then you’ll not only have the expense of a car, but have to be OK with driving in snow if normal winter weather returns. Travel to/from Ithaca from NC will be expensive. Airport nearby is tiny, not useful. Look for airfare to/from Syracuse not Ithaca. There are free (for freshmen) buses around the city for day to day life, and cheap after freshman year. Most stay on campus soph year and then for Jr/Sr year you would likely deal with the expensive off campus housing. On campus housing is REALLY REALLY NICE compared to most colleges, but they do not have enough for all students.
Personally based on your comments about wanting to be close to home, that’s not a bad thing to be close to family. I am sure UNC is not an easy workload, college is generally much more work than HS, but Cornell more than most colleges. If you are a workaholic, love nature, want to experience a different part of the country, Cornell is great. Also ILR is a very unique curriculum, one that you would not find at almost any other college. Either that’s a huge benefit to you or it’s not. If you prefer warmer weather and family nearby, I personally don’t think you give up much turning down Cornell for UNC. Don’t let prestige guide your decisions. While there are certainly many good things about Cornell, sometimes I wonder if it was best for my son, he had some other good options that were less prestigious but may have been better for him (and much cheaper with scholarships).
Good luck, you probably can’t go wrong with either choice, but they are very different.