With a total enrollment of almost 30,000 students, I wouldn’t call UNC “smallish”.
Ann Arbor’s significantly higher violent crime rate, close proximity to Detroit, and long harsh winters are certainly valid concerns for many prospective students.
Having lived in both Michigan and NC, UNC would be my choice.