Would it hurt my application if I were to decline an interview offer from an alumni?
You should bite the bullet and go with the interview. Escaping real-life scenarios like this will not benefit you in any way, short or long term.
Most college interviewers are down-to-earth people who just want to get to know you and to inform you of their own experiences at their colleges.
I’ve seen on other similar threads that if you decline the interview, most interviewers are required to report back to the college that you declined the interview. I don’t know how much of an impact this will have on your overall application, but declining and interview probably won’t make a positive impact on an admissions officer.
I think you should do the interview. You’ll have to interview in the future for jobs anyways, and so practice can’t hurt. If possible, try to schedule interviews with other colleges you applied to and aren’t as interested in first. This way, you can have some practice with interviews. Also google commonly asked college interview questions and practice answering them with a parent or friend.
Good luck. 
^^ Exactly. While the actual interview may not have much of a bearing on the admissions decision, declining it will send up a red flag. Don’t stress over it, you will do fine. For most kids, this is their first interview of any kind so they know you might be nervous or lack polish. They just want to have a conversation with you and their job is also to do PR about the school.
Set up a time and go have fun with it. Good luck!
Thanks guys… think I’mma go for it.