Do colleges take offense if you don't answer the phone?

Not really - e.g. you could be busy with something (running errands, interview, cooking dinner, driving), and they shouldn’t assume you will always be available. About a month ago, I missed a call from a prospective internship since I was working intently on a coding project. They should either email you or leave you a message, but if neither occurs, call them back (in my case they emailed me).