There are recent examples of exceptional Harvard athletes: Jeremy Lin (NBA) and Gabby Thomas (NCAA Indoor champ and collegiate record holder in 200m T&F)
OP, idk if the Ivy school recruiting you is Harvard but if they are, they seem to have a decent men’s team making it to NCAA tourney:
https://www.gocrimson.com/sports/mten/2017-18/releases/20180503hfsyrn
My son’s team (Cornell) had two athletes compete at 2016 Olympic Games for T&F.
Ivy League may not be a P5 conference but they do attract good coaches that can take an athlete to the next level. It just depends on the sport. It also depends on what the student athlete wants out of their collegiate experience.