One thing to keep in mind is that if you hadn’t sacrificed and gotten perfect everythings, who knows if you would have gotten in to an Ivy/equivalent? Plenty of kids with 2300/3.9’s at Ivies/equivalents, but for others with those scores, schools like UMich/Tufts (which still offer a ton of opportunities, mind you) are their best options.
But yes, college admissions in the US is somewhat idiosyncratic and may seem capricious. So doing what you enjoy and having fun may still be the best path. It would lead to less regret and may even be better for you.