Did you apply and/or get into SB University Scholars? Did you apply to Honors College (which selects the 60 best students in the freshman class for a top-notch experience)?
The honors college DOES make a big difference to the quality of education (especially wrt research opportunities, relationship with professors/mentors, REU advising then fellowship advising, funds to present your research at conferences…)
Based on what you said, I’d imagine SB Honors would be a great solution for you.
http://www.stonybrook.edu/ugadmissions/newhonors/
https://uaa.stonybrook.edu/honors-college
If you don’t make it into University Scholars or Honors College, there’s WISE - always a great resource if you plan to go on to grad school.
http://www.stonybrook.edu/ugadmissions/newhonors/wise.shtml
Also, if you went to UMass, would you not be able to double major in physics and astronomy?