Among other options, you could look into liberal arts colleges such as NESCACs (e.g., Williams, Wesleyan, Middlebury, Hamilton), Swarthmore/Haverford/Bryn Mawr, Colgate, Wellesley and Pomona. Note that while all of these colleges would be excellent for physics, some may limit their astronomy programs to a minor.