Prestige is a loaded term, especially when comparing LACs and research universities. Also, lay prestige is different than how grad schools and employers view a school. Plus, you are comparing top schools vs. top schools, so prestige should not be as important – all great schools. But I’ll take a subjective stab (not looking at any test scores):
Middlebury and Bowdoin – Brown, Cornell, Dartmouth, Wash. U, Vanderbilt, Georgetown, Emory, Carnegie Mellon, USC, Notre Dame, Johns Hopkins, Rice
Hamilton – similar to above, maybe a tick below in name recognition depending on where you live, but great outcomes for grad schools
Colgate – a tick below Hamilton, but great alumni network (this school seems to be less like the other three with D1 sports and more of a fraternity culture)