<p>If you are 100% sure you are going into physics and research as you sound like you are--</p>
<p>Go to Caltech hands down.</p>
<p>Yale has a notorious humanities bent and its science program is only recently starting to recover (and that's moreso in the biological sciences). Caltech will dish out Yale in any math/science major you can think of and it's much more respected from a graduate school point of view... (caltech-physics vs Yale.. physics...?)</p>
<p>THen again, if you're not sure you wanna do physics Yale gives you more freedom. Plus imo having only 250~ ppl in each year kinda sucks.</p>