If the school has a graduate program, especially a PhD program, then pay attention to the research being done there; if you’re interested in experimental particle physics, by all means apply to Yale, if you’re interested in condensed matter, try UIUC instead.