If you are interested in esoteric or highly-specific subjects, your best bet would either be a large research university where you could enroll in graduate-level courses, and have access to extensive resources, or a small, flexible college where you could design your own curriculum. Otherwise, most colleges have solid History departments. You might want to consider other ancillary subjects, especially if you think you’re interested in post-graduate work in the area. Those would include foreign languages (including classical and extinct ones), availability of primary sources, Anthropology and Archaeology classes, and opportunity for remote and foreign study.