It depends on both the student and the school. Some students thrive as the big fish, others some get pushed to new heights when surrounding by amazing people… Some get bored and/or develop bad habits if they aren’t challenged well, others get stressed and/or drop majors if they are not at the top of their class. Some colleges offer special honors type programs or advanced/accelerated courses that that may allow more academically similar peers than would be suggested by GPA/SAT/rank… type stats across the full college. Specific majors/colleges can have a similar differences from the full college. There are too many variables to make a simple generalization.
This type of strategy can backfire because the grad schools you mentioned emphasize GPA, rather than being at the top of the class. Colleges with a larger portion of exceptional students tend to give a larger portion of top grades. For example, most classes at HYPSM type colleges have a median grade of A- or A. Yale gives special honors for being in the top 30% of the class. The current cut-off for top 30% at Yale is a 3.81 GPA. The distribution suggests a median grade near 3.7, which is similar to average GPA reported at other similarly selective, private colleges. However, not quite as selective privates often have average GPAs under 3.5. It’s common for not as selective publics to have average GPAs near 3.0 or lower.