I think it’s a misconception to think that the only way to improve is through competition with other talented people. Some people really strive hard and improve when they are the best, some people strive hard and improve independent of the kind of people they are surrounded by, and naturally some people are pushed forward by having competition.
For a lot of people, “no matter how well you do someone else will do it better” is really demotivating and it burns them out. If that’s the supposed advantage of being in a place like SV/NY then it’s clearly not for everyone - and clearly not for every talented person either.