That isn’t an explanation, it’s merely a description of what is happening.
Acceptance to “elite” colleges is determined by criteria that are heavily tilted towards the wealthy. The relationship between standardized testing and wealth has been well studied, as has the relationship between GPA and wealth. The type of extracurricular activities which provide the largest boosts for admissions to “elite” colleges also generally require financial resources, like athletics, academic competitions, clubs, etc. Of course, there is much heavier recruitment from higher income populations - high schools in low income communities do not have representatives of Ivy league colleges holding college visits, and legacies are almost all wealthy (at least the ones who are accepted), as are the children of faculty/staff, and finally, donors are all very wealthy.
I will repeat - “elite” colleges were established to educate the children of the wealthy, and have not “strayed” from that unofficial mission all that much. As such, they are not geared to provide that boost in social mobility for students from low income background.
On the other hand, the public colleges and universities were specifically set up to do that, and many, in their quest for “prestige”, have failed miserably in that mission. Nonetheless, others lead the way, as this list demonstrates. There are also private colleges which were established for that very same purpose, and they also can be seen on this list.