He was not commenting, as far as I could tell, about downtown Charlottesville or about the university. He was talking specifically about the well-heeled suburban/exurban areas.
It fact he specifically said that he was concentrating more on delivering to restaurants, which presumably are more likely to be in town.
I think that the university’s diversity stats are irrelevant to his point.