It was pretty clear what you meant. The point is, in most cases, you really can’t know that that is the case. And even if it is true in a few cases, that doesn’t mean it happens often. Your implication is that it happens quite commonly – and that is far from true.
As @planner03 said:
Even more, think about what your life would’ve been like if you had lived these first 17-18 years as an African-American.