France does this; college admissions are done only on the basis of grades and your score on the BAC. A consequence of this if tried in the US is that, based on the data that the colleges themselves supply, the percentage of URM students would decrease significantly at many selective schools, and I doubt that US society at the moment would accept this.