Kids who get passable grades in high school despite serious outward disadvantages are more likely to have the grit needed to succeed in college than kids who get somewhat better grades despite every advantage. And it is especially valuable to increase the odds that kids who have overcome obstacles many others also face will gain a broader influence in society by admitting them to elite colleges. But outward disadvantage is much more than a matter of race alone.
You might be interested in this Twitter discussion of affirmative action in that light.
But the big injustice in education is that we haven’t fixed our K-12 schools for those at the bottom of the heap. Here is a start: