The long and short of all of this is we are giving person A an advantage over person B because person C did something bad to person D, and it just so happens persons B & C have some matching demographic factor.
Putting it that way, I think the racism / sexism of diversity is laid clear. I don't call it reverse racism because racism can go in any direction it's not a unidirectional graph pointed away from white people. An innocent person is catching the short end of the stick as a dark twisted way to right some wrong that this innocent person had nothing to do with, other than skin color. The sick joke of it is that the ameliorative steps to fix racism are also racist.
At the same time, both can be true and it's not the same in every situation.
It is possible for there to be situations where these types of programs are required and useful for lack of alternatives, and at the same time have unfairly negative effects on the out-group.
I don't know if anything can and will be done to remove the "you're tolerated, but we'd rather have literally anyone else" undertones.
But sometimes there are slippages and that’s when the jaw dropping moments start to roll. I seriously revised the view I had of my people skills when it comes to believing to know people and what they are capable of or not. I have been proven seriously wrong, I clearly wasn’t as good at judging people as I thought. It was for me double jaw dropping in a sense: the behaviour itself and the person exhibiting the behaviour, which I would have sworn up and down was unable of something like this.