It goes against the spirit of the thought experiment because it's an attack on that spirit.

Abstractly, Rawls says society shouldn't be racially/ethnically/socioeconomically/etc prejudiced. The vast majority of people agree with this obviously.

Concretely, he says this can be achieved by ignoring your racial/ethnic/socioeconomic/etc status.

The claim is that the more you ignore your own lived experiences, the less you can use the knowledge that it gives you.

Rawls also argued that people would maximize the minimum utility behind the veil of ignorance, as they wouldn't want to risk being a persecuted minority.

This didn't come up during the debate, although it should've since it's why "whatever rules society operates under need to work regardless of what your starting point in society is".