At the end of one of my onsite interviews, while showing me out the door, the interviewer turned to me with a big smile and said "you really look like you fit in around here". I almost burst out laughing.
I guarantee he had no malice but I just found it funny for one tall, white dude with glasses to say that to another tall, white dude with glasses. And he was right, I did blend in pretty well.
You might argue that perhaps he wasn't referring to my appearance but rather my uhhh.... hand gestures while talking, but at some point you have to call a spade a spade.
I left without saying anything just because I feel awkward sometimes, but in retrosepct, I should have said something. He was so good-natured, I'm sure he would have genuinely appreciated me pointing out that certain people might find his words harmful.