If you're serious about this and this isn't some "change my mind" meme, I'd recommend reading the culture documents of larger companies like Amazon. It's an attempt at standardizing how soft skills are evaluated and hired for. Basically, all interviews are an attempt to discern three things:

1. Can they do the job 2. Will they like the job 3. Will I like working with this person

The third is really where "culture fit" comes in. When you have multiple candidates that qualify for a role, how do you choose between them? You will naturally pick the person you hit it off with the most, which will more than likely be someone like yourself. It is definitely unconscious bias, but I'm not convinced it's unethical discrimination. This is the basis of all hiring ever...not just tech.

People are going to prefer who they choose to prefer. You're not going to build a better team or get better hires by consciously picking people you or the team doesn't want to work with...at least in collaborative fields. To call that unethical is disingenuous.

The way to minimize the influence of "culture fit" is doing more upfront thinking in the "can they do the job" category so you aren't just choosing between two people who both know javascript. Usually the only soft skill I see on a job description is "effective communicator," which is fluff. Think seriously about what it means to contribute to a team, how they contribute, and how you want teams to reach consensus and work together.