It's not really that scary that you notice something that seems off to you, everybody has that. But how do you confirm your "gut feeling"? It's not like you hire the people you dismiss in that manner to see if you're proven wrong, so what makes you so sure?

Having worked on a lot of mainly (or entirely) remote teams, I think digital/chat interaction is just as good as face to face. Sometimes face to face personalities don't entirely match your chat persona but chat does a good job of passing along enough information to decide. I've worked with people for years while never having met them in-person. No need.

Regardless of remote/in-person, a piece of paper should never be the sole hiring decision.

I may be wrong but I think you missed the point. If "working together" is code for "we both have commit access to the same repo and the same functional test criteria and same target ship date" then it really doesn't matter at all if the person likes to work in their underwear and tell cat jokes since you're only interaction with them is going be through commits.

In a strangely wonderful way, people who have conditions that would make their 'productive' work in an office impossible have found work and even success as a remote team member on the other side of an email alias and irc channel.

That said, managers are more comfortable when the know their folks a bit more, but they generally are watching for things to go off the rails and head that off rather than whether or not the code is up to standard.