According to style guides and grammar rules, sure, but if you don’t give your em-dashes room to breathe people might mistake them for a hyphen — true connoisseurs of fine punctuation require one space to announce their dignified entrance, and another to signal a triumphant exit.

Unless you're writing british english, german, or many other languages that use the — with spaces.