A bit more info here - it moves a small amount vertically only if both the upvote and downvote arrows are visible. Collapsing older comments where only the upvote arrow is visible doesn't have the same effect.