I took the opposite direction. I have a small chrome extension to remove all mentions of karma on HN.

I had the same issue on Reddit and with the Github streak counter. My brain stays focus on the number, even if I don't want to.