Personally, what causes me to feel moral outrage is when I encounter this sort of derogatory statements, aimed at the poor. Based on something once overheard on a bus, nonetheless.
The fact that there is a tiny percentage abusing the help is a very poor excuse for not giving help to the much, much larger percentage of people who are deeply, genuinely in need of it.
As a continuation of your realization that your opinions are probably based on irrational emotions: why is this irrational anger aimed at the part of society that is least well off? For instance, I wonder what is the percentage of actually rich people who are payed with tax dollars to do nothing of value, or even something destructive.