Yes, when a billionaire gives away billions of dollars he's not accountable to anybody else. Why the heck would he be? If he were, then he wouldn't have donated it in the first place but would have just "invested it" in a company he created and controlled.
There's no way to spin this where donating to a charity is more evil than what all the other .1%s do, hoard it.