▲ 78 points
back
81 comments
"But it does mean that if you’re involved in nonprofits (or predictions), you need to be much more careful about making sure you’re doing a good job. Unfortunately, few nonprofits do that."
Oh really? So he can cite research that says, for example, that only x% (where x < 50%) of nonprofits with an annual budget of $500k have ever done any sort of evaluation, either comprehensive or more basic?
His assertion may be true. But just throwing this out there with absolutely nothing to back it up, in a blog post titled "The Trouble With Nonprofits", is frankly insulting to me.
I spent > 4 years working in a technical role for a nonprofit, and I can assure people that measurement and evaluation were very much on the minds of staffers. Everyone (funders, management) was becoming more data-driven, so our organization got a grant to conduct a multi-year evaluation. It was a very positive and necessary process, and the results were very positive. Like Aaron, I definitely recommend some sort of measurable goals for most every nonprofit out there.
I don't take issue with his opinion. What I take issue with is the tone of the article.
For example: "...but an actual attempt to measure how much they’re improving people’s lives. For most nonprofits, I expect these numbers will be depressingly small. "
It can be easy, as young paragons of technology and entrepreneurship, to look over to the nonprofit world and disparage it, to think of it and it's practitioners as "lesser than". I should know; it's the attitude I had before I started working at a nonprofit.
Truth is, running a nonprofit can be a hell of a lot harder than running a for-profit. At any stage, you have all the issues that for-profits have - hiring, firing, managing employees, management issues, getting money in the door, technological problems - PLUS a whole other layer.
A lot of this other layer comes because of the weak link between financial health and results. Am I accomplishing my mission? How do I measure success? How do I maximize success based on dollar spent? How do I judge where to allocate capital or resources - when maximizing revenue isn't a goal?
Why am I wasting my time writing this? Because this article uses a lot of conjecture and one very poor example to make the entire nonprofit sector seem like a bunch of bumbling idiots who don't know how to use spreadsheets. They're not, and it's not fair for this community to see such a distorted view.
I've worked at two social ventures that had non-profits as clients (Web Networks, and most recently Urbantastic.com).
There emphatically is something wrong with non-profits. Just because there are some excellent ones (Sturgeon's law applies) doesn't mean that the field at large isn't ill.
An anecdote: we had a meeting with a guy who worked at a large company with a philanthropic streak. They provided, free of charge, a "dashboard" for planning, along with free training and tech support to any non-profit that wanted it (it cost considerable money otherwise). It was simple but very powerful, allowing you to do sensitivity analyses on projections to determine which courses of action would result in the best returns - both monetary and any other metric you wanted to put in (i.e. the social goals of the project).
It was like he was selling popsicles at an ice rink.
We ran into a very similar issue with Urbantastic. We had a web service that was very effective at getting small things done for non-profits: it's called micro-volunteering. There's no supervision required, you just write down what you want and someone always did it.
When we founded Urbantastic we thought the challenge was going to be in getting skilled people to help out for free. That was not a problem. The problem was that we'd go up to non-profits and say: here's a machine where all you have to do is post your todo list, and people will cross off items for you. For free. And we never got uptake.
People lined out the door offering to help, and almost no one took them up on it. Despite our offers to walk non-profits through it, and despite the recent funding cuts which meant that they needed all the help they could get.
I'm not saying that Urbantastic is perfect, but in our pilot cities this was overwhelmingly the result, and I still can't think of any incremental change that we could have done to get the non-profits engaged. A big part of it was that most of them were still uncertain about the whole email thing. It's not an exaggeration to say that some of them didn't know how to use a spreadsheet.
With our fancy web 2.0 ajaxy website, we felt like we were giving out iTunes gift cards to the lost tribes of Papua New Guinea. They didn't value it because they weren't even close to being at the point where they could use it.
I care about social good - I wouldn't have founded a social venture if not. And twice now I've been floored by the state of the non-profit world. Again, 10% of them are brilliant, but the other 90%... It's a big problem, and it's not going to get better by pretending that it's not there. That world is amazingly backwards. And it's saddening to see such important mandates being handled by such ineffective organizations.
My advice to anyone here planning on doing a social venture: drop backwards compatibility with the non-profit world. (one giant caveat: working exclusively with one or two of the few clued-in orgs might work).
A lot of what you are saying here resonates with me. I work part-time for a local charity in my town and have been involved in various others at times. There are a number of problems charities normally face which may explain your findings to some extent, including:
- well meaning but incompetent staff. People don't seem to understand that wanting to do good doesn't naturally translate into the ability to do it.
- no real management. I suppose this may be local phenomenon, but where I am (UK), there seems to be a tendency for small charities to be managed by wealthy housewives as a kind of hobby job. There is very definitely a social network of them that dominate the charities in my town. Often this means the management will be consistently absent from the organization. You can also see my previous point.
- no real check on the management. Charity trustees are often people who just want their name to be associated with a charitable organization. I can think of several trustees of the charity I work for who probably have only a vague idea of what we actually do.
- funding constraints. Charities in the UK who receive funding from government departments are usually monitored in terms of the volume of work they do rather than the outcomes they produce (which are much harder to quantify). Thus the incentive is often to not try to improve efficiency as reducing the work load may negatively impact funding.
Obviously my examples are to some extent just personal experiences that may not be universal. However I would guess that most non-profits face similar problems and that these problems may help explain why you got such a negative reception with your projects.
With regards to volunteers though, what did you to do assess their competency to perform the tasks they were willing to help out with? I ask because part of my job is to manage my organization's volunteers (which mostly come to us on work experience placements from the job centre). They frequently start working for us claiming to have various skills and then we discover that they often don't. For this reason I am usually quite unhappy about accepting new volunteers - when I have to get rid of them is is much harder (emotionally, for me) than sacking an incompetent employee. This might also help explain some of the resistance you encountered to what you were doing.
A big part of it was that most of them were still uncertain about the whole email thing. It's not an exaggeration to say that some of them didn't know how to use a spreadsheet.
They have an irrational fear of technology, and it may be because they believe nonprofits should be modeled after the 60s and 70s counterculture-based nonprofits where technology was distrusted as being a tool of the Man (a lot of money was poured into computers via AI funding, defense grants, etc).
It is awful and sad that they would not use Urbantastic because it sounds like a great idea.
I'm the "tech guy" (ie, code monkey) at Givology, and boy could we use some of your givers. We don't have a fancy website (since it's basically composed of my spare time), but the concept is to allocate money to specific students or projects (similar to Kiva and Global Giving). This is the first I've heard of Urbantastic, and it sounds like you've got the opposite problem we have; we have a lot of students and projects that we can put up on the site, but not a lot of donors.
What's the scoop with Urbantastic? Are you still working there? Drop me an e-mail if you could. Have questions.
I'd like to hear you name the nonprofit and some of the metrics they use; it'd be great to be able to share good examples.
I'd rather not post the name publicly, but I'd be happy to talk more with you offline if you're interested. oscillations at gmail.
> This isn't to say that we should have companies replace nonprofits
Yes we should.
Charities I donate to are selling me a product just as much as companies I buy from. That the product is a better world--by some measure that seems reasonable to me, rather than a simple benefit for myself, matters not a whit to my decision making process.
I don't give without a very concrete understanding of what the money will be used for, any more than I buy stock without a concrete understanding of what I'm buying and for how much.
By all means, sell me peace, charity, and goodwill! Perhaps that will motivate you to learn how to produce it, and to convince me that you have done so.
It's worth noting that "non-profit" doesn't always translate to charity. Microfinance, for instance, exists as a whole spectrum of quasi-for-profit operations. And on the strictly non-profit side you have things like ROSCAs, one of the foundations of microfinance, which are micro-level, semi-formal, non-charitable "non-profit" groups that resolve a number of different financial issues for participants, similar to tiny credit unions, and are neither charitable nor reasonably mappable to "companies."
The whole impetus behind the growth of the microfinance movement is that traditional banking simply can't serve these groups. There are also numerous non-obvious major core issues involved, such as how gender roles have a huge impact on microfinance or why group lending is so prominent.
In short, the reason it's not simply handled by companies is because it's just not that simple.
Oh, indeed, I'm well aware that there are for-profit philanthrophic ventures and questionably profitable non-profit groups. I don't mean to suggest that it is simple. Indeed, groups that adopt a hybrid, hard-to-classify approach often strike me as the healthiest. I am very fond of certain microloan groups or folks like TechnoServe.
But I do advocate approaching philanthropy as a consumer, however the group you're working with is legally classified. Be honest with yourself about what you're buying -- even if what you're buying is a warm fuzzy feeling. Perhaps especially in that case.
For-profit companies selling you goods and services often spend enormous amounts on marketing, advertising, and branding in order to convince you to part with your hard earned money. This doesn't mean their product is any better than their competitors though. It just means they've spun you into believing it is.
While I think it's a good idea to research your charity or donation to a non-profit, I definitely do not think they should operate like a for-profit business.
I think this is a great article, and there are some really pointed areas of improvement for non-profits. However, let's not turn to the free market gods for a solution please.
I think it really depends on the specific area/charitable cause - but where it comes to economic development, markets are a far better solution than charities and aid which have often done far more harm than good (e.g. food aid). In general, the potential for inefficiency and even fraud is arguably even higher given the lack of transparency and often weak governance).
Anyone who donates money to a charity should at least spend as much time researching what they're "selling" as they would a comparable product/service. The tax returns are available but some of these firms spend as much as 80% of their funds raised in administration and overhead - which is often far worse than most for profits. Yes, not for profits also "often spend enormous amounts on marketing, advertising, and branding in order to convince you to part with your hard earned money".
Fortunately at least anyone can look up any charity's IRS Form 990 that lists such things as salaries (which are sometimes as much if not higher than private market salaries). I usually start here: http://www.guidestar.org/ (using bugmenot for login/password)
What does this even mean? Right now nonprofits do things in exchange for money from wealthy people. How would "running them like a business" be any different?
can i have 17 kilos of better world please?
As a board member of two very active nonprofit organizations, and former board member of another, I really appreciate the suggestion to devote attention to demonstrable results. I'll bring that up at our next board meeting for each organization. It's easy to track membership, and reasonably routine to track member satisfaction in membership organizations, and some organizations set policy goals (within the scope of the IRS rules about political activity by some kinds of nonprofits) that are verifiable. But, yes, organizations doing work far from most supporters (e.g., foreign relief organizations or missionary organizations) often can gain funds for years without having to show results. It's regrettable when a willing donor isn't giving money to best effect.
"But, yes, organizations doing work far from most supporters (e.g., foreign relief organizations or missionary organizations) often can gain funds for years without having to show results. It's regrettable when a willing donor isn't giving money to best effect."
Not to be a shill for my own blog again, but the reason so few organizations run real evaluations is that it's really, really hard, expensive, and time-consuming to do. Consequently, funders by and large don't offer real money or incentives for it, and if they don't, there's not much of an impetus to improve, and I describe this basic idea in much greater detail here: http://blog.seliger.com/2008/04/24/studying-programs-is-hard...
Furthermore, some funders don't want real evaluations, even if they say otherwise; for example, we wrote a bunch of Community-Based Abstinence Education proposals for various clients, where the main purpose of the evaluation was to try and demonstrate that abstinence-only education works, and I described the problems with that in this post: http://blog.seliger.com/2008/10/12/what-to-do-when-research-...
I take his essay to mean that, if you want results, measure results and hold them to it. Chess experts are experts because they are measured by their results. Political pundits are hit hit or miss because they are not held accountable for their results. Non-profits should be measured by their results, not their process, PR, or grants. Makes sense to me, but quantifiable results are often hard to get, witness the discussions here about how to to tell goo programmers from bad.
Having worked extensively with not for profits, the biggest problem is the lack of accountability - this has as much to do with governance as metrics. Managers often have a say in the appointment of board members whereas at least power ultimately rests in shareholders in for profits. Then there's the problem of changing missions - take the number of family foundations that would have their original benefactors appalled (e.g. Ford, Rockefeller, etc.)
And then another problem with metrics is that there has to be a cost benefit at some point. Obviously some are better than others, but if the cost to measure is as much to achieve, is it worth it? Though this may say as much about the mission as it does about the metric (microfinance is an area where this debate is raging).
How do you measure the "results" of a non-profit, when most of them work in non-quantifiable fields?
You end up with the quantity versus quality argument, favoring large, superficial efforts over narrowly tailored, substantive efforts.
Another important thing to remember when comparing non-profits and companies is the power of the customer. Companies go bankrupt because the customer has the power to discontinue their patronage if they deliver a product or service poorly (unless there are anticompetitive practices in play - ahem comcast). However, in the non-profit world the recipient of the service or products has very little power to motivate a market. Those who contribute money to non-profits could theoretically be this force, but does someone who wants to invest in reducing malaria really have to invest in understanding the health metrics and do this assessment with each investment? I don't think that is a permanent solution either. No doubt feedback and accountability are vital to making non-profts more successful, but let's challenge ourselves to think of a more systematic way to implement this type of change rather than summing with measurement is needed.
What you're really addressing is the issue of incentives, as economists call it, and so is the original author of the story. I wrote this comment on his blog:
To my mind, this isn't news, but that might be in part because I've worked in my family's grant writing firm for so long. The real issue you're discussing is one of incentives: nonprofits don't have much incentive to measure how well they're doing because funders don't demand that they do, or at least don't as a large group.
If the funder is federal, state, or local, the funder doesn't have the resources (or the brains) to really measure effectiveness; if the funder is a foundation, the funder probably cares more about the appearance of doing good than effectiveness. I've addressed this point in some detail here: http://blog.seliger.com/2008/01/04/more-on-charities and here: http://blog.seliger.com/2008/01/23/foundations-and-the-futur... . The blow, which I contribute to, deals with the issues you raise on a regular basis, and if you want to learn more about why things are the way they are at the moment, take a look at the archives.
Companies go bankrupt because the customer has the power to discontinue their patronage if they deliver a product or service poorly
This definitely happens to many kinds of non-profit!
in the non-profit world the recipient of the service or products has very little power to motivate a market.
For 501-c3 organizations that provide classes, lectures, and performances, often the opposite is the case. Many non-profits have to be run much like a business. There can be something like a marketplace where the "customer" pays not with currency, but with their time. (And sometimes, both money and time are involved.) Also, very often, the "recipient" of services can easily directly involve themselves with running the organization. (KPFT radio in Houston is an example of this.)
What if you had some kind of nfp evaluating nfp that would help a person who wants to donate decide what is effective?
This is why I don't donate. Charities usually don't publish exact reports what's the money is used on and what have they achieved.
Building schools doesn't help if you don't hire teachers and get kids attend school and money used on clever guilt marketing doesn't solve ecological problems.
"This isn’t to say that we should have companies replace non-profits..."
Actually, that is not necessarily a good thing to just sidestep. A lot of things that non profits work on might work better if they were worked on by companies. A lot of things that companies work on might have to be filled by non profits if companies weren't doing it.
Think of second hand stores. They are often run not-for-profit in order to provide cheap stuff to poor people. These compete directly with commercial low cost stores and other second hand stores. If these beat the non profits, there is no societal loss. One of the worlds most celebrated not for profits, the Grameen Bank provides small loans to poor people. It seems very possible that this NGO (and its many imitators) will be put out of business by commercial banks.
The UN has started a free online University to help the world's poor. I would take an either-or bet on for profit variants making a lot of progress on this issue commercially.
A great way of making non profits more like chess masters is making non profits into businesses. You don't always have to. But if it is feasible, its worht considering.
"Grameen Bank provides small loans to poor people. It seems very possible that this NGO (and its many imitators) will be put out of business by commercial banks."
Traditional, for-profit banking simply can't, which is the entire point. Even when microfinance institutions lean further toward for-profit, they end up more and more like traditional banks making traditional big loans.
And that's the way it actually moves, with microfinance institutions edging toward becoming more commercialized, and when they drift too far away from poorer clients, new micro institutions will likely come in to take their place. In other words, the shift naturally moves in the opposite direction.
One useful way of thinking about nonprofits: a nonprofit fills a need that the free market on its own cannot meet.
For example, the free market will not provide free meals to homeless people with HIV in Manhattan.
Nonprofits should not be made into businesses. If a business can fill a need and turn a profit at the same time, great (in most cases)!
The difference between a well-run non-profit and a well-run business: the non-profit puts 100% of profits back into its operations, while the business may re-invest the profit or distribute it to its owners.
Most states already require non-profits to operate like businesses; indeed, non-profits are expected to operate better than businesses -- most states require non-profits to maintain levels of capitalization that require devilish efficiency and reuse.
This is also interesting in how it relates to the study of airplane crashes and the strong, relatively immediate feedback that these provide. It seems that auto safety was much more slow to catch on, and required odd forces to bring the topic to the forefront.
This also has an interesting interaction with a small startup essentially searching for the important metric for its survival.
For profits have the benefit of having profit/positive cash flow as a guide that not for profits do not. These are probably much better metrics for small startups given that success in any stated altruistic endeavours are ultimately going to show up in these two metrics anyway (and must in order for a startup to be sustainable).
This is one of the reasons why non-profit companies are flawed. There are no incentives in place for monetary gain. Non-profits are good at raising awareness, not being a sustainable entity.
If someone wants to make a difference and change world (especially developing countries), I believe there has to be financial incentives : social entrepreneurship.
If you can't make money doing something, is there any good evidence that it is worth doing at all? Leaving aside goods with "free" competition from the gov't (schools, roads, libraries, etc). EDIT: Also leaving aside things you do for your more direct benefit, like growing your own food or contributing to open source projects you use.
"Making a bad prediction isn’t like that... it’s months or years before your prediction is proven wrong. And then, you make yourself feel better by coming up with some explanation for why you were wrong: ...And so you keep on making predictions in the same way — which means you never get good at it."
I think this also applies to many macro-economists
There is of coure a very important problem charities trying to do this kind of measurments of efficiency face. They are in a market for public perceptiom to get donations, and (if we take the numbers the article cites for Oxfam), can you imagine the loss of donations and goodwill after a few newspapers and TV programs run the headline "Charity X spends half their donations on statistics instead of helping people!"? Even if the costs could be amortized over a few years, the charity would be dead by then.
Perverse incentive factors, again. (And the news might be right, collecting in depth statistics about the help program might just be a pretext to give a lucrative consulting contract to the director's best friend).
I would argue that there is a second, equally-important, albeit interrelated, problem with non-profits in general: they are highly risk averse. Many non-profits still operate under a create an endowment and dispense funds to proven -- or at least perceptive -- good projects model. Consequently, there is a "if it ain't broke, don't fix it" mentality, which is to say weak experimentation.
I should mention that most of my non-profit experience deals with medical research foundations. Political think tanks are limited by the general impossibility of proving all but trivial problems.
Does anyone have a recommendation for a charity with demonstrable results in improving the lives of people in a measurable way?
Nearly any education reform organization that gets parents talking to other parents through channels not controlled by schools. Often parents can do a lot of self-help without public resources as soon as they can compare notes about what works to help children who need help. Setting up an email list and moderating it can be just about free, and often has great effect.
One example:
Another example:
http://www.mcgt.net/index.html
A third example, with more expensive programs besides the kind of program I just mentioned:
http://www.davidsongifted.org/
After edit: To relate what I say here to what was said in the submitted article, an organization that gets all of its members talking to one another freely through one-to-many, many-to-one, and one-to-one channels available to all members gets a LOT of feedback that other organizations don't get, and learns what works and what doesn't work.
http://givewell.net/ investigates charities to make sure they achieve demonstrable results, they have a list of those that do.
There are also meta-charities like GiveWell and the MIT Poverty Action Lab that investigate other charitable attempts.
Grameen Bank, http://www.grameen-info.org/, one of the largest microfinance ventures which you all have been discussing.
I'm not sure if this qualifies as measurable, but the ICRC does a lot of good in a lot of really bad places, and they produce testimonial and other documentation of what activities they can publicly document. (As you may or may not be aware, it's their policy to hold all reports of detainee treatment in strict confidence. Their site can explain why this is better than I can.)
I'm fond of Children International. They're very good about letting you know exactly and specifically what your donations achieve. I suppose any evidence of the effect on people is anecdotal, but in such a well-established relationship, it's also self-evident.
Homeboys Industries, in Los Angeles, which has been very successful in transforming ex-gangbangers and low-income youth into productive members of society.
Legal Aid (anywhere) assists asylum-seekers, wrongfully evicted tenants, and indigents needing legal services.
Microfinancing (non-US operations) in India, China, and the Southeast have done so well that the pioneer of microfinancing received a Nobel.
This is extensible well beyond non-profits. The general form is still rough in my mind but it is something like "avoid situations where accountability and feedback mechanisms are not essential parts of the environment"
A good book just came out about this very issue: http://www.amazon.com/gp/product/0470454679/ref=ox_ya_os_pro...
Of course, if one looks at the profit-seeking world, one sees multiple examples of organizations blind to feedback: GM springs to mind these days.
True, but responding to feedback is a lot harder than being aware of it. And especially so in a larger organization. I'm not so sure that GM's problems stemmed from not being aware of the problems as not being able to respond for various structural, legal, and social reasons.
Somewhat related, on charity: http://news.ycombinator.com/item?id=800663