Who We Rate

We rate nonprofits that directly deliver services to people. Examples include soup kitchens, free clinics, homeless shelters, tutoring and so on. These direct service delivery nonprofits seek to make an observable difference in the life of an identifiable person. As a result, we can measure the change in that person’s life and estimate what change would have occurred without the nonprofit (the “counterfactual”). This enables us to directly estimate the impact of the program.

Not all nonprofits operate in this way. Some nonprofits operate through advocacy or systems change. For example, a nonprofit may lobby for a new piece of legislation or conduct marketing campaigns to change people’s minds on an issue. These programs may be highly effective, but they are much more difficult to measure. The link between the nonprofit’s work and the final outcome is longer, and often there are alternate explanations for why that particular piece of legislation passed or those minds changed. We do not (yet) have a good method for consistently estimating impact of these advocacy or systems change programs, and so we do not issue ratings.

We also do not rate religious organizations, community associations such as the local Elk’s Club or institutions such as museums. Broadly, we refer to these as “donor use charities”, i.e., the donor “uses” the charity, as do others. These organizations serve important roles in the lives of their members. As membership-based organizations, people support them because they value the organization’s role in their lives. Decisions to donate are commonly driven by the donor’s personal experience at the charity, and thus impact estimates are not particularly useful in understanding that value. After eating a restaurant, one decides whether to go back based on whether they enjoyed their experience, not based on reading an old review by someone else.

Impact audits and guided impact reports

Prior to issuing ratings, we conducted two types of analysis: impact audits, in-depth analyses of cost-effectiveness and the quality of evidence supporting it, and guided impact reports, collaborations with nonprofits to estimate impact. We have converted this analysis to ratings.