The Narrative vs. the Numbers
Why research matters when choosing nonprofits
As donors, many of us are instantly drawn to stories and images — something to help us grasp the otherwise hard-to-fathom circumstances of those experiencing ill health, poverty, and injustice. But fixating on the narrative alone can result in serious blind spots. In this post, we discuss what stories are good for — and where they fall short.
Research shows that a vivid story about a single beneficiary can cause donors to donate more than if they had been presented with a statistic about, say, a thousand beneficiaries being helped. One thousand is clearly a lot more than one beneficiary. Pure rationality would dictate that donors should care precisely one thousand times more about alleviating the suffering of the bigger group. But because of what’s known as the “identifiable victim effect,” many of us are more compelled by one individual, along with some personal information like her age and photo, compared to a mass of nameless, faceless figures. For decades, nonprofits have understood this human tendency and used it to great effect (remember the Sarah McLachlan ASPCA commercials?). Tried and true, storytelling in fundraising and marketing is showing no signs of going away.
But stories fall short in one big way: They just can’t tell you the actual impact of your giving. We don’t know if the single narrative featured on websites and commercials is at all representative of the experience of beneficiaries at large. Chances are, the words and images you see were carefully handpicked because they represented the extremes: the most evocative or the most dramatic representations. Most likely, they were not randomly selected from a sample. Beneficiary stories can also be prone to interviewer bias and response bias. For instance, a beneficiary might respond falsely because she feels pressured to please the nonprofit staffer. Qualitative research, just like its quantitative cousin, requires thoughtfully designed protocols for interviews and focus groups and thorough pre-testing of questions. Otherwise, the resulting stories could present a lopsided view of what the nonprofit has achieved.
When it comes to impact, qualitative methods alone can rarely stand in for numbers. At the very least, we need to know where, on average, beneficiaries started out before the program and where they ended up. For instance, how many were sick before versus after, the number of bullying incidents in a school before versus after, and so on. Best, of course, if these numbers reflect the usual experience of beneficiaries — not the extremes. That means the nonprofit needs to have asked a large enough number of beneficiaries and avoided cherry-picking which ones get asked.
As donors, we also need a sense of whether the nonprofit is responsible for all, some or none of the observed change. In other words, what change did the nonprofit cause compared to the counterfactual change that would have occurred in its absence? In the best case scenario, the counterfactual is estimated using a smart research design with some kind of comparison group. Or, if you’re searching via ImpactMatters, we bake counterfactual considerations into all our nonprofit ratings.
The most powerful cocktail for impactful giving? Narrative and numbers combined. Take Village Enterprise, for instance, whose microenterprise program in Uganda landed it on our Top List of poverty-fighting nonprofits. Village Enterprise uses a mix of “Most Significant Change” (a story-based technique), periodic surveys and a randomized controlled trial to understand its impact and communicate it to donors.
From the age of cave drawings to today’s Moth story slams, storytelling is human nature. But while narratives can motivate us to give, only numbers ensure we give effectively. For a primer on crunching nonprofit numbers, check out our tips for researching nonprofits.