Cataract Surgery

Overview

ImpactMatters generates estimates of impact — estimates that quantify the causal effect nonprofits have on social outcomes relative to cost. For example: a nonprofit that conducts free cataract surgeries cures one person of blindness for $100. Our estimates incorporate best principles in social science, described in our Impact Methodology.

This document describes our methodology for estimating the cost-effectiveness of nonprofit programs that conduct or fund surgery to remove cataracts. Cataract is the clouding of the lens of the eye, preventing clear vision. Cataract is treated by surgically removing the opaque lens and replacing it with an artificial intraocular (inside the eyeball) lens.1

In this document, we describe cataract surgery programs; the outcome by which we measure their impact; our methodology for estimating their cost-effectiveness; and a checklist of data required from nonprofits to calculate cost-effectiveness.


Description

An estimated 20 million people were blind due to cataract as of the World Health Organization’s (W.H.O.) last global assessment in 2010.2 Responsible for over half of all cases of blindness worldwide, cataracts are most often caused by aging, though some children are born with cataracts and some people develop cataracts as a complication of eye injuries, inflammation and other eye diseases. Surgery to remove cataracts is a quick and minimally invasive outpatient procedure with a high rate of success. However, many who need surgery in developing countries lack access. As a result, far more people become blind from cataract in developing countries than in developed countries. In Southeast Asia, for example, 42 percent of cases of blindness is caused by cataract, compared to 12.7 percent in North America.3

There are three common cataract surgical techniques: (1) manual small incision cataract surgery (M.S.I.C.S.); (2) extracapsular cataract extraction (E.C.C.E.); and (3) standard ultrasound phacoemulsification cataract surgery.4,5 In M.S.I.C.S., the surgeon uses instruments to fragment and remove the cataractous lens from the eye through a small incision (less than 6.5 mm, requiring no sutures) and then inserts a rigid artificial intraocular lens. In E.C.C.E., the cataractous lens is removed through a larger incision (12 mm, requiring sutures) and then also replaced by a rigid artificial intraocular lens. Phacoemulsification uses a high frequency ultrasound probe to fragment the lens and remove the fragments from the eye before a soft and foldable artificial intraocular lens is inserted through a small incision (less than 6.5 mm, requiring no sutures).

The three surgical techniques tend to result in differing rates of surgical efficacy.6 Providers of cataract surgery may choose a less efficacious technique due to cost constraints or contraindications (conditions or characteristics of a patient that would make a particular procedure inadvisable). Our analysis defines efficacy rate as the proportion of surgeries that achieve post-operative best corrected visual acuity of 6/18 or better.7 In other words, does the patient have good functional vision when wearing glasses of optimal prescription provided by medical staff at the time of checkup? Ideally, we would define efficacy as post-operative presenting visual acuity of 6/18 or better: Does the patient have good functional vision using whatever glasses she owns at the time of the checkup? Because the research literature on post-operative presenting visual acuity is thin, we defer to literature on post-operative best corrected visual acuity.8 In addition, we define “post-operative” as three months after surgery, as most rigorous studies measured visual acuity at the three-month mark.

We estimate the impact of nonprofits that run surgical outreach campaigns. In a typical surgical outreach campaign, a team of eye specialists visits a local health facility that lacks the expertise to perform cataract surgery. Eye specialists may be volunteers or nonprofit staff. Often, the health facility identifies people in the community who need cataract surgery before the team arrives. The health facility sometimes coordinates patients’ travel to receive surgeries. The nonprofit’s team performs surgeries at no cost over a short period of time, typically a week or two. See the Appendix for other programs related to cataract surgery, but which are not covered by this methodology.


Outcome

Number of people cured of blindness

We measure the success of cataract surgery programs as their cost to successfully curet one person of blindness due to cataract.


Methodology for estimating attributable outcomes

In algebra, we calculate the attributable outcomes of a cataract surgery program (O) as follows:

O = A * B * (1 - C) * (1 - D)
where:
A = Number of cataract surgeries performed by the nonprofit
B = Efficacy rate of cataract surgery
C = Counterfactual9 rate of vision recovery without surgery
D = Counterfactual cataract surgical coverage rate

We first record the number of cataract surgeries performed by a nonprofit during the analysis period (variable A), e.g. 5,554 surgeries performed by an India-based nonprofit in 2017.

If the nonprofit has specified the specific surgical technique used, we apply the surgical efficacy rate associated with that technique. In the nonprofit has not specified which technique it uses, we assume it could perform any one of the three major cataract surgical techniques:, with respective surgery efficacy rates of 99.26 percent, 98.66 percent and 86.74 percent.10 To estimate the overall surgery efficacy of the nonprofit’s cataract surgeries, we take the average of these three surgery efficacy rates, which is 94.89 percent (variable B).

To find the number of surgery patients that the nonprofit treated successfully, we apply the surgical efficacy rate to the number of surgeries performed (94.89% * 5,554 patients or variable A * variable B). We thus estimate that 5,270 of the 5,554 patients treated by the nonprofit achieved normal vision.

Next, we estimate how many of the 5,270 cases of blindness averted would have been averted even if the nonprofit had not — counter to fact — provided surgeries (the “counterfactual”). Subtracting counterfactual outcomes from observed outcomes is essential to calculating the number of patients successfully treated because of the nonprofit’s program, i.e., its contribution to reducing cataract blindness over and above existing services available to patients.

Of the 5,270 patients treated successfully, we assume 0 percent (variable C) would have recovered normal vision if their cataracts had gone untreated. This is based on the fact that cataract is a degenerative eye disease and no rigorous studies have found effective non-surgical programs that stop the progression of cataract.11

Then, we estimate the percent of patients who would have had access to surgery without the nonprofit. To do so, we research actual rates of cataract surgery among those who need it — the coverage rate. In India, approximately 58.8 percent of all severely visually impaired individuals with cataract in both eyes who were eligible for surgery actually receive surgery.12

We then net out the number of people in India who have access to essential health services. We do so because the average cataract patient served by the nonprofit likely does not resemble the average cataract patient in India as a whole. The nonprofit focuses on serving needier patients, perhaps of lower income or living in more remote areas than the average patient. Without information from the nonprofit about counterfactual access among needier patients, we adjust the 58.8 percent cataract surgical coverage rate based on estimates of health care coverage from the W.H.O. The W.H.O. publishes an index of universal health coverage.13 In India, the coverage index was 56 in 2015 (on a scale of 0 to 100). 14,15 We treat this as a proxy for the proportion of the population that already has access to essential health services when they need them, and assume that the nonprofit is not serving this population.

Therefore, from the 58.8 percent of cataract patients who actually receive surgery in India, we subtract the 56 percent who regularly have access to essential health services. The remaining 2.8 percent (variable D) are those who do not typically receive essential health services, but who have access to cataract surgeries (perhaps from other nonprofit providers in the region or paying themselves). Therefore, we estimate that the nonprofit reversed or prevented blindness for 5,122 cataract patients (5,270 * (1 - 2.8%) = 5,122). In other words, 5,122 patients would not have had successful cataract surgery if not for the nonprofit’s intervention.


Methodology for estimating cost

Below, we summarize the most important aspects of our methodology for estimating the costs of cataract surgery programs. For a detailed discussion of what sources of data we use, how we treat specific line items and accommodate variation in accounting practices, see Reference Manual on Data Analysis.

Costs we include

ImpactMatters estimates cost-effectiveness from the perspective of a socially minded donor. This means we count all important costs associated with a program regardless of who incurs them. Generally, the key cost-bearing parties are: the nonprofit itself; organizations with which it partners to run a program; the government (taxpayers); and the nonprofit’s beneficiaries.

Nonprofit costs

We include the total cost of the nonprofit’s cataract surgery program. This includes only the cost to run surgical outreach campaigns that directly perform cataract surgeries, and not the cost of adjacent activities like training local eye care professionals and building new eye care facilities.

We generally assign a cost of $0 to medical equipment, drugs and supplies donated by corporations to nonprofits. We assume that these goods have zero opportunity cost because they would have gone to waste if they had not been donated. It is possible that some of these goods could have been sold if they had not been donated. However, we assume that the benefits of donating — for instance, tax deductions and positive public image — sufficiently offset the foregone sales revenue.

Many cataract surgery programs benefit from skilled labor donated by medical professionals. We generally assign a cost of $0 to their time because we believe volunteers benefit in important non-monetary ways — fulfillment, for instance, in supporting a cause they are passionate about.

Beneficiary costs

Some nonprofits charge a nominal fee for cataract surgeries. We deduct this revenue from the nonprofit’s program cost, then add this revenue to beneficiary costs.

Partner costs

We assume partner costs are $0 unless they are reported.


Methodology for calculating impact

To calculate the impact of a cataract surgery program, we divide the total program-related costs incurred by all cost-bearing parties by the total number of people whom the nonprofit cured of blindness. Crucially, the numerator and denominator must match logically: The numerator reflects the costs incurred in generating the attributable outcomes reflected by the denominator.


Cost-effectiveness benchmarks

Using guidelines set by the W.H.O.,16 we set cost-effectiveness benchmarks based on the cost to avert one disability-adjusted life year (DALY) relative to the gross domestic product (G.D.P.) per capita of the country or countries in which the program operates. If a program can avert one DALY for less than three times the G.D.P. per capita, it is awarded 4 stars; if it averts one DALY for less than once the G.D.P. per capita, it is awarded 5 stars. The tacit assumption underlying the W.H.O. guidelines is that the willingness to pay for better health is linked to national income.17

We implement these benchmarks in five steps.

  1. First, we estimate the years of life lost due to premature death associated with cataract blindness. Based on a global study of cataract surgery patients, we assume an average patient age of 65.18 At age 65, we estimate that patients with untreated cataract have about 70 percent as long left to live than those with treated cataract, based on a population-based cohort study.19 To obtain additional life expectancies of the average 65 year old in each of the countries where nonprofits operate, we reference country-specific W.H.O. life expectancy tables.20 In Mexico, for instance, the additional life expectancy of a 65 year old is 18.5 years; assuming this figure also applies to patients with successfully treated cataract, the estimated additional life expectancy of a patient with untreated cataract is about 13 years (70 percent * 18.5 years). The years of life lost to premature death is therefore about 5.5 (18.5 years - 13 years).

  2. Next, we estimate the years of life lost due to living with a disability. We refer to disability weights reported by the Global Burden of Disease.21 The disability weight associated with blindness due to cataract is 0.187. This represents a loss of health of 18.7 percent, where a loss of 100 percent is death and a loss of 0 percent is full health. The disability weight of moderate vision impairment due to cataract is 0.031.22 We assume that the difference between the two, 0.156, represents the annual health loss averted by cataract surgery. We then multiply 0.156 by the number of remaining years an average 65 year old with untreated cataract will live (13 years in the above example for Mexico): 0.156 reduction in disability weight * 13 years = 2 years lost to disability.

  3. We add the years of life lost due to premature death and the years of life lost due to living with a disability: 5.5 + 2 = 7.5 DALYs.

  4. Next, we calculate each nonprofit’s cost to avert one DALY. For example, if a nonprofit cures one case of cataract blindness for $200, it averts one DALY for $27 ($27 = $200 / 7.5 DALYs).

  5. Finally, we compare the nonprofit’s cost-per-DALY-averted to the benchmarks described above. For example, Mexico’s G.D.P. per capita is $8,903. The 4-star benchmark is therefore $26,708 and the 5-star benchmark is $8,903. Because $27 is less than $8,903, the nonprofit is awarded 5 stars.


Nonprofit checklist of data needed to calculate impact

The following data is necessary to estimate the impact of cataract surgery programs. Some data points, such as the number of surgeries, must be reported by the nonprofit. Others, such as the efficacy rate of cataract surgeries, can be drawn from the research literature or administrative data.

Table 1

Checklist item

Required from nonprofit?

Details

Program activities

Yes

A program is a set of goods or services provided by the nonprofit to a population of beneficiaries with the goal of improving one or more outcomes. Generally, a program consists of the same components delivered to each beneficiary, with only minor deviations across different settings.

Geography

Yes

We recommend nonprofits report the number of surgeries they performed by country or more precise location. This allows us to use location-specific data in the impact calculation, e.g., cataract surgical coverage rate in that location.

Timeframe

Yes

We recommend nonprofits report annual figures that align with their fiscal year.

Type of cataract surgery

Yes

Surgery type is used to determine the efficacy rate of surgery. The major types include: manual small incision cataract surgery; extracapsular cataract extraction; and phacoemulsification.

Number of cataract surgeries performed

Yes

We recommend nonprofits report the number of cataract surgeries performed by geography and timeframe, counting as a single surgery any bilateral surgeries (operation on two eyes) for the same person.

Efficacy rate of cataract surgeries

No

If not reported by the nonprofit, an estimate from the research literature is used based on the type of cataract surgery performed.

Rate of vision recovery without surgery

No

Our default assumption is that cataract will progress toward blindness if left untreated. This rate is thus assumed constant at 0 percent.

Counterfactual treatment coverage rate

No

If not reported by the nonprofit, rates of existing national or regional coverage of cataract surgery are used. From the rate of coverage, the rate of access to essential health services is subtracted.

Program cost

Yes

We recommend reporting total costs, including costs paid out of pocket by volunteers. However, our cost calculations exclude volunteer time and resources donated in kind.

Beneficiary cost

No

We recommend reporting participant costs if they are substantial. They can be estimated at $0 if they are not substantial. Some nonprofits charge patients a nominal fee. We deduct this revenue from the nonprofit’s program cost, then add this revenue to beneficiary costs.

Partner cost

No

We recommend reporting partner costs if they are substantial. They can be estimated at $0 if they are not substantial. Partner costs are infrequently reported and are defaulted to $0 for cataract surgical campaigns.

Limitations of our analysis

Cost of reaching special locations

Some nonprofits may have to incur additional costs to reach particularly disadvantaged patients. For instance, a nonprofit may have to spend more on outreach to make the residents of a remote village aware of its cataract surgery program.

Cost of reaching special locations

The same goods and services may be priced and taxed differently in different locations, making it more or less costly to run an identical program depending on where the nonprofit chooses to work. This is, in part, accounted for by our country-specific cost-effectiveness benchmarks, which are set based on the national G.D.P. in the country or countries where the nonprofit’s program operates per W.H.O. standards.

Specific counterfactuals

To understand the impact of a program, we ask the counterfactual question: What would have happened to beneficiaries if the program had not, counter to fact, been there to serve them? Because the vast majority of nonprofits have not conducted impact evaluations, we need to construct our own counterfactuals based on public data sources and the research literature. But in doing so, we risk masking variation in effectiveness across nonprofits. For instance, under our methodology, any nonprofit working in the same region faces the same counterfactual assumption (the percent of patients who otherwise would have had successful cataract surgery). Nonprofit A might specifically seek out those on the margins of health care systems — and in so doing, expend substantial resources on, for instance, needs assessments to channel its resources where they are most needed. Meanwhile, Nonprofit B may work in a neighboring town with moderate existing access to services. But lacking data on counterfactual access specific to each nonprofit, we apply the same regional counterfactual in both cases, masking the variation in their targeting.

Data quality

Our estimates rely on data made public by nonprofits on their websites, annual reports, financial statements and Form 990s. There are, of course, ambiguities in the data and our interpretation of the data may not always match the nonprofit’s intention. For instance, it may not always be clear whether a nonprofit is reporting the number of eyes or the number of people it operated on, so our analysts must make inferences from contextual cues. Or, as is often the case, nonprofits may not report their own surgical efficacy rates, so we must draw on averages from field studies of surgical outreach camps elsewhere. If a nonprofit outperforms or underperforms field averages, our cost-effectiveness estimate will likely be an underestimate or overestimate, respectively. For more detail on our sources of data and how we interpret them, please see Reference Manual on Data Analysis.

Representativeness of (analyzed) programs

We only issue ratings for nonprofits if we can perform analysis on 15 percent or more of the nonprofit’s total program budget. This approach means some nonprofits are rated on only some of their programs. The remaining programs, which we could not analyze, could be more or less cost-effective than the programs we analyzed.