Cost Benefit and Cost Effectiveness Analysis in Program Eval

At What Price? Benefit-Cost Analysis and Cost-Effectiveness Analysis in Program Evaluation





James Edwin Kee, professor at George Washington University, discusses the purposes,strengths, and limitations of benefit-cost and cost-effectiveness analyses to determine the relative costs and benefits of the programs. The transcript of that lecture is given for the benefot of MPH students of Abasyn University, Peshawar.


Introduction

In our current age of accountability, public and private sector funders are increasingly
concerned with the relative costs and benefits of the programs they fund.
Benefit-cost (or cost-benefit) and cost-effectiveness analyses can be useful
quantitative tools to help address these concerns. However, they differ in
their purposes, and each has strengths and limitations.

Benefit-cost analysis is an applied branch of economics that attempts to assess
service programs by determining whether total societal welfare has increased
(in the aggregate, people have been made better off) because of a given project
or program. It can be used in evaluations of existing programs to assess their
overall success or failure, to help determine whether the programs should be
continued or modified, and to assess the probable results of proposed program
changes. Benefit- cost analysis consists of three steps: (1) determine the
benefits of a proposed or existing program and place a dollar value on those benefits;
(2) calculate the total costs of the program; (3) compare the benefits and the
costs.

Cost-effectiveness analysis is an alternative to benefit-cost analysis that
relates the cost of a given alternative to specific measures of program objectives.
A cost-effectiveness analysis helps to compare costs to units of program
objectives and may be the first step in a benefit-cost analysis if the analyst
then decides to attempt to place a dollar value on the benefits. Unlike benefit-cost
analysis, cost-effectiveness analysis does not produce a “net benefit” number, with benefits exceeding costs or costs exceeding benefits.

However, a cost-effectiveness analysis can determine that a program which costs
$1 million produces ten units of outcome x, twelve units of outcome y, and
twenty units of outcome z. Or, if the units are alike, it can determine the
cost per unit of outcome.

An example of these two methods of analysis using a hypothetical dropout
prevention program is presented in Box 2.



Box 2: Hypothetical
Cost-Effectiveness and Benefit-Cost

Results for Dropout Prevention Strategies










Challenges in Conducting Benefit-Cost and
Cost-Effectiveness Evaluations Identifying and Measuring Costs



Identifying and measuring costs, and in the case of benefit-cost analysis,
quantifying and placing a dollar value on the benefits, is the biggest challenge to the evaluator
trying to conduct these types of analyses. Direct costs (such as personnel, materials,
and equipment) are often relatively easy to account for. Indirect costs (such
as overhead, costs to other providers supporting the intervention, and costs to
participants) as well as capital costs (such as buildings and computers) can be
more difficult to calculate. Finally, intangible costs (such as the value of
wilderness) are those for which the evaluator either cannot assign an explicit price
or chooses not to. Lack of assigned price does not mean that intangible costs
are unimportant; indeed, in presenting any results of these types of analyses, the
evaluator should point out the intangible costs and benefits, thereby enabling the
decision maker to consider these as he or she examines those benefits and costs that are quantified. When identifying any benefit or cost, it is
important to state its nature clearly, to state how it is being measured, and
to list any assumptions made in the calculation of the dollars involved.



Identifying and Measuring Benefits



Identifying benefits can also be tricky. As with costs, there are direct,
indirect, and intangible benefits. In the case of benefit-cost analysis,
placing a dollar value on the benefits is also a challenge. The evaluator might
choose a market value, when one is available, or a surrogate such as
willingness to pay. Because of the redistributional nature of government
programs, public agencies and those who evaluate them must be concerned with
who benefits as well as the amount of benefits in addition to the costs.

Where quantifying benefits is difficult, costly, or viewed as inappropriate, cost-effectiveness
analysis can be used. Cost-effectiveness evaluation does not require that the
evaluator place a dollar value on the benefits. This is particularly useful in
cases where the benefit of a program is “lives saved.” While there are various
ways to place a dollar value on a life saved or lost, each is controversial. In
contrast to a benefit-cost analysis, a cost-effectiveness evaluation would
calculate the cost of the program per life saved without making a judgment
about the dollar value of that life. The evaluator would then present the
results to the decision maker who must decide whether an outcome is worth the dollar cost when viewed in light of alternative uses for the funds.

A major challenge in cost-effectiveness analysis is the fact that programs frequently
generate more than one type of benefit. For example, an education system might
target more than one population group in the school system. When conducting a
cost-effectiveness analysis comparing programs with multiple benefits, the
evaluator may need to place weights on the relative benefits to assist the decision
maker in making comparisons. If this is not done, the comparison becomes quite
subjective. Yet assigning weights often becomes at least as problematic as
assigning dollar values to each benefit: how do the benefits to one population group
outweigh those to another, for example?



Boundaries



Another challenge in conducting benefit-cost and cost-effectiveness analyses is
determining the geographic scope of an analysis. While the focus may be within a
certain jurisdiction, such as a state, there may be benefits or costs that spill
over to neighboring jurisdictions. It might be tempting to ignore spillover effects,
but this can be unwise since spillovers often have political consequences. The
question for the evaluator is whether to consider only those benefits and costs
that accrue to the population within the jurisdiction for which the evaluator is
doing the analysis.



Detail



One of the biggest dangers in these analyses, as in many other areas of
evaluation, is the “black box” syndrome. Instead of laying out the relevant
issues, assumptions, and concerns, the evaluator may be tempted to hide the
messiness of the analysis from the decision maker, presenting a concise answer
as to the net benefits or costs or cost-effectiveness. However, it is the
detail—the assumptions involved and the sensitivity of the analysis to
particular assumptions—which may be of most use to the decision makers in
judging the value and usefulness of the evaluator’s work.



Deciding Between Cost-Effectiveness Analysis and Benefit-Cost Analysis

Those faced with deciding between the two types of analysis may find it helpful
to keep three basic questions in mind:




1. How will you use the results? Benefit-cost analysis enables you to compare strategies
that do not have the same outcomes, or to compare strategies across different
areas of public expenditure (e.g., health, welfare, justice).
Cost-effectiveness analysis is useful for comparing strategies that are trying
to achieve the same objective (e.g., increased graduation rates).




2. What resources do you have? Benefit-cost analyses typically require more resources,
because they take more time for analysis and involve significant methodological
expertise (often in economics), such as the capacity for determining the
discounted present value of a stream of benefits and costs.




3. How difficult are costs and benefits to value? While you may want to have as
much information as possible on both benefits and costs, you must weigh the value
of the increased accuracy gained from the accumulation of new data against the
costs associated with the data collection. Thus, any analysis should begin by
assimilating existing data to determine whether it is sufficient.

The more intangible the benefit (for example saved wilderness), the more likely
it is that a cost-effectiveness analysis will be of greater use to decision makers.
This type of analysis can help them assess whether a cost is justifiable, when
compared with other uses of the same funds.

It is important to note that benefit-cost analysis and cost-effectiveness analysis
could lead to different conclusions about the same program, depending upon how
benefits are valued in dollar terms. However, if the evaluation is concerned
with a program with a single objective (or closely related objectives), programs
or alternatives achieving the highest cost-effectiveness should also achieve
the highest benefit-cost ratio.

Neither benefit-cost analysis nor cost-effectiveness analysis is a panacea.

Both require judgments on measurement issues that should be brought to the
attention of the decision maker. However, both techniques are useful to provide
a format for analysis that can lead to better decisions.




James Edwin Kee

Giant Food, Inc. Professor of Public/Private Management

School of Business and PublicManagementGeorgeWashingtonUniversity