| Step
7
Evaluate Alternatives
|
|
|
|
|
 |
|
| |
|
We can now evaluate the full range of
workable alternatives to see what will best solve the problem
in this situation.
|
|
|
|
|
|  Purpose
| 
|
| Pitfall:
Preselecting a solution before you evaluate all alternatives
is a big mistake. |
| |
|
|
- To evaluate and refine alternatives
- To find the optimal, "best," or most desirable alternative
- To present a clear analysis
 and comparison of
alternatives
- To document this analysis for the decisionmakers and
affected publics
|
|
|
|
|
| Why?
| |
| Frequently, unimplementable or extreme
alternatives are evaluated along with workable alternatives
to show why they won't work and to find some component
or compromise that will work. |
| |
|
|
In our search for the most desirable alternative,
an analysis of trade-offs among competing needs and solutions
is essential. Multi-purpose programs and projects meeting more
than one need must consider that optimizing the solution for
individual needs may cause problems in other areas (e.g. the
optimum economic alternative may create problems in the environmental
area...and vice versa.). By simultaneously viewing the impacts
in both areas, balance can be achieved. A compromise alternative
can allow both interests to consent
to a desirable resolution of problems which is stronger than
any alternative addressing merely a single interest.
The evaluation process is not to justify what you are doing
but to refine the alternatives. The "best" alternative cannot
be developed in the first run. Rather, the alternative that
best meets the needs and situation will evolve as a result of
the evaluation process.
|
|
|
|
|
| How?
| |
| At this point, no alternative is the
"best" one. Not all needs can be met to everyone's satusfaction
with one alternative. Evaluating alternatives involves
tradeoffs and compromises. You will probably end up with
very different alternatives from the ones you started
with. |
| |
|
|
This is where most of the analytical work
takes place. Disciplines work together to compare alternatives through
various analytical techniques. Through analysis, the information
needed for a decision is generated. At this stage, professional
judgment is replaced with factual data as much as possible.
These analyses will vary, depending
on what the decisionmakers need to make a decision. Sometimes,
e arlier work requires repeating
at a higher level of detail; for instance, a quick windshield
survey may have been
adequate for initial assessments, but detailed maps and samples
may be needed to authoritatively evaluate and screen for higher
levels of detail (accuracy).
|
|
| Context
| |
| Communicate with potential implementors
to ensure you evaluate the factors that actually count.
|
| |
|
|
Analyze the systems (e.g., physical, biological,
social, economic, and organizational) to determine how alternatives
will interact with other processes. Ask:
- What is the setting in which the decision will be made?
- How will the solution be implemented?
- How will it be monitored and followed up?
- How will it interact with other processes and actions?
Develop indicators for each significant
issue to track and compare all alternatives consistently.
|
|
| Evaluation
Criteria
| |
| Criteria used to
screen options were to ensure that options would work.
Now evaluation criteria weigh workable alternatives to
chose the one that best meets the needs. |
| |
|
|
Evaluation criteria reflect what drives the
decision. The criteria applied here are based on a comparative
analysis of alternatives to provide the information necessary
to select the alternative. The basic question here is: What
do I need to know about the alternatives to choose one?
Relative weights assigned to the importance of the evaluation
criteria provide a basis for evaluating the relative merits
of these . Weights are what tell you what is more important:
85-percent fish flows with some power generation or 100-percent
fish flows precluding any other uses.
Carefully craft these criteria and weights so that you can develop a rationale behind your judgment--
why one alternative outranks another. No matter what the alternative,
no one will be totally satisfied. Tradeoffs among benefits and
impacts will have to be made. Look carefully at that last incremental
benefit to determine if it is worthwhile. For example, you may
have to settle for 90-percent solution if the 100-percent solution
costs four times as much.
Criteria may conflict--high flows may be needed for an endangered
species while stable flows are necessary for riparian habitat.
Determine and agree upon which criteria are more important (Is
meeting need x more important than meeting need y? Is speed
more important than cost? Is easy maintenance more important
than comprehensiveness?). These priorities should more or less
mirror the prority stack of objectives.
|
|
| Identify
Impacts
| |
| If you push down in one place, the problem
may pop up somewhere else. |
| |
|
|
Once you have alternatives that fit the evaluation
criteria, look at what effects they will have on the overall
systems. It won't do any good to solve one problem if you are
going to create larger problems elsewhere. For example, if providing
a water supply to one area robs the water supply from another
area, the overall problem isn't solved. Look at indirect effects
as well by thinking through the process (if x happens, then
y might occur, which would influence z). Examine the alternatives
within the context of other actions to determine cumulative
effects. One house on a mountain may not prove to be a problem,
but many subdivisions on that mountain could be.
|
|
| Determine
Data Needs
| |
| If the decision can't be made without
that particular data, then the data are significant, |
| |
|
|
Data needs will vary, depending on participants, solution
requirements, and problemshed interactions. To find
out what you need, ask:
- "Are these data significant to the decision?"
- "Do we need these data to make an effective decision?"
If your answer is:
- "Yes" then:
-
- Identify the data needed
- Identify and agree on indicators
- Determine how the data will be analyzed
to answer the relevant questions
- Collect only essential data germane to the current
level of detail
- "No" then:
- Don't waste time or money with it
- Document and support your decision not to gather
the data
Don't forget to get input from participants--and from interested
public! If you miss something now, it will come haunt you later.
Determine what issues are significant
and develop indicators to show impacts
to these issues. Determine the level of
detail needed for analyses by working backwards--figure
out what level of detail is needed for an effective decision
and for implementing the solution. Reaching agreements
on evaluation techniques and weights of objectives and criteria
is essential. Unresolved disagreements at this stage can drive
a wedge into the process, creating "splinter groups" with their
own methodology and even counter-analyses. These counter-analyses
may be equally valid, but they will cloud the overall issues
and focus attention and energy away from the real problem onto
relatively unimportant side issues. For example, a counter-analysis
may find that fish mortality is 14.9 percent, rather than 10.5
percent. This relatively small difference in analysis may not
shed more light on the comparative merits of alternatives and
may focus the conflict on numbers rather than solutions.
|
|
| Determine
Analysis Methods
|
|
Agree upon a procedure to analyze tradeoffs,
evaluation criteria, and impacts for each alternative. Ensure
that each alternative is treated in the same manner. Determine
methodology, schedule,
and priority. Laying out methodologies
and analyses now will save money later. It will help ensure
that disciplines work together so that results can be compared
and the necessary data for each analysis is collected. (Don't
waste time finding out flow temperatures if the biologists need
flow rates instead!) Also, schedules will show the interaction
of analyses (e.g., the hydrology work may need to be done before
the biology.)
|
|
| Refine
Alternatives and Re-iterate Analysis |
|
The first iteration will reduce the range
of alternatives to those considered most reasonable (despite
the ambiguity of the term) for further, more detailed study.
Although there is no magic cutoff, costs and time usually limit
alternatives carried through each further iteration to five
or less. Now that you have been able to compare and
evaluate alternatives and recognize tradeoffs, you can revisit
and revise earlier steps in the process to reflect the diversity
of input from all technical disciplines, publics, agencies,
etc. Keep decisionmakers in the loop to refine alternatives
and determine which ones to carry to the next iteration. A popular
myth-conception, that the set of alternatives
is locked in place, may rear up here. Showing the benefits of
refining the alternatives may help avoid this. However, if the
decisionmakers have reasons to narrow the alternatives under
consideration, this should be clearly documented. The second
iteration will lead to an even more detailed study and comparison
of the best alternatives. (Again, figure out the level of detail
you really need.)
|
|
| Document the Analysis
|
|
Showing the results of comparative analyses
in a draft document gives the overall picture. This helps form
agreements-- small compromises in content, wording, and presentation
may bridge the gap in larger conflicts. The draft document can
then be refined into a document for decisionmakers (e.g., a
NEPA compliance document,
initiative, resource management plan, or proposed guidance).
|
|
|
|
|
| Tools
|
|
A variety of analytical tools will be used
at this point--both to gather data and display information on
comparisons.
Get together with the core team and determine what tools
will be used to measure what. Be sure that measurements, comparisons,
and evaluations used are consistent assumptions and methods.
While you are comparing apples to oranges, at least use the
same properties and measurements to show how effective the alternatives
will be to meet the objectives and evaluation criteria.
Evaluation tools include:
|
|
| Tradeoff Analysis |
|
A tradeoff analysis examines many components,
factors, and criteria within the problem's context. This moves
away from the simplistic emphasis on only one factor which a
single discipline might apply.
In these analyses, factors (criteria, impacts, costs, etc.)
are weighted to reflect their relative importance. Impacts on
the factors are compared for each alternative to analyze benefits,
costs, and tradeoffs. Participants can change weights or data
to determine the significance of changes in criteria or priority
stacks. Sensitivity analysis provides room for comparisons and
enables participants to:
- Set sideboards for minimum performance and point out
fatal flaws (Even if you made the cost of the water as low
as possible, you could not compensate for water quality below
x standard.)
- Determine what is most important (What are the deciding
factors in a preferred alternative: wetlands, benefit-cost
ratio, amount of water delivered, etc.?)
- Build consent (Can
participants agree on an 85-percent solution if the 100-percent
solution takes twice as long?)
- Look beyond analytical differences (What would really
happen if juvenile salmon mortality rates at the gates were
20 percent rather than 5 percent?)
Consider software programs, such as MATS
(multi- attribute tradeoff system). MATS is designed to help
decisionmakers (or publics) make choices among alternatives
when many pieces of information must be considered. MATS provides
a framework for decision analysis and documentation, with content
provided by the user. The MATS process helps reduce the complexity
of developing alternatives by tracking all assumptions, factors,
weights, alternative comparisons, and tradeoffs
Sensitivity to changes in facts or values is easily displayed
for evaluation during the decisionmaking process. MATS can be
used to encourage tradeoff discussion among publics and technical
disciplines.
|
|
| Matrices
|
|
Evaluating alternatives comes down to which altenatives
offer what. Use matrix ,
rating , and ranking
tables to quickly show how alternatives stack up against weighted
evaluation factors. (For example, lining a canal may cost less
than piping, but will need more repairs and maintenance.)
|
|
| Pilot
Studies |
|
Consider testing the waters first with a pilot
study, phased implementation, or test market. For
example, an administrative function might be tested within one
group or division first. Participants in the test group can
provide insights into what worked and why. Armed with this information,
you can expand, modify, or drop the alternative. Draft documents
can also serve to test ideas and analyses.
|
|
| Interrelationships |
|
Analyses that examine interrelationships,
such as GIS mapping systems, can
help determine:
- How the components of alternatives would work together
- How alternatives would affect existing or future related
projects.
|
|
|
|
|
| Look Forward
|
|
Based on what you know at this point, begin
paving the way for selection and implementation. Most opposition
or conflicts occur because people
feel either that their concerns were ignored or the process
is unreasonable. This stage presents many opportunities to head
off these conflicts. Make sure that participants and especially
potential implementors have a chance to review the analysis.
If they are not involved now, it may be very difficult to persuade
people to implement the selected alternative.
|
|
| Recommending
|
|
The technical/partnership team can now recommend an alternative
to the decisionmakers. If an alternative doesn't clearly stand
out over the others, it may be necessary to select additional
evaluation criteria and refine the evaluation process.
Present the results to the decisionmakers and the public
in a final document. This usually concludes the activities of
the technical team. Celebrate! Final
documents:
- Provide a reality check (Did participants hear and understand
each other? Were the relevant factors, objectives, and tradeoffs
evaluated?)
- Provide something concrete for representatives to take
back to their groups
|
|
|
|
|
| Go On
|
|
Executive
Summary  and Process  Tours:
Alternatives <-----> Select
|
|
| |
|
|
page contents
See the 
Step 6, Develop Alternatives
Step 8, Select