Step 5 Establish and Apply Screening Criteria

We can now establish screening criteria to winnow the options and get ones that will meet the objectives.

Purpose

At this point, you are examining various sets of potential trains and tracks and figuring out what would cause a train wreck . If a train wreck (or fatal flaw) is unavoidable, drop that option. If it can be avoided, incorporate changes.

• To identify and eliminate options with fatal flaws

   

• To develop criteria that ensure options :
   
  • Can be supported by various publics and participants

     

  • Can withstand the scrutiny of adversarial entities

     

  • Can respond to the needs and objectives

     

  • Can be accomplished within the resources and constraints

   

• To retain workable options

   

• To document the results

Why?

You don't want to waste time or money evaluating options that won't work, yet you want a broad range of options.

You need to focus on solutions which will work by meeting the needs in environmentally acceptable, economically efficient, and politically implementable ways.

Carefully identifying and applying screening criteria is an indispensable step! A willy-nilly elimination of options will:

 
• Destroy credibility (Why wasn't my option considered?)

   

• Block workable solutions (x could have worked if only we had done y!)

   

• Let unworkable options continue (Why didn't you consider that factor?).

Screening criteria are applied to identify fatal flaws of proposed actions or elements. Once you screen for these fatal flaws, you can then concentrate on the remaining viable options. You also need to be able to determine scales or thresholds for these criteria. This will ensure that you can apply the screening criteria objectively and consistently. For example, a threshold might be that the option must ensure effluent meets a standard for concentration of a specific toxin. Any option not meeting this threshold would be considered fatally flawed.

How?

Right now, you are simply weeding out options that won't work--for one reason or another. Participants must determine what criteria are appropriate. Involve sponsors, decisionmakers, partners, and selected publics to help establish some of the criteria that are particularly meaningful to them. Doing this can be an excellent way to establish trust and confidence in the Federal presence. Specialists need to contribute their expertise to the overall whole so that criteria are as comprehensive, consistent, and interactive as possible. For larger studies, work groups may be assigned to deal with specific values and disciplines, but groups must coordinate closely to avoid conflicts and Catch-22's. Recognize that conflicts over criteria can happen without shooting down options now.

Technological, social, economic, and environmental analyses , as well as evaluations of public acceptability are performed and the results displayed for comparison. Applying the screening criteria is as much a documentation process as it is an analytical process. Documenting the process provides the paper trail that shows where and why some options were discarded and others were carried forward.

Search for Fatal Flaws

Remember why Reclamation is involved.

Keep "too difficult" options: Difficulty is a factor in evaluating the relative worth of alternatives. It is not a fatal flaw. For example, moving the castle may be difficult, but not impossible. (Check out the London Bridge now in the U.S.)

Constraints are a jumping off place to identify possible train wrecks. Ask:

 
• What can't we do? (This will provide a list of fatal flaw triggers to work with.)

   

• At what level or under what conditions would this option be unacceptable? (This will provide a range of extremes that can become fatal flaw triggers.)

   

• Why can't we do that? (This will explain why the action would trigger a fatal flaw.)

Potential implementors can explain what factors you need to look at to ensure that options will be workable. This involvement will not only look at the factors that count in the real world, but it will also enhance your credibility. These actions will help implementors conclude that you are serious and you do care about solving the problem in a realistic, responsible manner.

Before dismissing any option as unworkable--think through the "what if"--what if the fatal flaw or roadblock were removed? (e.g., NEPA requires that we look beyond existing legislation). A lot of great ideas were once scoffed at .

Each process will have its own set of fatal flaws. For example, implementation time may be a fatal factor in a court-mandated action, while it may not be in a long-term basin management plan. Fatal flaws may lurk in any facet of the option: institutional, organizational, economic, social, physical, etc. Some factors to consider:

Workability.

Will it meet the need and fulfill the objective? Are there enough resources to implement and support it?

 

Decision factors.

What are the policy issues? What are the institutional judgments, laws, and philosophy?

 

Technical factors.

How will this affect physical and biological interrelationships within the ecosystem? (Consider factors such as organizational, environmental, hydrology, social, or geology.) It may be necessary to repeat some of the methods used to assess resources.

 

Key variables.

What are the key interrelationships and dependencies? How will they influence the different possible solutions?

 

Exclusion factors.

What are the internal and external constraints? (Consider laws, endangered species, time, organizational policy, etc.)

 

The list of fatal flaws helps form your criteria. Examine this list to ensure that you have all the criteria you need and no more. Ask:

 
• What will drive the decision? Do the criteria reflect this?

   

• What does each criterion add?

   

• Why must each option meet that criterion?

Determine Thresholds and Scales

Establishing criteria without establishing limits fashions too wide a screen--practically every option will fail. For example, if water quality is a criteria, determine what standards options must meet. If time is a criteria, determine which timeframes are acceptable (implementable in 1 to 5 years or in 5 to 20)? Thresholds and scales will vary with each process. Ask:

 
• At what threshold would an option fail to meet the need or objective?

   

• When would it violate a limit ?

Be as specific as possible, given the level of detail (e.g., At what flow level would flood damage occur? At what point would water supplies or flow demands fail to be met?)

Do a Reality Check

Pitfall: If your criteria are too limiting or site-specific, you may exclude options needed to assemble a full range of potential alternatives.

Relate the criteria back to the needs, objectives, and resources to ensure that they are on track. Potential implementors are in the best position to tell you what will and won't work. Ask them what factors will need to be considered and what thresholds are realistic. Do a preliminary analysis on the interrelationships of options--what would need to be done first? What actions depend on other actions? What actions depend on seasons or other time factors? Why?

Determine Procedures

The team should agree on a procedure to measure these scales or thresholds. How will the options be measured? What analyses will be used? Whatever criteria the team selects, consider several factors when settling on a procedure for applying the criteria:

Value measures.

How will we measure values such as public acceptability , effectiveness , feasibility , efficiency , and completeness ?

 

Accuracy.

How accurate do measurements, analyses, and projections need to be? Are data adequate to support the selected criteria? All specialists should review previous decisions and agree on a congruent level of detail for this step. This detail must be appropriate to the type of action or study, the complexity, controversy, and data available.

 

Reliability.

What is needed to ensure that the solution will continue to work?

If you can't agree on procedures, redraft the criteria and scales.

Apply the Screen

Use the criteria you have decided upon to determine which option to keep. Make sure that you consider each option in the same way. Keep options that meet the criteria for now and drop only those that clearly fail the criteria.

While it is important and cost effective to limit options to those which only address the stated objectives, being too adamant at this time may create roadblocks by:

• Overlooking previously unidentified needs
• Not recognizing changes in identified needs
• Not considering all of the participants' input
• Losing credibility
• Losing support of publics

Comparing all criteria at the same time can provide an important overall picture:

Document the Results

Documentation will help new players understand the constraints and show decisionmakers this was a reasonable, valid process.

Your efforts will be meaningless if you don't clearly document the results so participants can determine for themselves if the process is fair and reasonable. A summary document with concise text and the graphic displays can be distributed to all participants to ensure that nothing has been overlooked. The idea is to present factual information in a readily readable form supported by visual confirmation. The combination will help retention and understanding, and should make it easier to build consent for the decisions made.

You will need two kinds of documentation:

 

For viable options.

Agree on the type of displays you intend to use and use similar approaches for each criteria. Displays must be simple, consistent, and easily understood. Graphs and tables help readers compare various aspects. Make sure that each graph, table, and text lists the components and criteria in the same order. This helps readers find information quickly. A brief explanation of the methodology used puts the tables and graphs in context. Each discipline should maintain a working file of all the analysis for future backup and appendices. This working file is part of your personal work record. It is an official record and must be preserved.

 

For nonviable options.

Whenever you eliminate an alternative, document that you considered it and the rationale behind eliminating it. The analysis for a nonviable option should go only as far as identifying the fatal flaw. Document the flaw and present the analysis in the report. (This may be as short as one sentence.)

Tools

Useful tools may range from simple ratings to complex computer models. Display the screening results in a consistent manner: in tables, bar charts, pie charts, etc. Issue and process maps and influence diagrams focus attention on those parameters that affect or influence specific elements.

Rating Tables

(See the Toolbox for more tools)

 

Define, explain, and clearly document the classifications, ranking scales, and other measurements to show your thinking.

Results of initial screening could be displayed in simple comparative tables for which specific criteria are stated or displayed in a rating scale. For example, preliminary factors considered for geologic suitability of a structural site might include basic foundation geology, seismic risk, and availability of construction materials. Results of the rating could be displayed in a table similar to the following:

Brine replacement reservoirs
Alternate sites	Geology	Seismic	Embankment
Sad Iron Peak	3	2	2
Setter's Creek	2	4	2
Sugar Boy Mine	3	6	2
Sites are rated on a scale of increasing risk from 1-10 for each criteria. Risks less than 4 are acceptable for further examination. Ranking criteria are discussed in section 8.1 of the Miner's Creek Water Supply report.

(In this example, the Sugar Boy Mine site has an unacceptable seismic risk, and should not be examined further. Sad Iron Peak and Settler's Creek can continue as workable options--if they don't fail another screening test)

Risk And Uncertainty

Determine what an acceptable range is (e.g., less than a 5 risk) and then use a rating or matrix table to screen out options outside that range. Another rating scale useful in communicating with the public is an evaluation summary. This type of display could be used for any level of screening or evaluation process. You may not be able to precisely quantify something without further data. At this stage, merely identifying the level of concern ("none," "minor," "major," "unresolvable") will be enough--if you can justify that classification.

Constraints Analysis

Use constraints tables, force field analyses, or other tools that show what can and can't be done. Using a variety of analyses will help ensure that nothing is overlooked. You may be able to develop strategies to eliminate or reduce restraining forces to retain the option as workable. Be sure to document any changes to the option.

Decision Process Worksheets

Using the decision process worksheets to list your criteria will provide an overall view of the process so far. You can then take each option , match it against the criteria, and eliminate ones with fatal flaws.

Where information is suspect, identify and clearly describe the reliability of the data or estimates being used in screening. Future projections, measurement errors, and complex studies may make data uncertain. Sensitivity analyses, which adjust data to discover the allowable margin for error, help evaluate outcomes with uncertain data. Some ways to reduce risk and uncertainty about potential fatal flaws include:

• Gathering more detailed and refined data
• Including more safety features
• Committing fewer resources.

Scatter Diagram Analysis

Use a scatter diagram to spot possible fatal flaws. Determine two critical vectors that interact with each other, for example power output and redd survival. Graph each vector on the x and y axes. Plot results from each alternative. Draw a "safety region" where the results are ok. Anything falling outside of this region may be a fatal flaw.

Acceptability

At this point, acceptability will likely be limited to select publics, such as actual users of the resource and concerned environmental groups. A broader exposure will be applied when the various solutions are combined into alternatives. It may be best to focus public participation to those who helped identify the solutions. Use your public involvement specialist to help conduct an effective public participation program.

Look Forward

Carefully following and documenting this step should ensure that the options that remain are doable. Ensure that:

 
• The criteria focus on identified needs and stated objectives.

   

• The criteria are acceptable to the technical community.

   

• The important criteria provided by the public was included.

   

• The criteria thresholds and measurements were accurate and reliable for your detail level.

   

• All participants were invited to help develop the screening criteria.

   

Potential implementors were consulted.

Go/No Go

Ask potential implementors--Did we miss any key factors? Are the options that remain realistic and doable?

By analyzing the results of the screening process, you should be able to report to decisionmakers and the affected publics whether or not:

 
• There are workable options

   

• These options can be combined into implementable alternatives

   

• There is still justification for a Federal/Reclamation role to develop alternatives

   

• There are options that will meet identified needs and help achieve stated purposes (objectives)

This is a crucial go/no go decision point. If you have not identified options that work to meet the needs and fall within the Reclamation's role, then you must either:

Determine if the purpose and scope are appropriate.

Do you need to redefine the problem?

 

Review the process so far.

Is there anything that was overlooked? Have needs, objectives, or circumstances changed? Can you go beyond the constraints to change the process and free up solutions?

 

Develop more options .

Go back to the drawing board.

 

Redefine your involvement and rebuild the foundation .

Are there aspects of the problem we can solve that do fall within Reclamation's role ?

 

Terminate the involvement

See if someone else can act to solve the problems and work with those groups and individuals. Keep in touch with other participants.

Go On

Executive Summary and Process Tours:

Options <- ----> Alternatives

Step 4, Develop Options

Step 6, Develop Alternatives