PROJECT PLAN:
USDI - Geological Survey,Biological Resources Division,Midcontinent Ecological Science Center

Study Title:

Background and Justification: A decision support system can be described as an interactive, computer-based system designed to help decision makers solve poorly structured problems. Using a combination of models, analytical techniques, and information retrieval, such systems help develop and evaluate appropriate alternatives (Adelman 1992; Sprague and Carlson 1982). Decision support systems should focus on strategic decisions, not operational ones. More specifically, they should contribute to reducing the uncertainty faced by managers when they need to make decisions regarding future options (Graham and Jones 1988). It is in this light that we have chosen to develop a decision support system to assist biologists with the management of the Rocky Mountain population of trumpeter swans (Cygnus buccinator).

The many agencies represented by the Rocky Mountain Population of Trumpeter Swans Subcommittee of the Pacific Flyway Council (Subcommittee) wish to be able to integrate what is known about swan ecology into a model that can be used to simulate and test different management scenarios. They are concerned by their lack of a mechanism to objectively explore management options and identify critical information gaps that will contribute towards population recovery. They recognize that their decision making is cyclic, and they wish to iteratively plan, implement, evaluate, and improve their management strategies. Unfortunately, optimizing any management of migratory birds throughout a flyway with cyclic planning is so complex that it is often all but impossible to implement without computerized decision support (Sojda et al. 1994). And, past conditions and future needs are ecological constraints to current decisions. Distributed decision making approaches suit problems where the complexity prevents an individual decision maker from conceptualizing, or otherwise dealing with the entire problem (Boland et al. 1992; Brehmer 1991). The Swan Decision Support System will allow realistic and ecologically-based management for migratory birds at multiple geographic and temporal scales. It will do this through implementations of artificial intelligence methods such as expert systems (Rich and Knight 1991, Russell and Norvig 1995), blackboards (Corkill 1991, Nii 1986a, 1986b), and cooperative distributed problem solving (Carver et al. 1991, Durfee et al. 1989).

The USDI - Fish and Wildlife Service has been the primary federal agency recommending the development of a Swan Decision Support System. Through their encouragement, the primary principle investigator (Sojda) has focused on this problem in pursuing a PhD at Colorado State University. Part of the work outlined in this study plan is intended to form the basis for his dissertation. Field managers in both the USDI - National Park Service and the USDA - Forest Service have expressed an interest in this research, as well. State waterfowl biologists from Oregon, Idaho, Montana, Wyoming, Utah, and Nevada have also supported the concept.

Objectives: The broad thesis proposed is: cooperative distributed problem solving can be used to implement a flyway approach to trumpeter swan management. Unfortunately this is not directly testable for several reasons. Foremost are (1) the lack of any gold standard, in an ecological sense, against which the system's performance can be evaluated; (2) the need to have a working system that is forward-looking, simulating possible future scenarios as they are actually unfolding; and (3) the complications of verification and validation of any knowledge-based system that relies heavily on heuristics. Therefore, we have identified hypotheses (in Objective B) at a lower-level than the broad thesis. Based on the outcomes from testing these hypotheses, a qualitative assessment of broad goal attainment will be made. This project has two objectives:

Objective A. Provide a decision support system that allows the Subcommittee to examine management actions addressing population and migration objectives at a flyway scale, and allows individual land managers to evaluate management actions at a site specific scale.

Objective B. Test hypotheses regarding ecological issues and system operation.

1. Are trumpeter swan management potentials increased by decision support technology that provides the capability to plan in multiple geographic and temporal scales? For the technology to foster this, swan distributions as suggested by the Subcommittee need to be simulated, habitat recommendations must be satisfactory for supporting increasing populations, and recommendations must be reliable over a specified time period. (Management potentials are those ecological conditions that can be exploited in pursuing trumpeter swan objectives. Included are habitat quality, quantity, distribution, and availability, and freedom from disturbance.) Specific hypotheses to be subsequently tested are:
a. Recommendations are provided that simulate distributions of trumpeter swans in time and space different than those observed during 1985-1994.

b. Recommended acreages and availability of wetland habitats are satisfactory for maximizing the numbers of trumpeter swans and distributing them appropriately by season.

c. Simulated distributions of swans differ when the system is not allowed to consider future population objectives.

d. Simulated distributions of swans differ when the system is not allowed to consider wetland habitat conditions and related future constraints and options.

e. Simulated distributions of swans differ when the system is not allowed to consider the effects of disturbance.

f. There is an optimum time scale over which the system can make effective recommendations, in terms of both simulating long term consequences, and exceeding the capabilities of the knowledge and rule bases. This will establish temporal bonds beyond which the decision support system should not be used.

g. Recommendations do not allow simulated swan populations and distributions to fall below predetermined levels.

2. Does cooperative distributed problem solving among refuges, parks, other management areas, and the internal knowledge bases effectively integrate local management actions with small-scale landscapes? For this integration to occur, information must be shared among human and electronic nodes, individual knowledge bases should contribute to recommendations, and principles of adaptive management must be incorporated. Hypotheses to be subsequently tested are:
a. The system support module directs the internal rules so that individual local areas share information with each other and with the knowledge bases.

b. Simulations using the knowledge bases as actual inputs result in increased trumpeter swan management potentials when compared with simulations based on random inputs.

c. Certain individual knowledge bases have greater influence on swan management recommendations than others as evaluated using sensitivity analysis.

d. The system is able to identify information gaps. When it can, it is able to prioritize the contribution that eliminating each gap would make towards increasing the precision of the recommendations.

Simulation #1. If a particular management action is implemented at a particular site and a particular time, what are the consequences for that site and for other sites in the flyway?

Simulation #2. Given an objective for spatial and temporal distribution of swans, what is the best set of management actions across all sites to achieve this? The decision support system will also have the capability for the manager to provide an alternative objective.

Simulation #3. Given some subset of management action(s) across all sites, and given an objective for spatial and temporal distributions of swans, what is the best complementary subset of management actions to achieve this?

Simulation #4. Given a satisfactory set of management actions across all sites to achieve an objective for swan distribution, if an alternative management action were to be implemented at a particular site, what are the consequences for that site and for other sites in the flyway in terms of reaching their respective objectives?

To address Simulations #2-4, a more complex search of the solution space will be required, and cooperative distributed problem solving will be used. Each geographic node in the system will need to function both independently and collaboratively with themselves and with the knowledge bases, exchanging "tentative and partial results in order to converge on a solution" (Carver et al. 1991). These more complex simulations will require the concurrent development and posting of partially completed plans and potential management options from all geographic sites. The goal is to find a satisfactory set of solutions for management at all sites. This will be done by sharing information among themselves and with the knowledge bases and databases, and by recursively searching for a set of management options that satisfices the population level and distribution objectives, and that addresses the constraints in the system.

This project will build the decision support system in two distinct phases. Phase One will assemble a prototype system to demonstrate the feasibility of applying blackboards and cooperative distributed problem solving to flyway management of trumpeter swans. It will not concentrate on methodsof input and output, but rather on the underlying algorithms to address Simulations #1 and #2. Fundamental knowledge bases and databases will be built. Phase One will conclude during year three with the testing of the above hypotheses and the completion of Sojda's dissertation.

Phase Two will be dependent on the amount of programming and travel support available to effectively interact with swan managers, and is scheduled to begin in year 3. It will involve major expansion, refinement, and enrichment of the rudimentary knowledge bases to increase the system's applicability to swan management issues. The system will also be expanded to address Simulations #3 and #4. During Phase Two graphical user interfaces for input and output will be developed and field tested that will allow swan managers to use the models developed in Phase One on their own and without significant technical computer support. It is anticipated that the system will be accessible via the internet as a telnet or web application. In contrast during Phase One, simulations will be available but require some direct guidance of project personnel.

Data Handling and Analysis: Verification and validation of knowledge-based systems is known to be more problematic than in other modelling efforts for many reasons (Gupta 1991). Under some conditions, modelling research can test performance against a preselected gold standard. Often, as in this case, such a standard does not exist. This is particularly true with near real-time decision support that is expected to predict and guide future scenarios while those scenarios are, in fact, unfolding. Also, not only is it important for a system to handle the most common cases, it ought to be able to deal with extreme events. This latter attribute is one often only found with human experts. To overcome such problems, analysis can focus on verification and validation of the system. The "Objectives" section of this Study Plan delineates the hypotheses to be examined in this vein. Adelman (1992) hinges successful implementation of decision support and expert systems on incorporating three evaluation procedures, and they are incorporated in our verification and validation methods. However, we consider user satisfaction as part of validation.

Verification is ensuring that the system is internally complete, coherent, and logical. We will ensure that the rules associated with the system support module direct the system internally to adequately and appropriately share information in a cooperative distributed problem solving framework. In procedural programming environments, modularity allows for more straightforward verification than in symbolic, knowledge-based systems. In the latter, especially those using backward chaining inference engines, the number of possible states and interactions among those states eliminates the possibility of more traditional, exhaustive testing (Geissman and Schultz 1988). Pedersen (1989a, 1989b, 1989c) summarizes the need to consider knowledge base development as part of good verification procedures, and describes some of the common pitfalls to avoid. Verification of this system will be an iterative process of testing and altering the system for improvement. At a minimum, outputs resulting from using the internal knowledge bases as input ought to provide more useful recommendations than random input of knowledge.

Validation relates to examining whether the system is providing a realistic and useful model of swan ecology and management. Sprague and Carlson (1982) recommend that an organization building their first decision support system recognize that it essentially is a research activity, and that evaluation should center on a general, "value analysis". They state that iterative prototyping will ensure a quality product from the managers' perspectives, but recognize the qualitative nature of such evaluation. Our approach is an attempt to add analytic and quantitative rigor beyond that. Sensitivity analysis can be a powerful tool for validation, especially for heuristic-based systems, and for systems where few or no test cases are available for comparison (O'Keefe et al. 1987). Another issue suggested by Rushby (1991) is that it is necessary to show not only how well a system performs, but also to show that it can avoid a catastrophic recommendation. This is important in a species like trumpeter swan where there is great concern for low population levels. Wilkins and Buchanan (1986) provide one option, using penalty constraints, for debugging such situations when reasoning under uncertainty. However, their algorithms are specific to systems that provide diagnoses, and do not directly apply here.

It is sometimes possible to test expert system performance against an independent panel of experts (O'Keefe et al. 1987). We will not do that in this case for two reasons. First, the panel of experts needed for such an evaluation would be the same people who will be closely connected to system development itself. This would add such confounding effects that no reasonable experimental design is feasible. Second, one of the basic tenets of distributed decision making is that the system is addressing questions that are beyond the capability of single persons to conceptualize and solve (Boland et al. 1992; Brehmer 1991).

Validation of our system will ensure that the recommendations for timing of availability of habitats will be sufficient to support the requisite number of birds. We will also examine whether recommended distributions of swans change when the system is not allowed to consider particular parameters such as population objectives, wetland habitat conditions, or disturbance. Again, validation will be an iterative process. The overall intent will be to determine whether cooperative distributed problem solving is an effective technique for imparting a flyway management approach to the Rocky Mountain population of trumpeter swans. Additionally, we will query swan managers and experts about their satisfaction with the system. The encompassing question to be addressed by this qualitative and quantitative mix of verification and validation is simply: Was Objective A achieved?

Users: Members of the Rocky Mountain Population of Trumpeter Swan Subcommittee will be the primary users. It is composed of state wildlife agency waterfowl biologists, state and federal land managers, Fish and Wildlife Service's Regional Migratory Bird Coordinators and Flyway Representatives, and university experts.

Technology Transfer: Because the development of this system can only occur with close collaboration with the community of swan managers, technology transfer is part and parcel with the research effort. A rough prototype has already been developed and presented to a meeting of the Subcommittee, and subsequently comprised the base of an experimental webpage [http://webmmesc.mesc.nbs.gov/swan]. This will be continued but migrated to Sojda's workstation. Small workshops and personal interactions will continue throughout the project as a result of knowledge engineering efforts and prototype testing.

The system itself is scheduled to be developed using Exsys software on a PC, and then served as a Unix application on the workstation. It will be accessible to the managers either as a telnet or web application via the internet. Likely other internet technologies will be developed and available by the time the system is ready for final distribution.

A minimum of two manuscripts will be submitted to scientific journals, and presentations will be made at two scientific meetings.

Location: Richard Sojda is located with the Greater Yellowstone Research Group of MESC at Montana State University in Bozeman, MT. Development will primarily be accomplished there, with the system residing on a Sun workstation with internet access in his office. Sojda is completing his PhD at Colorado State University and resources there will also be utilized. David Hamilton is located at MESC in Fort Collins, CO.

Figure 2 depicts primary locations of concern to swan managers, showing the geographic venue of the project.

Work and Product Schedule (by Calendar Year):
 

Principal: Richard S. Sojda [web page] - sojda@montana.edu [email]

Collaborator: David B. Hamilton [web page] - david_b_hamilton@usgs.gov [email]

Other collaborators include:

Dr. John E. Cornely, Regional Migratory Bird Coordinator, USDI - Fish and Wildlife Service, Denver, CO; john_cornely@fws.gov

Dr. Denis J. Dean, Assistant Professor, Natural Resources Information Systems, Department of Forest Sciences, Colorado State University, Fort Collins, CO; denis@cnr.colostate.edu

Dr. Leigh H. Fredrickson, Professor & Director, Gaylord Laboratory, University of Missouri, Puxico, MO; leigh_fredrickson@muccmail.missouri.edu

Dr. Daniel Goodman, Professor, Environmental Statistics Group, Biology Department, Montana State University, Bozeman, MT; goodman@rivers.oscs.montana.edu

Dr. Adele E. Howe, Assistant Professor, Department of Computer Science, Colorado State University, Fort Collins, CO; howe@cs.colostate.edu

Dr. John B. Lomis, Associate Professor, Department of Agricultural and Resource Economics, Colorado State Univeristy, Fort Collins, CO; jloomis@ceres.agsci.colostate.edu

Adelman, L. 1992. Evaluating decisions support and expert systems. John Wiley and Sons, Inc. New York, N Y. 232 pages.

Boland, R. J., A. K. Mahewshwari, D. Te'eni, D. G. Schwartz, and R. V. Tenkasi. 1992. Sharing perspectives in distributed decision making. Pages 306-313 in: Proceedings of the Conference on Computer-Supported Cooperative Work. Association for Computing Machinery. New York, N Y.

Brehmer, B. 1991. Distributed decision making: some notes on the literature. Pages 3-14 in: Distributed Decision Making: Cognitive Models for Cooperative Work, J. Rasmussen, B. Brehmer, and J. Leplat, editors. John Wiley and Sons, Chichester, England.

Carver, N., Z. Cvetanovic, and V. Lesser. 1991. Sophisticated cooperation in FA/C distributed problem solving systems. Pages 191-198 in: Proceedings of the National Conference on Artificial Intelligence 1991.

Corkill, D. D. 1991. Blackboard Systems. AI Expert 6(9): 40-47.

Durfee, E. H., V. R. Lesser, and D. D. Corkill. 1989. Cooperative distributed problem solving. Pages 84-147 in: The handbook of artificial intelligence, vol. IV, A. Barr, P. R. Cohen, and E. A. Feigenbaum, editors. Addison-Wesley. Reading, MA.

Geissman, J. R., and R. D. Schultz. 1988. Verification and validation of expert systems. AI Expert 3(2):26-33.

Graham, I., and P. L. Jones. 1988. Expert systems: knowledge, uncertainty, and decision. Chapman and Hall. New York, New York. 363 pages.

Gupta, U. 1991. Validating and verifying knowledge-based systems. IEEE Computer Society Press. Washington, DC. 423 pages.

O'Keefe, R. M., O. Balci, and E. P. Smith. 1987. Validating expert system performance. IEEE Expert 2(4):81-90.

Pedersen, K. 1989a. Well structured knowledge bases - part I. AI Expert 4(4):44-55.

Pedersen, K. 1989b. Well structured knowledge bases - part II. AI Expert 4(7):45-48.

Pedersen, K. 1989c. Well structured knowledge bases - part III. AI Expert 4(11):36-41.

Rich, E., and K. Knight. 1991. Artificial intelligence. McGraw-Hill, Inc. New York, NY. Second edition. 621 pages.

Rushby, J. 1991. Validation and testing of knowledge-based systems: how bad can it get? pages 77-83 in: Validating and verifying knowledge-based systems. U. Gupta, editor. IEEE Computer Society Press. Washington, DC.

Russell, S., and P. Norvig. 1995. Artificial intelligence: a modern approach. Prentice-Hall, Inc. Englewood Cliffs, NJ. 932 pages.

Sojda, R. S., D. J. Dean, and A. E. Howe. 1994. A decision support system for wetland management on national wildlife refuges. AI Applications 8(2):44-50.

Sprague, R. H., Jr. and E. D. Carlson. 1982. Building effective decision support systems. Prentice-Hall. Englewood Cliffs, NJ. 329 pages.

Wilkins, D. C., and B. G. Buchanan. 1986. On debugging rule sets when reasoning under uncertainty. Proceedings of the Fifth National Conference on Artificial Intelligence (AAAI-86) 1:448-454.