Evaluating Multiple-Attribute Decision Support Systems

Those who conduct integrated assessments (IAs) are aware of the need to explicitly consider multiple criteria and uncertainties when evaluating policies for preventing global warming. MCDM methods are potentially useful for understanding tradeoffs and evaluating risks associated with climate policy alternatives. A difficulty facing potential MCDM users is the wide range of different techniques that have been proposed, each with distinct advantages. Methods differ in terms of validity, ease of use, and appropriateness to the problem. Alternative methods also can yield strikingly different rankings of alternatives. A workshop was held in which climate change experts and policymakers evaluated the usefulness of MCDM for IA. Participants applied several methods in the context of a hypothetical greenhouse gas policy decision. Methods compared include value and utility functions, goal programming, ELECTRE, fuzzy sets, stochastic dominance, min max regret, and several weight selection methods. Ranges, rather than point estimates, were provided for some questions to incorporate imprecision regarding weights. Additionally, several visualization methods for both deterministic and uncertain cases were used and evaluated. Analysis of method results and participant feedback through questionnaires and discussion provide the basis for conclusions regarding the use of MCDM methods for climate change policy and IA analyses. Hypotheses are examined concerning predictive and convergent validity of methods, existence of splitting bias among experts, perceived ability of methods to aid decision-making, and whether expressing imprecision can change ranking results. Because participants gained from viewing a problem from several perspectives and results from different methods often significantly differed, it appears worthwhile to apply several MCDM methods to increase user confidence and insight. The participants themselves recommended such multimethod approaches for policymaking. Yet they preferred the freedom of unaided decision-making most of all, challenging the MCDM community to create transparent methods that permit maximum user control.

show abstract

“…Similar questionnaires have been used previously to evaluate MCDM methods (e.g., Hobbs and Meier, 2000;Le ! o on, 1997;Zapatero et al, 1997).…”

Section: The Experimentsmentioning

confidence: 99%

An evaluation of multi‐criteria methods in integrated assessment of climate policy

Bell

Hobbs

Elliott

et al. 2001

Multi Criteria Decision Anal

View full text Add to dashboard Cite

show abstract

“…Such approaches will undoubtedly be built around decision support systems. Because of that, Zapatero et al (1997) tried to make further investigation to examine the relationship between the amount of structure offered by a DSS and the decision-making style of the decision maker. Using five software packages, 24 college faculty members participated in a repeated measures experiment.…”

Section: Mcdss In Educationmentioning

confidence: 99%

“…The use of the DSS to support the multi-criteria decision-making process is increasing, and the most popular and widely applied MCDA methods are those for which there is a computerized system to Production and supply chain management Chandrasekaran and Ramesh (1987); Bohanec and Rajkovic (1988); Troutt and Pettypool (1989); Dror et al (1991); Dyer and Forman (1992); Weber and Ellram (1993); Brans and Mareschal (1994); Norbis and Smith (1996); Belton et al (1997); Ghodsypour and O'Brien (1998) Barfod et al (2011). Education Weitz and Jelassi (1992); Zapatero et al (1997); Agrell and Steuer (2000). Human resources management Constantopoulos (1989); Tavana et al (1996) (2008).…”

Section: Evaluation Of Mcdss Studiesmentioning

confidence: 99%

Decision Support System and Multi‐Criteria Decision Aid: A State of the Art and Perspectives

Razmak

Aouni

2014

Multi Criteria Decision Anal

View full text Add to dashboard Cite

The aim of this paper is to highlight the role of the Decision Support System within the field of multi-criteria decision aid (MCDA). The MCDA tools have been incorporated into systems to create Multi-Criteria Decision Support Systems (MCDSSs). In our literature review, we noticed that more than 100 papers have been written over a 20-year period in which MCDSS was used as a decision-making tool. The present paper describes some real applications of MCDSS in different fields, harmoniously combined with decision-making methods such as analytic hierarchy process, Utility Additive, and Goal Programming. The present study proposes an integrative MCDSS evaluation through guidance on the tools most useful for a specific user with a particular decision problem.

show abstract

“…Evaluation methods and questions asked were similar to those used in other MCDA and decision support comparisons and evaluations (Evans and Riha, 1989;Hobbs et al, 1992;Gunderson et al, 1994;Zapatero et al, 1997;Qureshi et al, 1999;Bell et al, 2001;Lai et al, 2002). Participants were asked for their opinions on: the suitability of MCDA for the decision; the importance of some of the benefits of MCDA methods; how well the process met what they considered important in evaluating their decision and any other benefits or drawbacks they identified; their perceptions of each stage of the process (e.g.…”

Section: Evaluation Of Mcdamentioning

confidence: 99%

Application of multiple criteria decision analysis in the New Zealand agricultural industry

Dooley

Smeaton

Sheath

et al. 2009

Multi Criteria Decision Anal

View full text Add to dashboard Cite

Agricultural decisions typically involve multiple criteria, some of which are subjective. Business, environmental, and lifestyle criteria are all important criteria in these decisions. These criteria can be difficult to trade-off using traditional methods. Multiple criteria decision analysis (MCDA) provides a formal, quantitative means of evaluating agricultural decisions taking all these factors into account. MCDA for agricultural decisions was evaluated using three applied case studies (use of new processing technology, selection of beef policies, and selection of farm systems in an environmentally sensitive catchment). The case studies all differed in their problem types and decision-maker requirements. A multi-attribute value theory approach was used for all the cases. This approach was selected using a descriptive framework which took the method limitations, problem attributes, and decision-maker requirements into account. The differences between the case studies, the application and the implementation of MCDA, the overall success of the process, and the potential use of MCDA in agriculture going forward are discussed. While MCDA was used to help identify the best decision, the main benefits that the decision makers identified included: learning about the decision, a better understanding of their own and others' perspectives, a structured way to work through the decision, a means to explain the decision, and stimulation of discussion and sharing of ideas. These benefits were particularly important for the group decisions. Participants were not overly concerned with the ranking accuracy. Problems in implementation included an initial lack of commitment to the process, understanding of the process and decision, and ownership of the decision. The limited time decision makers had available contributed to this. The majority of the decision makers liked the MCDA process for these strategic decisions. The quantitative approach and the graphic presentation appealed to them. Hence, simple MCDA approaches may be as effective as more complex ones, and can deliver many of the benefits particularly where the time is limited. There has been interest in the use of MCDA going forward.

show abstract

Evaluating Multiple-Attribute Decision Support Systems

Cited by 25 publications

References 11 publications

An evaluation of multi‐criteria methods in integrated assessment of climate policy

An evaluation of multi‐criteria methods in integrated assessment of climate policy

Decision Support System and Multi‐Criteria Decision Aid: A State of the Art and Perspectives

Application of multiple criteria decision analysis in the New Zealand agricultural industry

Contact Info

Product

Resources

About