A Case Study on Selecting a Best Allocation of New Data for Improving the Estimation Precision of System and Subsystem Reliability Using Pareto Fronts

Appl Stoch Models Bus & Ind

2018

Self Cite

Everyone knows the expression "A picture is worth a thousand words," and this effectively summarizes the ability of graphical summaries to convey information and persuade. However, in many cases, the goal for the right visualization is to encourage and guide discussion while helping focus a team to make carefully considered, defensible, and data-driven decisions. The aims of graphics differ if we are trying to communicate the merits of a single choice versus outlining several contending alternatives for further comparison and discussion.These choices each have their own strengths and weaknesses depending on how we value different criteria. They also serve different purposes at various stages of decision making. Often the role of statisticians is not to provide a single answer but to provide rich information and summaries in a manageable and compact form to enable productive discussion among team members. Through a series of diverse examples, we present principles and strategies for encouraging discussion and informed decision making and discuss how they can be integrated with versatile use of graphical tools for examining multiple objectives, framing trade-offs between alternatives, and examining the impact of subjective priorities and uncertainty on the final decision.

Section: Pareto Fronts For Design Optimizationmentioning

confidence: 99%

Graphics to facilitate informative discussion and team decision making

Appl Stoch Models Bus & Ind

2018

Self Cite

“…Choosing between the different CAs on the Pareto front has to do with the priorities of the experimenter and involves subjective choices about how to trade‐off between improvements in one criterion at the expense of the other. Lu et al provide guidance and some graphical summaries that allow exploration of different alternatives based on the goals of the experiment. In this case, we suppose that an experimenter wants to emphasize Ave { r ij } more strongly than Ave { r i } but without exclusively focusing on the first criterion.…”

Section: Wild Card Examplesmentioning

confidence: 99%

Selecting the Best Wild Card Entries in a Covering Array

Kim

Jang

Quality & Reliability Eng

2017

Self Cite

For some covering arrays, there are some wild card positions in the array where there is flexibility about what factor levels can be chosen with no impact on the basic properties of covering array, because all of the required pairs have been covered. The choice of how to fill these wild card positions can influence other properties, such as the degrees of orthogonality or three-way coverage of the array. In this paper, some criteria are proposed for identifying the best choices for the wild card positions to create covering arrays with highly desirable properties. Accompanying graphical summaries are also described to highlight differing performance for several examples.

“…Constructing the PF is straightforward from an enumerated list of solutions, with several software packages having an implementation of this automatically. If the PF needs to be constructed within a search algorithm of candidate solutions, the Pareto aggregating point exchange (PAPE) algorithm and its enhancements for design selection have been used.…”

Section: Introductionmentioning

confidence: 99%

Selecting a near‐optimal design for multiple criteria with improved robustness to different user priorities

Burke¹,

Quality & Reliability Eng

et al. 2018

Self Cite

In a decision‐making process, relying on only one objective can often lead to oversimplified decisions that ignore important considerations. Incorporating multiple, and likely competing, objectives is critical for balancing trade‐offs on different aspects of performance. When multiple objectives are considered, it is often hard to make a precise decision on how to weight the different objectives when combining their performance for ranking and selecting designs. We show that there are situations when selecting a design with near‐optimality for a broad range of weight combinations of the criteria is a better test selection strategy compared with choosing a design that is strictly optimal under very restricted conditions. We propose a new design selection strategy that identifies several top‐ranked solutions across broad weight combinations using layered Pareto fronts and then selects the final design that offers the best robustness to different user priorities. This method involves identifying multiple leading solutions based on the primary objectives and comparing the alternatives using secondary objectives to make the final decision. We focus on the selection of screening designs because they are widely used both in industrial research, development, and operational testing. The method is illustrated with an example of selecting a single design from a catalog of designs of a fixed size. However, the method can be adapted to more general designed experiment selection problems that involve searching through a large design space.