Applications of systems engineering for testing and evaluation: A human systems integration perspective.

Abstract

Our overall objective in this chapter is to demonstrate how systems engineering (SE) and human systems integration (HSI) help manage complexity, integrate human elements, and mitigate risks when conducting testing and evaluation (T&E). SE supports T&E by connecting key information elements to enable decision makers to verify that system requirements and performance parameters have been met. The basic methods of T&E are described by Charlton and O'Brien (2002) in their Handbook of Human Factors Testing and Evaluation.The achievements of SE have contributed greatly to the exponential technological advancement of complex systems. The systematic, visual presentation of complex systems reduces inconsistency across hardware, software, and manpower requirements and allows one to trace the origins of challenging and costly design problems. Additionally, the application of SE methodology has proven to be a very powerful tool for anticipating safety issues and conducting extensive T&E.Specifically, this chapter provides a conceptual background in SE with a focus on T&E and the advantages it offers in terms of cost savings and increased safety over more traditional forms of system development. The chapter also provides a description of tools (e.g., systems modeling language [SysML]) that can be used in T&E to effectively streamline systems and products during life-cycle development. SYSTEMS ENGINEERINGSE is the structured approach developed to address two interrelated shortfalls in system development:(1) substantial failures to meet overall performance goals, and (2) gross overruns in cumulative program life-cycle costs. Regarding the latter, SE has focused on the very earliest program developmental phases, in which substantial proportions of life-cycle costs are fixed (Asiedu & Gu, 1998;Layer, Brinke, Van Houten, Kals, & Haasis, 2002; U.S. Department of Defense, 2013). As an example, computer systems outperform humans in computational capabilities, whereas humans outperform computers in cognitive and pattern recognition capabilities. Joining computer systems and human capabilities allows designers to build the most efficient and optimized systems for T&E.In conducting system T&E, research by David, Idasiak, and Kratz (2010) indicated that most systems have an inherent risk, which can be estimated using reliability engineering, performance and safety evaluation methods, calculations from a failure modes and effects analysis, or calculations based on the mean time between failures. Detailed methodology on how to conduct failure modes and effects analyses or determine mean failure times is given in Abernethy (2006) and Montgomery (2004).According to the International Council on Systems Engineering, a system is "an artifact created by humans consisting of components that pursue a common goal unattainable by each of the single elements" (Object Management Group, 2011, 5.1 SysML Block Definition Diagrams [BDD], 14). SE practice represents both the application of structured methods and the utilization of appropriate tools to cons...