This paper discusses the role that qualitative methods can and should play in engineering systems research and lays out the process of doing good qualitative research. As engineering research increasingly focuses on sociotechnical systems, in which human behavior and organizational context play important roles in system behavior, there is an increasing need for the insights qualitative research can provide. This paper synthesizes the literature on qualitative methods and lessons from the authors’ experience employing qualitative methods to study a variety of engineering systems. We hope that by framing the key issues clearly, other engineers who hope to join the qualitative path will build on what we have learned so far to enable greater insight into engineering systems.

Controlled experiments ranging from laboratory experiments to field experiments are becoming increasingly important within systems engineering and design (SE&D) research. Controlled experiments provide unique capabilities for testing hypotheses, and guiding the development of scientific theories, but their potential is not being fully exploited. As the systems engineering research community embraces different research methods commonly adopted in other domains, there is a need for understanding the role and appropriateness of those methods within the research process. While the design of experiments in SE&D research is not unique per se, we contend that aspects of representativeness in the SE&D context pose different experimental design challenges than do the problems typically studied by experimental social scientists. There is therefore a need to translate established practices to the SE&D context, and also belay widely held concerns about the value of experiments in SE&D research. To address this need, the goals of the paper are to discuss the potential role of controlled experiments in systems engineering and design research, to review the benefits and limitations of different types of experiments, and to highlight the key issues in validating experimental studies. The taxonomy and the discussion on validation build on the rich history in social and behavioral sciences, with specific adaptation to the SE&D context. The discussion is anchored in specific examples from the authors’ own research on open innovation in systems engineering.

No abstract

Open innovation methods, which broadly include: prize competitions, grand challenges, and collaborative communities, are increasingly being applied to complex systems problems. While external solvers (i.e., the crowd) have historically contributed some important innovations, there appears to be a limit to the kind of systems where open innovation can be successful. We contend that open innovation activities can be more broadly effective if seekers (i.e., problem owners) formulated their system architectures with "openability" in mind. In this paper, we build on existing theory to categorize the kinds of problems that are most amenable to crowd contributions, based on the mechanisms of solving. We call these "openable" problems. We then develop and demonstrate an architecting process that uses a facilitated expert workshop to elicit "openable" subproblems within a given systems-level problem. It treats the need for some openable subproblems as a constraint on the architecture. While not all systems should be designed with "openability" in mind, many organizations have faced political-level pressure to better leverage the crowd. In that context, we provide guidance on how to do so effectively. We demonstrate the potential value of our approach through application to the NASA Asteroid Grand Challenge-a highly complex system. The paper concludes by reflecting on the implications of this exercise for open innovation and system decomposition in general. K E Y W O R D S complex systems, decomposition, open innovation, system architecting System Engineering. 2018;21:47-58.

No abstract