The Effects of Information Overload on Software Project Risk Assessment*

Abstract

Managers are responsible for providing effective information technology governance of the software development process. Ineffective governance leads to serious resource misallocations and negative consequences concerning Sarbanes-Oxley compliance. In order for managers to make informed decisions about software development projects, they often need more information than is available through normal information channels, that is, they need an in-depth review of the at-risk project. Such in-depth reviews, however, are costly. Hence, accurate identification of at-risk projects for in-depth review is critical to management's ability to govern. This research considers how two factors, information load and time pressure, affect the quality of the project-selection process. We examine quality by observing the decision strategies involved and then relating these strategies to subsequent decision making. An experiment was conducted with experienced information systems auditors using a combination of policy-capturing and computerized process-tracing techniques. The participants in our study cope with information overload by accelerating their decision-making process and adopting noncompensatory decision processes. Contrary to prior research, our process-tracing analysis suggests that participants rarely filter information, thus implying that decision makers are unable to process all the information. Coping by resorting to noncompensatory strategies did not decrease decision quality unless combined with accelerated information processing. Participants also increase their weight on the software project risk factors that they repeatedly access and that they view for longer periods of time. The theoretical contributions and practical implications regarding what actions managers can take to reduce the negative impact of information overload are discussed.

How much is enough? An investigation of nonprofessional investors information search and stopping rule use

“…On average, participants in this study indicate gathering 5.32 pieces of information, an amount consistent with prior research that shows individuals have limited information processing capacity (Miller, 1956;Chewning and Harrell, 1990;Stock and Harrell, 1995;Tuttle and Burton, 1999;Pennington and Tuttle, 2007). As predicted, participants that used the absolute standard rule report gathering more information in total than those using the differences and single criterion rules.…”

“…Specifically, investors using an absolute standard rule indicate gathering more information items and more detailed (i.e., financial) information than those using a differences rule. On average, investors describe gathering five pieces of information, which is consistent with findings from human information processing research that individuals are unable to process more than seven information cues at one time (Miller, 1956;Chewning and Harrell, 1990;Stock and Harrell, 1995;Tuttle and Burton, 1999;Pennington and Tuttle, 2007). Additionally, even though investors using an absolute standard rule report using the most financial information, the overall average is only one piece of financial information.…”

“…Information overload theory posits that due to limited cognitive capacities, individuals will utilize effort minimization strategies, such as heuristics, to reduce the cognitive burden associated with the task. Accordingly, individuals will not acquire all information but rather will use approximately six to nine pieces of information depending on how severe the overload conditions are to the decision making task (Chewning and Harrell, 1990;Stock and Harrell, 1995;Tuttle and Burton, 1999;Pennington and Tuttle, 2007). Understanding the heuristics used to stop searching for information is thus critical to better understand investment-related judgments and decisions.…”

“…Specifically, individuals' cognitive capacities are limited and they often use heuristics, or decision rules, to lessen the cognitive burden of decision making tasks (Miller, 1956;Chewning and Harrell, 1990;Stock and Harrell, 1995;Tuttle and Burton, 1999;Pennington and Tuttle, 2007). However, prior survey research often uses long lists of items which far exceed an individual's processing capacity.…”

Estimating the Assessment Difficulty of CVSS Environmental Metrics: An Experiment

“…Whereas it is obvious that the vulnerability assessment process can benefit from additional information on the vulnerable system, it is less clear whether current standard implementations result in better assessments when that information is added to the process. Human processing of information is known to significantly correlate with judgment errors (for example, [14] reports on the effects of 'information overload' on risk assessment practices in software engineering), but no account of the 'scalability' of existing security measurement best practices currently exists in the literature.…”