The exception handling effectiveness of POSIX operating systems

Abstract

AbstractÐOperating systems form a foundation for robust application software, making it important to understand how effective they are at handling exceptional conditions. The Ballista testing system was used to characterize the handling of exceptional input parameter values for up to 233 POSIX functions and system calls on each of 15 widely used operating system (OS) implementations. This identified ways to crash systems with a single call, ways to cause task hangs within OS code, ways to cause abnormal task termination within OS and library code, failures to implement defined POSIX functionality, and failures to report unsuccessful operations. Overall, only 55 percent to 76 percent of the exceptional tests performed generated error codes, depending on the operating system being tested. Approximately 6 percent to 19 percent of tests failed to generate any indication of error despite exceptional inputs. Approximately 1 percent to 3 percent of tests revealed failures to implement defined POSIX functionality for unusual, but specified, situations. Between 18 percent and 33 percent of exceptional tests caused the abnormal termination of an OS system call or library function, and five systems were completely crashed by individual system calls with exceptional parameter values. The most prevalent sources of these robustness failures were illegal pointer values, numeric overflows, and end-of-file overruns. There is significant opportunity for improving exception handling within OS calls and especially within C library functions. However, the role of signals vs. error return codes is both controversial and the source of divergent implementation philosophies, forming a potential barrier to writing portable, robust applications.

Using Behavioral Profiles to Detect Software Flaws in Network Servers

“…Robustness testing and fuzzers are some approaches that study the system's ability in handling with erroneous input conditions at their interface [2,18,22]. Throughout the years, these approaches have evolved into more intelligent, and less random, vulnerability detectors [6,31,36].…”

Robustness Testing of the Windows DDK

“…The paper uses robustness testing to measure how well these OS handle the inputs from a DD [1,5,12]. A group of functions from the Windows interface (for kernel-mode DD, these functions are defined in the Device Driver Toolkit (DDK)) was selected and experimentally evaluated.…”

Robust software - no more excuses

“…During our initial investigations with Ballista, we noticed a number of robustness failures in the math libraries in FreeBSD [24]. As an initial study, we chose to address these issues and determine how robust the libraries could be made, and at what cost.…”

“…Operating systems have shown to have at times poor robustness [24] [12]. Commercial distributed computing frameworks such as CORBA client software tend to exhibit problems as well [31].…”