Reliability Assessment of Commercial Off-the-Shelf Operating System Software: An Empirical Study

Santos, Caio Augusto Rodrigues dos; Prince, Marcela; Matias, Rivalino; Assuncao, Lucas Miranda; Maciel, Vinicius Fonseca

doi:10.1109/sbesc.2018.00038

Cited by 1 publication

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on quantitative analyses, the studies in [29] and [35] demonstrate that OS reliability metrics computed considering failures in both levels (Kernel and User spaces) are more accurate than metrics that consider only failures in the Kernel space. Figure 2.2 compares the OS reliability considering only Kernel failures (traditional approach) with an approach that considers OS failures at both Kernel and User spaces.…”

Section: Software Failures Analyzedmentioning

confidence: 99%

“…It becomes evident that following the traditional approach (dashed line) the OS reliability would be considered higher; nevertheless, the authors of [35] advocate that this result is not representative of the user experience since, from his or her perspective, the OS reliability is affected by any failure that causes the impairment or interruption of its features or services instead of only those confined to its Kernel space. The authors in [35] state that the solid curve (see Figure 2.2) would better represent the OS reliability perception by the users, which can be observed in practice, for any OS, by comparing the reliability figures provided by the manufacturer (based on the traditional approach) with the actual users' quality of experience regarding the OS reliability. Therefore, following the approach introduced in [29], this study considered OS failures observed at both (Kernel and User) spaces.…”

Section: Software Failures Analyzedmentioning

confidence: 99%

“…In most computer systems, the software layer is divided into User Application software (or simply application) and Operating System software. Given the hierarchical nature of these two tiers, applications fully depend on the operating system (OS) to be executed [35]. Due to this technical dependence, OS failures can severely impair even the most reliable application software [30].…”

Section: Software Under Studymentioning

confidence: 99%

“…To identify the main failed components in each category, all failures in the dataset were subdivided into 1,366 failure types; 35 Table 3.5 lists the five types of failures that consistently occurred in the dataset analyzed. They were ranked by three criteria; (i) greater number of groups the type is observed in; (ii) greater number of computers the type is observed in; (iii) greater number of occurrences of the type.…”

Section: Data Characterizationmentioning

confidence: 99%

See 3 more Smart Citations

Software failure prediction based on patterns of multiple-event failures

Santos¹

View full text Add to dashboard Cite

A fundamental need for software reliability engineering is to comprehend how software systems fail, which means understanding the dynamics that govern different types of failure manifestation. In this research, I present an exploratory study on multiple-event failures, which is a failure manifestation characterized by sequences of failure events, varying in terms of length, duration, and combination of failure types. This study aims to (i) improve the understanding of multiple-event failures in real software systems, investigating their occurrences, associations, and causes; (ii) propose analysis protocols that take into account multiple-event failure manifestations; (iii) take advantage of the sequential nature of this type of software failure to perform predictions. The failures analyzed in this research were observed empirically. In total, I analyzed 42,209 real software failures from 644 computers used in different workplaces. The major contributions of this study are a protocol developed to investigate the existence of patterns of failure associations; a protocol to discover patterns of failure sequences; and a prediction approach whose main concept is to calculate the probability of a certain failure event to occur within a time interval upon the occurrence of a particular pattern of preceding failures. I used three methods to tackle the prediction problem; Multinomial Logistic Regression (w/ and w/o Ridge regularization), Decision Tree, and Random Forest. These methods were chosen due to the nature of the failure data, in which the failure types must be handled as categorical variables. Initially, I performed a failure association discovery analysis which only included failures from a widely used commercial off-the-shelf Operating System (OS). As a result, I discovered 45 OS failure association patterns with 153,511 occurrences, which were composed of the same or different failure types and occurring within wellestablished time intervals, systematically. The observed associations suggest the existence of underlying mechanisms governing these failure occurrences, which motivated the improvement of the previous method by creating a protocol to discover patterns of failure sequences using flexible time thresholds and a failure prediction approach. To have a comprehensive view of how different software failures may affect each other, both methods were applied to three different samples -the first sample contained only OS failures, the second contained only User Application failures, and the third encompassed both OS and User Application failures altogether. As a result, I found 165, 480, and 640 different failure sequences with thousands of occurrences, respectively. Finally, the proposed approach was able to predict failures with good to high accuracy (86% to 93%).

show abstract

Section: Software Failures Analyzedmentioning

confidence: 99%