Actionable Analytics for Strategic Maintenance of Critical Software: An Industry Experience Report

Port, Dan; Taber, Bill

doi:10.1109/ms.2017.4541055

Cited by 4 publications

(7 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Díaz-Ley et al [25] studied a measurement program (synonymous with metrics programs) targeted towards small-andmedium-size enterprises (SMEs), and found that use of metrics can help practitioners define measurement goals that are well aligned with their organization's maturity. Port and Taber [16] provide empirical evidence of MP use and their actionability in a large industrial context. With the help of metrics and analytics programs, the authors illustrate the supporting role an MP played in strategic maintenance of a critical system at NASA's Jet Propulsion Laboratory.…”

Section: Related Workmentioning

confidence: 99%

“…MPs also enable objective evaluation of software development processes, contributing to continuous improvement and learning in industrial software development context [9]. Current literature reports on successful adoption of MPs [10][11][12][13] and characteristics that make metrics actionable [14][15][16] in Agile software development (ASD). However, empirical investigation into actual MP use for software development in industrial context, and the resulting benefits, remains scarce.…”

Section: Introductionmentioning

confidence: 99%

“…However, the limited study scope prevents discussion of successful MP adoption translating into successful use, especially in ASD. Similarly, studies like [16] and [17] provide evidence of MPs influencing actions in organizations using ASD, but do not detail the motivation of such a use and consequent benefits, if any. Studies like [11][12][13]16] and [17] provide a curtailed view of MPs in practice, falling short of discussing MP's use in industrial context and how practitioners benefit from them.…”

Section: Introductionmentioning

confidence: 99%

“…Similarly, studies like [16] and [17] provide evidence of MPs influencing actions in organizations using ASD, but do not detail the motivation of such a use and consequent benefits, if any. Studies like [11][12][13]16] and [17] provide a curtailed view of MPs in practice, falling short of discussing MP's use in industrial context and how practitioners benefit from them. We argue that providing such details, supported by clear empirical evidence, can act as a strong motivator for practitioners to adopt a favorable view towards MPs.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

An Empirical Investigation into Industrial Use of Software Metrics Programs

Ram

Rodríguez

Oivo

et al. 2020

Product-Focused Software Process Improvement

View full text Add to dashboard Cite

Practitioners adopt software metrics programs to support their software development from the perspective of either overall quality, performance, or both. Current literature details and justifies the role of a metrics program in a software organization's software development, but empirical evidence to demonstrate its actual use and concomitant benefits remains scarce. In the context of an EU H2020 Project, we conducted a multiple case study to investigate how two software-intensive Agile companies utilized a metrics program in their software development. We invited practitioners from the two case companies to report on the actual use of the metrics program, the underlying rationale, and any benefits they may have witnessed. We also collected and analyzed metrics data from multiple use cases to explain the reported use of the metrics. The analysis revealed improvements like better code review practices and formalization of quality requirements management, either as a direct consequence or as a byproduct of the use of the metrics. The contrasting contexts like company size, project characteristics, and general perspective towards metrics programs could explain why one company viewed the metrics as a trigger for their reported improvements, while the other company saw metrics as the main driver for their improvements. Empirical evidence from our study should help practitioners adopt a more favorable view towards metrics programs, who were otherwise reluctant due to lack of evidence of their utility and benefits in industrial context.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Empirical Investigation into Industrial Use of Software Metrics Programs

Ram

Rodríguez

Oivo

et al. 2020

Product-Focused Software Process Improvement

View full text Add to dashboard Cite

show abstract

“…Port and Taber [16] present examples of metrics and analytics program used to support strategic maintenance of a critical system at NASA's Jet Propulsion Laboratory. The authors claim that these metrics are actionable, as they have been validated by their proven utility in maintaining the critical systems.…”

Section: Related Workmentioning

confidence: 99%

Actionable Software Metrics

Ram

Rodríguez

Oivo

et al. 2020

Proceedings of the Evaluation and Assessment in Software Engineering

View full text Add to dashboard Cite

Modeling Performance of Microservices Systems with Growth Theory

Camilli

Russo

2022

Empir Software Eng

View full text Add to dashboard Cite

Context The microservices architectural style is gaining momentum in the IT industry. This style does not guarantee that a target system can continuously meet acceptable performance levels. The ability to study the violations of performance requirements and eventually predict them would help practitioners to tune techniques like dynamic load balancing or horizontal scaling to achieve the resilience property. Objective The goal of this work is to study the violations of performance requirements of microservices through time series analysis and provide practical instruments that can detect resilient and non-resilient microservices and possibly predict their performance behavior. Method We introduce a new method based on growth theory to model the occurrences of violations of performance requirements as a stochastic process. We applied our method to an in-vitro e-commerce benchmark and an in-production real-world telecommunication system. We interpreted the resulting growth models to characterize the microservices in terms of their transient performance behavior. Results Our empirical evaluation shows that, in most of the cases, the non-linear S-shaped growth models capture the occurrences of performance violations of resilient microservices with high accuracy. The bounded nature associated with this models tell that the performance degradation is limited and thus the microservice is able to come back to an acceptable performance level even under changes in the nominal number of concurrent users. We also detect cases where linear models represent a better description. These microservices are not resilient and exhibit constant growth and unbounded performance violations over time. The application of our methodology to a real in-production system identified additional resilience profiles that were not observed in the in-vitro experiments. These profiles show the ability of services to react differently to the same solicitation. We found that when a service is resilient it can either decrease the rate of the violations occurrences in a continuous manner or with repeated attempts (periodical or not). Conclusions We showed that growth theory can be successfully applied to study the occurences of performance violations of in-vitro and in-production real-world systems. Furthermore, the cost of our model calibration heuristics, based on the mathematical expression of the selected non-linear growth models, is limited. We discussed how the resulting models can shed some light on the trend of performance violations and help engineers to spot problematic microservice operations that exhibit performance issues. Thus, meaningful insights from the application of growth theory have been derived to characterize the behavior of (non) resilient microservices operations.

show abstract

Actionable Analytics for Strategic Maintenance of Critical Software: An Industry Experience Report

Cited by 4 publications

References 2 publications

An Empirical Investigation into Industrial Use of Software Metrics Programs

An Empirical Investigation into Industrial Use of Software Metrics Programs

Actionable Software Metrics

Modeling Performance of Microservices Systems with Growth Theory

Contact Info

Product

Resources

About