A Methodology for Measuring FLOSS Ecosystems

Amreen, Sadika; Bichescu, Bogdan C.; Bradley, R. S.; Dey, Tapajit; Ma, Yuxing; Mockus, Audris; Mousavi, Sara; Zaretzki, Russell

doi:10.1007/978-981-13-7099-1_1

Cited by 9 publications

(8 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These risks in the "supply chain" of OSS are increasingly recognized and quantified in the empirical software engineering literature [30], [31]. Decan, Mens, and Grosjean describe the evolution of the dependency networks of seven large OSS ecosystems, finding an increasing trend in the number of direct and indirect (sometimes called transitive) dependencies in all of them [1].…”

Section: B Vulnerabilities Of Oss Ecosystemsmentioning

confidence: 99%

Modeling Interconnected Social and Technical Risks in Open Source Software Ecosystems

Schueller¹,

Wachs²

2022

Preprint

View full text Add to dashboard Cite

Open source software ecosystems consist of thousands of interdependent libraries, which users can combine to great effect. Recent work has pointed out two kinds of risks in these systems: that technical problems like bugs and vulnerabilities can spread through dependency links, and that relatively few developers are responsible for maintaining even the most widely used libraries. However, a more holistic diagnosis of systemic risk in software ecosystem should consider how these social and technical sources of risk interact and amplify one another. Motivated by the observation that the same individuals maintain several libraries within dependency networks, we present a methodological framework to measure risk in software ecosystems as a function of both dependencies and developers. In our models, a library's chance of failure increases as its developers leave and as its upstream dependencies fail. We apply our method to data from the Rust ecosystem, highlighting several systemically important libraries that are overlooked when only considering technical dependencies. We compare potential interventions, seeking better ways to deploy limited developer resources with a view to improving overall ecosystem health and software supply chain resilience.

show abstract

Section: B Vulnerabilities Of Oss Ecosystemsmentioning

confidence: 99%

Modeling Interconnected Social and Technical Risks in Open Source Software Ecosystems

Schueller¹,

Wachs²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Further application of our approach might include: a) detecting if a developer is actually a bot by analyzing the concentration of their skill vector (similar to [49], [50]); b) checking the alignment between skill vectors of different developers for identity resolution (similar to [47]); c) analyzing the skill vectors of the developers in a project to infer the transparency of the corresponding software supply chain [51], [52], [53], [54], [55], [56].…”

Section: Lim I T a T I O N Smentioning

confidence: 99%

Representation of Developer Expertise in Open Source Software

Dey

Karnauch

Mockus

2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

Background: Accurate representation of developer expertise has always been an important research problem. While a number of studies proposed novel methods of representing expertise within individual projects, these methods are difficult to apply at an ecosystem level. However, with the focus of software development shifting from monolithic to modular, a method of representing developers' expertise in the context of the entire OSS development becomes necessary when, for example, a project tries to find new maintainers and look for developers with relevant skills. Aim: We aim to address this knowledge gap by proposing and constructing the Skill Space where each API, developer, and project is represented and postulate how the topology of this space should reflect what developers know (and projects need). Method: we use the World of Code infrastructure to extract the complete set of APIs in the files changed by open source developers and, based on that data, employ Doc2Vec embeddings for vector representations of APIs, developers, and projects. We then evaluate if these embeddings reflect the postulated topology of the Skill Space by predicting what new APIs/projects developers use/join, and whether or not their pull requests get accepted. We also check how the developers' representations in the Skill Space align with their self-reported API expertise. Result: Our results suggest that the proposed embeddings in the Skill Space appear to satisfy the postulated topology and we hope that such representations may aid in the construction of signals that increase trust (and efficiency) of open source ecosystems at large and may aid investigations of other phenomena related to developer proficiency and learning.

show abstract

“…However, a number of studies have looked at the aspects of software quality metrics, especially the quality perceived by the customers, e.g., [50,46,24,61,44]. [3] described a general way to measure Software Quality and related metrics for Open Source Software ecosystems. A notable nonacademic work involves a study of mobile app monitoring company's (Crittercism) data [23].…”

Section: Related Workmentioning

confidence: 99%

Deriving a usage-independent software quality metric

Dey¹,

Mockus²

2020

Empir Software Eng

Self Cite

View full text Add to dashboard Cite

Context: The extent of post-release use of software affects the number of faults, thus biasing quality metrics and adversely affecting associated decisions. The proprietary nature of usage data limited deeper exploration of this subject in the past. Objective: To determine how software faults and software use are related and how, based on that, an accurate quality measure can be designed. Method: Via Google Analytics we measure new users, usage intensity, usage frequency, exceptions, and release date and duration for complex proprietary mobile applications for Android and iOS. We utilize Bayesian Network and Random Forest models to explain the interrelationships and to derive the usage independent release quality measure. To increase external validity, we also investigate the interrelationship among various code complexity measures, usage (downloads), and number of issues for 520 NPM packages. We derived a usage-independent quality measure from these analyses, and applied it on 4430 popular NPM packages to construct timelines for comparing the perceived quality (number of issues) and our derived measure of quality during the lifetime of these packages. Results: We found the number of new users to be the primary factor determining the number of exceptions, and found no direct link between the intensity and frequency of software usage and software faults. Crashes increased with the power of 1.02-1.04 of new user for the Android app and power of 1.6 for the iOS app. Release quality expressed as crashes per user was independent of other usage-related predictors, thus serving as a usage independent measure of software quality. Usage also affected quality in NPM, where downloads were strongly associated with numbers of issues, even after taking the other code complexity measures into consideration. Unlike in mobile case where exceptions per user decrease over time, for 45.8% of the NPM packages the number of issues per download increase. Conclusions: We expect our result and our proposed quality measure will help gauge release quality of a software more accurately and inspire further research in this area.

show abstract

A Methodology for Measuring FLOSS Ecosystems

Cited by 9 publications

References 35 publications

Modeling Interconnected Social and Technical Risks in Open Source Software Ecosystems

Modeling Interconnected Social and Technical Risks in Open Source Software Ecosystems

Representation of Developer Expertise in Open Source Software

Deriving a usage-independent software quality metric

Contact Info

Product

Resources

About