9.6 Million Links in Source Code Comments: Purpose, Evolution, and Decay

Abstract

Links are an essential feature of the World Wide Web, and source code repositories are no exception. However, despite their many undisputed benefits, links can suffer from decay, insufficient versioning, and lack of bidirectional traceability. In this paper, we investigate the role of links contained in source code comments from these perspectives. We conducted a large-scale study of around 9.6 million links to establish their prevalence, and we used a mixed-methods approach to identify the links' targets, purposes, decay, and evolutionary aspects. We found that links are prevalent in source code repositories, that licenses, software homepages, and specifications are common types of link targets, and that links are often included to provide metadata or attribution. Links are rarely updated, but many link targets evolve. Almost 10% of the links included in source code comments are dead. We then submitted a batch of link-fixing pull requests to open source software repositories, resulting in most of our fixes being merged successfully. Our findings indicate that links in source code comments can indeed be fragile, and our work opens up avenues for future work to address these problems.12 https://stackoverflow.com/q/312443 13 We used LWP::UserAgent and LWP::RobotUA.

Docable: evaluating the executability of software tutorials

“…In these empirical studies, a common factor for low rates of executability include missing configuration information, such as dependencies, and software decay due to reliance on older versions (e.g., the code only works with older versions of APIs). Our observation of documentation rot is closely related to other forms of software decay, such as unavailable urls in source code [9]. In summary, executability is a useful property for understanding the quality and replicability of software artifacts, including those found in software tutorials.…”

Contextual Documentation Referencing on Stack Overflow

“…Outside of Stack Overflow, Hata et al [51] studied the role of links contained in source code comments in terms of prevalence, link targets, purposes, decay, and evolutionary aspects. Similar to our findings, Hata et al also report that link can be fragile since they are vulnerable to link rot and link targets change frequently.…”

Research Artifact: The Potential of Meta-Maintenance on GitHub

“…a) Repository preparation: To pursue the feasibility of meta-maintenance, we collect a large amount of software development repositories that have been actively developed. We follow the same procedure as in a previous study [46] to identify candidate repositories. We target software development repositories on GitHub written in seven common programming languages, that is, C, C++, Java, JavaScript, Python, PHP, and Ruby.…”

“…These languages have been ranked consistently in the top 10 languages on GitHub from 2008 to 2018 (based on the number of repositories from 2008 to 2015 [47], the number of pull requests from 2014 to 2018 [48], and top languages from 2014 to 2018 in the official report [49]). Using the GHTorrent dataset 1 [50], we identify active repositories for the seven languages with the following criteria [46]: (i) having more than 500 commits in their entire history (the same threshold used in previous work [51]), and (ii) having at least 100 commits in the most active two years to remove long-term less active repositories and short-term projects that have not been maintained for long. We determine repositories' languages based on the GHTorrent information.…”