Augmenting API documentation with insights from stack overflow

Treude, Christoph; Robillard, Martin P.

doi:10.1145/2884781.2884800

Cited by 235 publications

(125 citation statements)

References 59 publications

Supporting

Mentioning

124

Contrasting

Unclassified

Order By: Relevance

“…The data used for the analysis below is available for download and inspection in our online appendix. (8,485). As the fourth column shows, 8,137 of these tokens were identical between the two libraries, accounting for 94% of all tokens (i.e., number of identical tokens divided by the average number of tokens produced by the two libraries).…”

Section: B Findingsmentioning

confidence: 99%

“…An example of a disagreement is the word "Shape" in the sentence "class provides a way to fill a Shape" which Google's SyntaxNet tagged as a proper noun (NNP) in agreement with our manual annotation, while the other three libraries tagged it as a regular noun (NN). 8 …”

Section: A) Manual Annotation Tokenization Comparisonmentioning

confidence: 99%

“…While it is common for researchers to rely on publicly available NLP libraries, some researchers develop their own tooling for specific tasks. For example, Allamanis et al [7] developed a customized system called Haggis for mining code idioms and in our own previous work, we added customizations to the Stanford NLP library to improve the accuracy of parsing natural language text authored by software developers [8], [9]. In this work, we aim to identify how the choice of using a particular publicly available NLP library could impact the results of any research that makes use of an NLP library.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Choosing an NLP Library for Analyzing Software Documentation: A Systematic Literature Review and a Series of Experiments

Omran

Treude

2017

2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR)

Self Cite

View full text Add to dashboard Cite

Abstract-To uncover interesting and actionable information from natural language documents authored by software developers, many researchers rely on "out-of-the-box" NLP libraries. However, software artifacts written in natural language are different from other textual documents due to the technical language used. In this paper, we first analyze the state of the art through a systematic literature review in which we find that only a small minority of papers justify their choice of an NLP library. We then report on a series of experiments in which we applied four state-of-the-art NLP libraries to publicly available software artifacts from three different sources. Our results show low agreement between different libraries (only between 60% and 71% of tokens were assigned the same part-of-speech tag by all four libraries) as well as differences in accuracy depending on source: For example, spaCy achieved the best accuracy on Stack Overflow data with nearly 90% of tokens tagged correctly, while it was clearly outperformed by Google's SyntaxNet when parsing GitHub ReadMe files. Our work implies that researchers should make an informed decision about the particular NLP library they choose and that customizations to libraries might be necessary to achieve good results when analyzing software artifacts written in natural language.

show abstract

Section: B Findingsmentioning

confidence: 99%

Section: A) Manual Annotation Tokenization Comparisonmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Choosing an NLP Library for Analyzing Software Documentation: A Systematic Literature Review and a Series of Experiments

Omran

Treude

2017

2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Treude and Robillard [53] also used crowd knowledge available on Stack Overflow, as Souza et al [5] did, to automatically augment API documentation with insight sentences. They defined that insight sentences are sentences related to a particular API type and that provide insight not contained in the API documentation of that type.…”

Section: Related Work (Re)documenting Apismentioning

confidence: 99%

Redocumenting APIs with crowd knowledge: a coverage analysis based on question types

et al. 2016

View full text Add to dashboard Cite

Background: Software libraries and frameworks play an important role in software system development. The appropriate usage of their functionalities/components through their APIs, however, is a challenge for developers. Usually, API documentation, when it exists, is insufficient to assist them in their programming tasks. There are few API documentation writers for the many potential readers, resulting in the lack of explanations and examples concerning different scenarios and perspectives. The interaction of developers on the Web, on the other hand, generates content concerning APIs from different perspectives, which can be used to document APIs, also known as crowd documentation. Methods: In this paper, we present a study regarding the knowledge generated by the crowd on the Stack Overflow question-and-answer website. Our main goal is to understand how the crowd can contribute for API documentation on two programming tasks: how to implement a scenario using an API (how-to-do-it), and how to fix domain-independent bugs in an existing code where there was a misunderstanding regarding the usage of an API (debug-corrective). We classified questions available on Stack Overflow by the main concerns of askers, and we used those classified as how-to-do-it and debug-corrective to analyze the coverage of API elements on the discussions related to such questions. Our cases included the well-known and popular Swing and Android APIs. Results: Our main findings showed that the crowd provides more content for debug-corrective tasks than for how-to-do-it tasks, regardless of the API. Android API elements are more discussed by the crowd compared to Swing. Moreover, we observed that some API elements are frequently mentioned together in discussions, and that there is a strong association between API coverage on Stack Overflow and its usage in real software systems. Conclusions: Crowd documentation may not be a complete substitute for official documentation because of its partial coverage, especially for how-to-do-it tasks. However, it can still significantly enhance the existent documentation, especially for the most commonly used API elements, providing code samples and explanations on a large variety of usage nuances. Finally, taking advantage of the high coverage for debug-corrective tasks, a new kind of debugging assistant may be conceived.

show abstract

“…This amount of knowledge is unmanageable for humans and applications try to make this knowledge more accessible to humans, e. g. Treude and Robillard (2016) make natural language text about software programming more accessible through a natural language processing (NLP) application.…”

Section: Introductionmentioning

confidence: 99%

NLP Lean Programming Framework: Developing NLP Applications More Effectively

Schreiber

Kraft

Zündorf

2018

Proceedings of the 2018 Conference of the North American Chapter Of the Association for Computational Linguistics: De

View full text Add to dashboard Cite

This paper presents NLP Lean Programming framework (NLPf), a new framework for creating custom natural language processing (NLP) models and pipelines by utilizing common software development build systems. This approach allows developers to train and integrate domain-specific NLP pipelines into their applications seamlessly. Additionally, NLPf provides an annotation tool which improves the annotation process significantly by providing a well-designed GUI and sophisticated way of using input devices. Due to NLPf's properties developers and domain experts are able to build domain-specific NLP applications more efficiently. NLPf is Opensource software and available at https:// gitlab.com/schrieveslaach/NLPf.

show abstract

Augmenting API documentation with insights from stack overflow

Cited by 235 publications

References 59 publications

Choosing an NLP Library for Analyzing Software Documentation: A Systematic Literature Review and a Series of Experiments

Choosing an NLP Library for Analyzing Software Documentation: A Systematic Literature Review and a Series of Experiments

Redocumenting APIs with crowd knowledge: a coverage analysis based on question types

NLP Lean Programming Framework: Developing NLP Applications More Effectively

Contact Info

Product

Resources

About