Natural language processing in mining unstructured data from software repositories: a review

Abstract

With the increasing popularity of open-source platforms, software data is easily available from various open-source tools like GitHub, CVS, SVN, etc. More than 80 percent of the data present in them is unstructured. Mining data from these repositories helps project managers, developers and businesses, in getting interesting insights. Most of the software artefacts present in these repositories are in the natural language form, which makes natural language processing (NLP) an important part of mining to get the useful results. The paper reviews the application of NLP techniques in the field of Mining Software Repositories (MSR). The paper mainly focuses on sentiment analysis, summarization, traceability, norms mining and mobile analytics. The paper presents the major NLP works performed in this area by surveying the research papers from 2000 to 2018. The paper firstly describes the major artefacts present in the software repositories where the NLP techniques have been applied. Next, the paper presents some popular open-source NLP tools that have been used to perform NLP tasks. Later the paper discusses, in brief, the research state of NLP in MSR field. The paper also lists down the various challenges along with the pointers for future work in this field of research and finally the conclusion.

Applications of Natural Language Processing in Software Traceability: A Systematic Mapping Study

“…A more recent review was done, broadly focusing on adopting NLP to mine unstructured data in software repositories (Gupta and Gupta, 2019). The review was done by looking into general applications of mining repositories, with a sub-focus on traceability efforts.…”

Fine-tuning Pretrained Transformers for Sentiment Analysis on Twitter Data

“…These are neural networks that have been trained on large amounts of text data. They can be tuned to perform well on specific tasks, such as sentiment analysis [3], [4].…”