Exploiting Natural Language Structures in Software Informal Documentation

Abstract

Communication means, such as issue trackers, mailing lists, Q&A forums, and app reviews, are premier means of collaboration among developers, and between developers and end-users. Analyzing such sources of information is crucial to build recommenders for developers, for example suggesting experts, re-documenting source code, or transforming user feedback in maintenance and evolution strategies for developers. To ease this analysis, in previous work we proposed DECA (Development Emails Content Analyzer), a tool based on Natural Language Parsing that classifies with high precision development emails' fragments according to their purpose. However, DECA has to be trained through a manual tagging of relevant patterns, which is often effort-intensive, error-prone and requires specific expertise in natural language parsing. In this paper, we first show, with a study involving Master's and Ph.D. students, the extent to which producing rules for identifying such patterns requires effort, depending on the nature and complexity of patterns. Then, we propose an approach, named NEON (Nlp-based softwarE dOcumentation aNalyzer), that automatically mines such rules, minimizing the manual effort. We assess the performances of NEON in the analysis and classification of mobile app reviews, developers discussions, and issues. NEON simplifies the patterns' identification and rules' definition processes, allowing a savings of more than 70% of the time otherwise spent on performing such activities manually. Results also show that NEON-generated rules are close to the manually identified ones, achieving comparable recall.

An NLP-based Tool for Software Artifacts Analysis

“…The Parser (i) preprocesses the text through sentence splitting and tokenization, and (ii) generates the semantic graph 1 of each sentence present in the input text. Such semantic graphs are then provided to the PathsFinder module, which implements the approach presented in our previous work [18]. More specifically, given two generic semantic graphs, G 1 and G 2 , that share a common grammatical structure, the PathsFinder (i) selects the nodes of the verb and noun types from both graphs, (ii) identifies the pairs of similar nodes In particular, a rule able to recognize a grammatical structure in a generic sentence is defined through the XML grammar illustrated in Figure 2.…”

“…However, such approaches treat unstructured text as a bag of word (or, in the best case, infer latent topics/concepts from them). This makes them ineffective when a deeper level of detail in the text analysis and interpretation is needed [16]- [18].…”

“…The main challenge of leveraging approaches based on NL parsing techniques is that they require the manual definition of sets of NL rules [15], [16], [25] to recognize natural language patterns. This manual task has proven to be effort-intensive and error-prone, since it requires specific domain-knowledge in natural language parsing [18]. For this reason, recent research [19] attempted to automate and generalize intention mining by experimenting with deep learning-based methods.…”

“…To reduce the manual work required for manually identifying patterns in textual documents, in this paper we present a tool named NEON (Nlp-based softwarE dOcumentation aNalyzer), able to automatically mine recurring patterns from software informal documents. NEON, by implementing an approach that we presented in previous work [18], inspects natural language sentences received as input and identifies the recurrent grammatical structures appearing in them. NEON is able to output a list of syntactical rules and each rule has the purpose of detecting a specific natural language pattern.…”

“…NEON is able to output a list of syntactical rules and each rule has the purpose of detecting a specific natural language pattern. In previous work [18], we showed that NEON allows for saving more than 70% of time otherwise spent in the manual identification and definition of NL rules.…”