An Exploratory Study on Automatic Architectural Change Analysis Using Natural Language Processing Techniques

Mondal, Amit Kumar; Roy, Banani; Schneider, Kevin A.

doi:10.1109/scam.2019.00016

Cited by 6 publications

(22 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mondal et al [22] explored the relation between free-form natural language text (e.g. developers' communication and commit messages) and automated architectural change analysis.…”

Section: Related Workmentioning

confidence: 99%

Using Natural Language Processing to Extract Information from Unstructured code-change version control data: lessons learned

Ronchieri¹,

Canaparo²,

Yang³

2021

Proceedings of International Symposium on Grids &Amp; Clouds 2021 — PoS(ISGC2021)

View full text Add to dashboard Cite

Context: Natural Language Processing (NLP) is a branch of artificial intelligence that extracts information from language. In the field of software engineering, NLP has been employed to extract key information from free-form text, to generate models from the analysis of text or to categorize code changes according to their commit messages. In literature, most of the approaches NLP-based focused on the impact of code changes on program execution or software architecture. Objective: In this study, we have applied NLP to code-change data to identify patterns of software code modifications and used Machine Learning techniques to build a model that determines how software has evolved over time and identifies area of code that presents problems. Method: Considering that software projects use version control systems, such as github, to manage their code, we have collected software information by using git commands. These data contain different unstructured information about the various files in a project. Each modification entry includes a message that explains the reasons for the change. According to the content of the message, it is possible to identify key terms that can be used during the classification of the entries. Results: In this study, we have considered the change history of software available on github to the High Energy Physics community. With the use of NLP techniques we have cleaned the messages and extracted some key terms to categorize both software problems and some other changes performed by developers, like the addition of a third party dependency or a script that starts a given service. We have built a code change dictionary combining the terms already in existing literature with the ones gathered directly from the software and its github repository. Finally, we have applied some Machine Learning (ML) techniques to determine any connection between code changes and software problems: we have removed redundant entries to avoid any bias in the outcomes of the ML techniques. Conclusion: We show in detail our approach adopted to construct historical code change datasets of categorized commit messages by following a multi-label classification methodology. Our model performance seems promising in terms of accuracy, precision, recall and F1-score.

show abstract

“…Mondal et al [22] explored the relation between free-form natural language text (e.g. developers' communication and commit messages) and automated architectural change analysis.…”

Section: Related Workmentioning

confidence: 99%

Using Natural Language Processing to Extract Information from Unstructured code-change version control data: lessons learned

Ronchieri¹,

Canaparo²,

Yang³

2021

Proceedings of International Symposium on Grids &Amp; Clouds 2021 — PoS(ISGC2021)

View full text Add to dashboard Cite

show abstract

“…Moreover, appropriate architectural formulation is becoming more critical to circumvent software bloat, scalability, and security backdoors [11]. However, elements of architecture can be changed [20] continuously as code components of a software system changes to support continuous development and maintenance [29] such as adding new features, restructuring the design models, and fixing flaws. Architectural change can affect many aspects of a software system and, for this, change analysis is a crucial task.…”

Section: Introductionmentioning

confidence: 99%

“…Development team can group architectural changes leveraging change classification process based on the cause of the change, type of change, location of the change, the size of the code modification, and impact of change [40], [15]. For example, four major causes of architectural changes have been defined explicitly in the literature [40], [8], [29]: (i) perfective DOI reference number: 10.18293/SEKE2021-060 -adjusting new behaviour, (ii) preventive -prevent bad design, (iii) corrective -correct discovered problems, and (iv) adaptive -adapting to new platform.…”

Section: Introductionmentioning

confidence: 99%

“…However, while architectural change can be identified from source code change, identifying the design decision, reason, and categories of changes requires analyzing the development team's intention. The intention can be extracted from textual description of the developer's tasks and discussions [6], [29]. Literature has focused on classifying typical software changes, architectural design concerns and design solutions [41], [28], [14].…”

Section: Introductionmentioning

confidence: 99%

“…Literature has focused on classifying typical software changes, architectural design concerns and design solutions [41], [28], [14]. Yet, supporting architectural change classification is still in its infancy [29], [11], perhaps, due to lack of benchmark data and requirements of laborious human analysis. Nevertheless, a few of the studies explored for both manual [8], [32] and semi-automated [29] techniques for classifying architectural changes.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

ArchiNet: A Concept-token based Approach for Determining Architectural Change Categories

Kumar¹

2021

Proceedings of the 33rd International Conference on Software Engineering and Knowledge Engineering

View full text Add to dashboard Cite

Causes of software architectural change are classified as perfective, preventive, corrective, and adaptive. Change classification is used to promote common approaches for addressing similar changes, produce appropriate design documentation for a release, construct a developer's profile, form a balanced team, support code review, etc. However, automated architectural change classification techniques are in their infancy, perhaps due to the lack of a benchmark dataset and the need for extensive human involvement. To address these shortcomings, we present a benchmark dataset and a text classifier for determining the architectural change rationale from commit descriptions. First, we explored source code properties for change classification independent of project activity descriptions and found poor outcomes. Next, through extensive analysis, we identified the challenges of classifying architectural change from text and proposed a new classifier that uses concept tokens derived from the concept analysis of change samples. We also studied the sensitivity of change classification of various types of tokens present in commit messages. The experimental outcomes employing 10fold and cross-project validation techniques with five popular open-source systems show that the F1 score of our proposed classifier is around 70%. The precision and recall are mostly consistent among all categories of change and more promising than competing methods for text classification.

show abstract

A survey of software architectural change detection and categorization techniques

Mondal

Schneider²,

Roy³

et al. 2022

Journal of Systems and Software

View full text Add to dashboard Cite

An Exploratory Study on Automatic Architectural Change Analysis Using Natural Language Processing Techniques

Cited by 6 publications

References 32 publications

Using Natural Language Processing to Extract Information from Unstructured code-change version control data: lessons learned

Using Natural Language Processing to Extract Information from Unstructured code-change version control data: lessons learned

ArchiNet: A Concept-token based Approach for Determining Architectural Change Categories

A survey of software architectural change detection and categorization techniques

Contact Info

Product

Resources

About