A BERT-based transfer learning approach to text classification on software requirements specifications

Abstract

In a software development life cycle, software requirements specifications (SRS) written in an incomprehensible language might hinder the success of the project in later stages. In such cases, the subjective and ambiguous nature of the natural languages can be considered as a cause for the failure of the final product. Redundancy and/or controversial information in the SRS documents might also result in additional costs and time loss, reducing the overall efficiency of the project. With the recent advances in machine learning, there is an increased effort to develop automated solutions for a seamless SRS design. However, most vanilla machine learning approaches ignore the semantics of the software artifacts or integrating domain-specific knowledge into the underlying natural language processing tasks, and therefore tend to generate inaccurate results. With such concerns in mind, we consider a transfer learning approach in our study, which is based on an existing pre-trained language model called DistilBERT. We specifically examine the DistilBERT's ability in multi-class text classification on SRS data using various finetuning methods, and compare its performance with other deep learning methods such as LSTM and BiLSTM. We test the performance of these models using two datasets: DOORS Next Generation dataset and PROMISE-NFR dataset. Our numerical results demonstrate that DistilBERT perform well for various text classification tasks over the SRS datasets and shows significant promise to be used for automating the software development processes.

Automatic Requirements Classification Based on Graph Attention Network

“…Hey et al [25] proposed NoRBERT based on the pre-trained model BERT, which achieved excellent results. In addition, Kici et al [26] proposed distillation BERT, which is superior to LSTM and BiLSTM in requirements classification tasks, through experimental comparison.…”

“…BERT is a pre-trained mask language model. Previous studies have demonstrated that BERT can significantly improve classification performance [25,26], far exceeding the performance of traditional feature-extraction methods.…”

The Use of NLP-Based Text Representation Techniques to Support Requirement Engineering Tasks: A Systematic Mapping Review

“…Word Embedding Related Papers word2vec [43], [50], [58], [59], [67], [76], [101], [104], [106], [139] BERT [11], [47], [56], [64], [84], [91], [98] Embedding Layer [39], [51], [112]…”

“…Aggregation-based [11], [47], [56], [58], [59], [64], [66], [76], [84], [91], [98], [101], [104], [128] RNN-based [43], [51], [54], [67], [112], [137] CNN-based [39], [50] TABLE 3. the used statement embedding techniques with their related paper…”

The Use of NLP-Based Text Representation Techniques to Support Requirement Engineering Tasks: A Systematic Mapping Review

“…We recognized 21 papers proposing solutions to various requirements classification tasks. Most of them focused on Functional/Non-Functional classification tasks [39,40,41,10,11,42,43,44,45,46,47,48,49], while the remaining focused on other classification tasks: security/Not security [50,51,52], topic-based classification [53], and classification based on requirements importance level [54].…”