Software effort estimation accuracy prediction of machine learning techniques: A systematic performance evaluation

Mahmood, Yasir; Kama, Nazri; Azmi, Azri; Khan, Ahmad Salman; Ali, Mazlan

doi:10.1002/spe.3009

Cited by 66 publications

(36 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to evaluate the classification effects of the trained classification model, some evaluation indexes are needed to measure it. The author mainly selected the classification accuracy [ 44 ], precision [ 45 ], recall [ 46 ], and F value [ 47 ] to evaluate the classification effects achieved by the classification model on the data set.…”

Section: Methodsmentioning

confidence: 99%

Recognition Method for Broiler Sound Signals Based on Multi-Domain Sound Features and Classification Model

Tao

Wang

Sun

et al. 2022

Sensors

View full text Add to dashboard Cite

In view of the limited number of extracted sound features, the lack of in-depth analysis of applicable sound features, and the lack of in-depth study of the selection basis and optimization process of classification models in the existing broiler sound classification or recognition research, the author proposes a recognition method for broiler sound signals based on multi-domain sound features and classification models. The implementation process is divided into the training stage and the testing stage. In the training stage, the experimental area is built, and multiple segments of broiler sound signals are collected and filtered. Through sub-frame processing and endpoint detection, the combinations of start frames and end frames of multiple sound types in broiler sound signals are obtained. A total of sixty sound features from four aspects of time domain, frequency domain, Mel-Frequency Cepstral Coefficients (MFCC), and sparse representation are extracted from each frame signal to form multiple feature vectors. These feature vectors are labeled manually to build the data set. The min-max standardization method is used to process the data set, and the random forest is used to calculate the importance of sound features. Then, thirty sound features that contribute more to the classification effect of the classification model are retained. On this basis, the classification models based on seven classification algorithms are trained, the best-performing classification model based on k-Nearest Neighbor (kNN) is obtained, and its inherent parameters are optimized. Then, the optimal classification model is obtained. The test results show that the average classification accuracy achieved by the decision-tree-based classifier (abbreviated as DT classifier) on the data set before and after min–max standardization processing is improved by 0.6%, the average classification accuracy achieved by the DT classifier on the data set before and after feature selection is improved by 3.1%, the average classification accuracy achieved by the kNN-based classification model before and after parameter optimization is improved by 1.2%, and the highest classification accuracy is 94.16%. In the testing stage, for a segment of the broiler sound signal collected in the broiler captivity area, the combinations of the start frames and end frames of multiple sound types in the broiler sound signal are obtained through signal filtering, sub-frame processing, endpoint detection, and other steps. Thirty sound features are extracted from each frame signal to form the data set to be predicted. The optimal classification model is used to predict the labels of each piece of data in the data set to be predicted. By performing majority voting processing on the predicted labels of the data combination corresponding to each sound type, the common labels are obtained; that is, the predicted types are obtained. On this basis, the definition of recognition accuracy for broiler sound signals is proposed. The test results show that the classification accuracy achieved by the optimal classification model on the data set to be predicted is 93.57%, and the recognition accuracy achieved on the multiple segments of the broiler sound signals is 99.12%.

show abstract

Section: Methodsmentioning

confidence: 99%

Recognition Method for Broiler Sound Signals Based on Multi-Domain Sound Features and Classification Model

Tao

Wang

Sun

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Neural network-based models are a relatively new addition to the arsenal of machine learning techniques [6]. They are utilized in software effort estimating because they learn faster and more efficiently with more accurate results [23], [32].…”

Section: Related Workmentioning

confidence: 99%

Comparing Multiple Linear Regression, Deep Learning and Multiple Perceptron for Functional Points Estimation

et al. 2022

View full text Add to dashboard Cite

This study compares the performance of Pytorch-based Deep Learning, Multiple Perceptron Neural Networks with Multiple Linear Regression in terms of software effort estimations based on function point analysis. This study investigates Adjusted Function Points, Function Point Categories, Industry Sector, and Relative Size. The ISBSG dataset (version 2020/R1) is used as the historical dataset. The effort estimation performance is compared among multiple models by evaluating a prediction level of 0.30 and standardized accuracy. According to the findings, the Multiple Perceptron Neural Network based on Adjusted Function Points combined with Industry Sector predictors yielded 53% and 61% in terms of standardized accuracy and a prediction level of 0.30, respectively. The findings of Pytorch-based Deep Learning are similar to Multiple Perceptron Neural Networks, with even better results than that, with standardized accuracy and a prediction level of 0.30, 72% and 72%, respectively. The results reveal that both the Pytorch-based Deep Learning and Multiple Perceptron model outperformed Multiple Linear Regression and baseline models using the experimental dataset. Furthermore, in the studied dataset, Adjusted Function Points may not contribute to higher accuracy than Function Point Categories.

show abstract

“…The final results suggested that the ensemble model improved the accuracy of estimation. Mahmood et al 41 investigated the machine learning techniques which were implemented in the construction of ensemble effort estimation techniques. Their results suggested that the machine learning ensemble techniques perform better than solo techniques.…”

Section: Literature Reviewmentioning

confidence: 99%

An evolutionary ensemble analogy‐based software effort estimation

Shahpar

Bardsiri

2021

Softw Pract Exp

View full text Add to dashboard Cite

Analogy-based estimation (ABE) is the most commonly applied method for estimating using software. Although there are several analogy-based estimation techniques, there is no consistent conclusion of which technique is the best in all circumstances. Therefore, this article presents an evolutionary ensemble ABE (EEABE) for software cost estimation. Ensemble effort estimation (EEE) models forecast software development endeavors through using multiple estimation methods. In this article, EEABE combines GA as an evolutionary algorithm with six ABE models. The proposed method has been evaluated on four well-known datasets comprising Maxwell, Albrecht, Kemerer, and Desharnais, by the k-fold cross-validation technique and based on the performance criteria of MRE, MMRE, MDMRE, BMMRE, and PRED(0.25).The simulation results demonstrate that the use of EEABE increases the accuracy of estimation and reduces the cost. Besides, EEABE is a flexible and adaptable model with any type of dataset and software development project, and it has been optimized with the possibility of using various ABE techniques.

show abstract

Software effort estimation accuracy prediction of machine learning techniques: A systematic performance evaluation

Cited by 66 publications

References 42 publications

Recognition Method for Broiler Sound Signals Based on Multi-Domain Sound Features and Classification Model

Recognition Method for Broiler Sound Signals Based on Multi-Domain Sound Features and Classification Model

Comparing Multiple Linear Regression, Deep Learning and Multiple Perceptron for Functional Points Estimation

An evolutionary ensemble analogy‐based software effort estimation

Contact Info

Product

Resources

About