Investigating the Significance of Bellwether Effect to Improve Software Effort Estimation

Mensah, Solomon; Keung, Jacky; MacDonell, Stephen G.; Bosu, Michael Franklin; Bennin, Kwabena Ebo

doi:10.1109/qrs.2017.44

Cited by 11 publications

(56 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent studies [11], [12], [18] support the theory of Lokan and Mendes that the use of a moving window as the training set improves the prediction accuracy of SEP models. This moving window approach has recently been supplemented with weighting functions, which have been shown to further improve the accuracy of SEP models.…”

Section: Introductionmentioning

confidence: 75%

“…This paper represents a substantive extension of one of our prior research efforts. In this section, we describe the original study [11] and its results as well as the goals and distinctive elements of the current study, following the replication guidelines introduced by Carver [22].…”

Section: Background Of Studymentioning

confidence: 99%

“…The main goal of our original study [11] was to investigate the existence of exemplary projects (i.e., the Bellwether effect) in SEP, focusing on chronological projects. We investigated whether the use of exemplary and recently completed projects in a BMW could improve prediction accuracy as compared to using the entire collection of historical projects over time (GP).…”

Section: A Original Studymentioning

confidence: 99%

“…Using theoretical and empirical analysis, we argue that MCMC modeling based on a novel step-by-step method can be used to sample relevant and recently completed exemplary projects to achieve relatively higher prediction accuracy than would be the case otherwise. We refer to this subset of exemplary and recently completed projects as the BMW [11].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Investigating the Significance of the Bellwether Effect to Improve Software Effort Prediction: Further Empirical Study

et al. 2018

Self Cite

View full text Add to dashboard Cite

Context: In addressing how best to estimate how much effort is required to develop software, a recent study found that using exemplary and recently completed projects [forming Bellwether moving windows (BMW)] in software effort prediction (SEP) models leads to relatively improved accuracy. More studies need to be conducted to determine whether the BMW yields improved accuracy in general, since different sizing and aging parameters of the BMW are known to affect accuracy. Objective: To investigate the existence of exemplary projects (Bellwethers) with defined window size and age parameters, and whether their use in SEP improves prediction accuracy. Method: We empirically investigate the moving window assumption based on the theory that the prediction outcome of a future event depends on the outcomes of prior events. Sampling of Bellwethers was undertaken using three introduced Bellwether methods (SSPM, SysSam, and RandSam). The ergodic Markov chain was used to determine the stationarity of the Bell-wethers. Results: Empirical results show that 1) Bellwethers exist in SEP and 2) the BMW has an approximate size of 50 to 80 exemplary projects that should not be more than 2 years old relative to the new projects to be estimated. Conclusion: The study's results add further weight to the recommended use of Bellwethers for improved prediction accuracy in SEP.

show abstract

Section: Introductionmentioning

confidence: 75%

Section: Background Of Studymentioning

confidence: 99%

Section: A Original Studymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Investigating the Significance of the Bellwether Effect to Improve Software Effort Prediction: Further Empirical Study

et al. 2018

Self Cite

View full text Add to dashboard Cite

show abstract

“…We benchmarked the prediction results from the ElasticNet model to a complex and robust prediction model, namely a Deep learning model which yielded better prediction accuracy in a previous study [8]. We constructed a DNN which makes use of multiple hidden layers and an output layer with their respective neurons to automatically learn from a set of project cases and gives the resulting prediction for the target (in our case, the software effort of new projects).…”

Section: Effort Estimation Modelsmentioning

confidence: 99%

Revisiting the Conclusion Instability Issue in Software Effort Estimation (S)

Bosu

Mensah

Bennin

et al. 2018

International Conferences on Software Engineering and Knowledge Engineering

Self Cite

View full text Add to dashboard Cite

Conclusion instability is the absence of observing the same effect under varying experimental conditions. Deep Neural Network (DNN) and ElasticNet software effort estimation (SEE) models were applied to two SEE datasets with the view of resolving the conclusion instability issue and assessing the suitability of ElasticNet as a viable SEE benchmark model. Results were mixed as both model types attain conclusion stability for the Kitchenham dataset whilst conclusion instability existed in the Desharnais dataset. ElasticNet was outperformed by DNN and as such it is not recommended to be used as a SEE benchmark model.

show abstract

A classification scheme to improve conclusion instability using Bellwether moving windows

Mensah

Kudjo

2022

J Software Evolu Process

Self Cite

View full text Add to dashboard Cite

Context The use of a subset of recently completed and exemplary data, namely, Bellwether moving window (BMW) has proven successful to result in improved accuracy in software effort estimation (SEE). These outcomes were achieved based on the theory that estimation outcome of a future event depends on previous events. Thus, the existence of a BMW yield improved prediction accuracy for new project estimation. However, the conclusion instability problem across learners still threatens the reliability of SEE for new projects. Such instability concerns are attributed to the data subset considered for the training and validation needs of learners. Objective To investigate whether the use of BMWs together with an effort classification scheme can minimize the conclusion instability problem across learners. Method We apply a Bellwether method comprising of three operators, namely, SORT+CLUSTER, GENERATE_TPM, and APPLY to sample the BMW from a pool of chronological projects from the Maxwell and International Software Benchmarking Standards Group (ISBSG) datasets. The sampled BMW is benchmarked against the entire collection of preprocessed projects, namely, growing portfolio to evaluate prediction and classification accuracy across a set of learners–ElasticNet regression, deep neural networks, and automatically transformed linear model. Results (1) BMW exists in the studied projects and (2) training the learners with a BMW of average window size 28.5%–75.5% of the growing portfolio (not older than 3 years) relatively minimizes the conclusion instability of prediction results. Conclusion When BMWs are available, we recommend their use for estimating the effort for a new project to minimize the conclusion instability problem.

show abstract

Investigating the Significance of Bellwether Effect to Improve Software Effort Estimation

Cited by 11 publications

References 39 publications

Investigating the Significance of the Bellwether Effect to Improve Software Effort Prediction: Further Empirical Study

Investigating the Significance of the Bellwether Effect to Improve Software Effort Prediction: Further Empirical Study

Revisiting the Conclusion Instability Issue in Software Effort Estimation (S)

A classification scheme to improve conclusion instability using Bellwether moving windows

Contact Info

Product

Resources

About