Preliminary guidelines for empirical research in software engineering

Kitchenham, Barbara; Pfleeger, Shari Lawrence; Pickard, Lesley M.; Jones, Peter W.; Hoaglin, David C.; Emam, Khaled El; Rosenberg, Jarrett

doi:10.1109/tse.2002.1027796

Cited by 1,171 publications

(778 citation statements)

References 43 publications

Supporting

Mentioning

746

Contrasting

Unclassified

Order By: Relevance

“…Our plan was to conduct a controlled experiment [5,4]. We managed to satisfy many requirements for a controlled experiment as: randomization of participants, and testing specimens, and the set up of a controlled environment, the detailed planning of the experimentation process, and others.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Comparing OpenBRR, QSOS, and OMM Assessment Models

Petrinja

Sillitti

Succi

2010

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

Abstract. The objective of this study was to investigate the quality and usability of three Free/Libre Open Source Software assessment models: the Open Business Readiness Rating (OpenBRR), the Qualification and Selection of Open Source software (QSOS), and the QualiPSo OpenSource Maturity Model (OMM). The study identified the positive and negative aspects of each of them. The models were used to assess two Free/Libre Open Source Software projects: Firefox and Chrome (Chromium). The study is based on a set of controlled experiments in which the participants performed the assessment using only one model each. The model used and the Free/Libre Open Source Software project assessed were randomly assigned to the participants. The experiment was conducted in a controlled environment with defined tasks to be performed in a given time interval. The results revealed that the three models provided comparable assessments for the two assessed projects. The main conclusion was that all the three models contain some questions and proposed answers that are not clear to the assessors, therefore should be rewritten or explained better. The critical aspects of each model were: Functionality and Quality for OpenBRR; Adoption, Administration/Monitoring, Copyright owners, and Browser for QSOS ; and Quality of the Test Plan, and the Technical Environment for OMM. Participants perceived the quality and usability of the three models of comparable level.

show abstract

Section: Methodsmentioning

confidence: 99%

“…We planned to conduct a well structured and executed experimentation process according to the operational definition of the experimentation process stated in one of the most frequently cited papers about this subject [5]:…”

Section: Methodsmentioning

confidence: 99%

Comparing OpenBRR, QSOS, and OMM Assessment Models

Petrinja

Sillitti

Succi

2010

IFIP Advances in Information and Communication Technology

View full text Add to dashboard Cite

show abstract

“…When evaluating an enterprise system, measures of these dimensions represent a snapshot of the organization's experience of the enterprise system at a point in time. The impact dimensions are an assessment of benefits that have followed (or not) from the system [18]- [20]. Evaluation of success is a difficult approach and it only makes sense if the result of an evaluation is used as a basis for actions which can result in an improvement of the systems performance (Stephan A.…”

Section: Literature Surveymentioning

confidence: 99%

A Framework for Investigating the ERP System Quality from Users Point of View in Indian Industries

Mukti¹,

Tripathi²,

Rawani³

2013

JOEBM

View full text Add to dashboard Cite

Abstract-It has been close to 15 years since the word ERP has been used in the Indian market. The Indian ERP market is currently estimated to be worth Rs 40, 0000 million and is expected to grow at a CAGR of 25 percent in the next 3-4 years. Avery high failures rate of ERP, 93% (approx) makes it a very critical issue which needs an immediate attention. Less work has been done to evaluate the quality of ERP in its operative phase in Indian Industries. The authors have proposed a systematic design of quality evaluation method, in order to investigate the quality of ERP in Indian Industries. Four quality factors namely usability, functionality, reliability and efficiency have been considered to evaluate the ERP to measure the quality from the user's point of view. By combining the outcome of each factor, we find out the Global Quality Preference of the ERP in that industry. ISO/IEC 9126 has been used as a base model for this work.

show abstract

“…The reason for this minimal change could be traced back to the issue of unbalanced data sets [2]. To solve this issue, and to enable the model to reach an accuracy level which allows it to be used in real-world enterprise settings, we needed to further analyze the nature of the data collected, as described for instance in [10,11,14,13].…”

Section: Towards Accurate Cost Predictionsmentioning

confidence: 99%

ONTOCOM Revisited: Towards Accurate Cost Predictions for Ontology Development Projects

Simperl

Popov

Bürger

2009

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Reliable methods to assess the costs and benefits of ontologies are an important instrument to demonstrate the tangible business value of semantic technologies within enterprises, as an argument to encourage their wide-scale adoption. The economic aspects of ontologies have been investigated in previous work of ours. With ONTOCOM we proposed a cost estimation model for ontologies and ontology development projects. This paper revisits this model and presents its latest achievements. We report on a comprehensive calibration of ONTOCOM based on a considerably larger data set of 148 ontology development projects. The calibration used a combination of statistical methods, ranging from preliminary data analysis to regression and Bayes analysis, and resulted a significant improvement of the prediction quality of up to 50%. In addition, the availability of a representative data set allowed us to identify meaningful directions for customizing the generic cost model along particular types of ontologies, and ontology-like structures as those specific to the emerging Web 3.0. Last but not least, we developed a software tool that allows ontology development project managers to easily use and adapt and to systematically calibrate the model, thus facilitating its adoption in real-world projects.

show abstract

Preliminary guidelines for empirical research in software engineering

Cited by 1,171 publications

References 43 publications

Comparing OpenBRR, QSOS, and OMM Assessment Models

Comparing OpenBRR, QSOS, and OMM Assessment Models

A Framework for Investigating the ERP System Quality from Users Point of View in Indian Industries

ONTOCOM Revisited: Towards Accurate Cost Predictions for Ontology Development Projects

Contact Info

Product

Resources

About