Topic selection in industry experiments

Abstract

This paper shares our experience with initial negotiation and topic elicitation process for conducting industry experiments in six software development organizations in Finland. The process involved interaction with company representatives in the form of both multiple group discussions and separate face-to-face meetings. Fitness criteria developed by researchers were applied to the list of generated topics to decide on a common topic. The challenges we faced include diversity of proposed topics, communication gaps, misunderstanding about research methods, initial disconnect between research and industry needs, and lack of prior work relationship. Lessons learned include having enough time to establish trust with partners, importance of leveraging the benefits of training and skill development that are inherent in the experimental approach, uniquely positioning the experimental approach within the landscape of other validation approaches more familiar to industrial partners, and introducing the fitness criteria early in the process.

Challenges and best practices in industry-academia collaborations in software engineering: A systematic literature review

“…Furthermore, the differences in objectives [56,55,51] (C22), reward systems [46,23,48] (C25), and what is perceived as useful (C23) hinder to establish collaborations. For example, Glass and Hunt [57] point out that only few researchers would be interested in doing the development ("D") of R&D. With regard to rewards, it was pointed out that academia does not value industry impact and that there is no academic reward for industry collaborations and success [58,45].…”

“…'Proper' topic selection (BP19) is of importance early in the process [56,23] and topics need to be prioritized [51]. To choose a topic of interest, the problems of the companies have to be well understood.…”

“…Deep investigations of the problem area should be undertaken [61] to understand potential reasons for companies to participate [56].…”

An industry experiment on the effects of test-driven development on external quality and productivity

“…Causevic et al (2010) also conducted an industrial survey to examine the expected and actual level of usage for a number of test-related activities and found that "respondents would like to use TDD to a significantly higher extent than they actually do" due to TDD's claimed benefits in the literature. Our face-to-face meetings with professionals from six software companies in Finland, as part of our on-going project (Juristo 2016), also revealed that TDD is a highly popular topic, as professionals are interested in learning its key practices and its impact on their development process through experimentation (Tosun-Misirli et al 2014).…”

On the effectiveness of unit tests in test-driven development

“…[27]) and controlled experiments in academic settings (e.g. [18]), the industrial adoption of TDD has not been completed yet [29]. Causevic et al [10], in their literature review, highlighted 18 factors that could limit the industrial adoption of TDD.…”