On the Understanding of Experimentation Usage in Light of Lean Startup in Software Development Context

Abstract

Experimentation can be used to validate software solutions, reducing waste on efforts and costs that would be spent had the solution not been validated from the beginning. Aware of experimentation benefits, a methodology named as Lean Startup have incorporated experiments and build-measure-learn loop as a manner to reduce waste from product development by focusing on efforts that will create value to customers, such as discovering the best solution. Aiming to characterize the use of experimentation in light of Lean Startup, we conducted a case study with two software development teams from a multinational information technology company. We characterize how experimentation works by teams perception, what are the benefits and challenges in the use of it. We identified in the study that experimentation was rooted in the workers' mindset and came up naturally should its need arise. Our findings indicate that the process of experimentation, albeit still a systematic effort, happens organically, and that it helps in development decisions and in reducing development efforts. Challenges are related to organizational culture and lack of belief in the approach by stakeholders. CCS CONCEPTS• Software and its engineering → Agile development.

Improving Agile Software Development using User-Centered Design and Lean Startup

“…As mentioned in our previous studies on this case, the teams reported a series of perceived benefits after adopting the combined approach, from increased code quality and trust among team members [11] to rapid feedback and reduced development effort [9]. The combined approach appears to be a suitable fit for ORG, as senior staff members continue to invest organizational resources in it with the intent of spreading it throughout the whole organization.…”

“…Ever since the teams started working at the PUCRS lab, we have analyzed several aspects of this case, from comparing the teams' use of the combined approach to existing literature [32] to analyzing how experimentation takes place during their development process [9] and what are the success and failure factors to adopting the combined approach from the teams' perspective [12]. As previously mentioned, our goal is to better comprehend the combined approach with two main contributions in mind: practice-wise, to help ORG in devising and rolling out a reduced-cost transformation effort throughout their organization by understanding the intricacies of doing so; and academic-wise, to conceptually develop, in the long-run, what we dubbed as an ''acceleration'' model [33]-a model to assist in such kind of transformation effort by identifying what knowledge (e.g., on experimentation, on developing empathy with the user, on pair programming) is required for a software team to develop a certain product using the combined approach, diagnosing the gap between the required and the actual team knowledge, and offering support to follow-up on actions aiming to fill in this knowledge gap.…”

“…The ORG teams perceived a series of benefits when using this combined approach, from increased code quality and trust among team members [11] to rapid feedback and reduced development effort [9].…”

“…Some authors argue that agile methods alone are not enough to tackle business-level issues [6] and that they provide insufficient customer involvement [18]. Combining agile methods with other approaches to development has been suggested as a way to fix these issues [6], and combining them specifically with Lean Startup and UCD has shown great promise as it seemingly increases the engagement of business stakeholders and end users while also enabling rapid experimentation, among other benefits [9,11].…”

Adopting Agile Software Development Combined with User-Centered Design and Lean Startup: A Systematic Literature Review on Maturity Models

“…Adding Lean Startup to this "method combo" is rather of a novelty given the time frame of the three approaches. Current literature encompasses studies contemplating several aspects of the approach itself (e.g., benefits, challenges, or use of experimentation) [66,67,74] and also studies that propose models to using the combined approach with varying degrees of abstraction [13,22,75]. The combined approach can be very different from typical agile development as it requires a certain degree of developer empowerment that larger organizations might not be used to, making adaptation efforts difficult and the use of maturity models and enticing choice.…”