Deploying Search Based Software Engineering with Sapienz at Facebook

Alshahwan, Nadia; Gao, Xinbo; Harman, Mark; Jia, Yunyi; Mao, Ke; Mols, Alexander; Tei, Taijin; Zorin, I. V.

doi:10.1007/978-3-319-99241-9_1

Cited by 67 publications

(84 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examples of such research programs and papers are empirical studies of TDD in practice, e.g., (Fucci et al 2015;Bhat and Nagappan 2006), and many other papers which are published in top-quality SE venues, such as ICSE, IEEE TSE and the Springer's Empirical Software Engineering journal, e.g., (Zimmermann et al 2005). Many of the topquality papers published by researchers working in corporate research centers also fall in this category, e.g., a large number of papers from Microsoft Research, e.g., (Johnson et al 2019;Kalliamvakou et al 2019), and successful applications of search-based SE in Facebook Research (Alshahwan et al 2018). Other examples are patents filed on practical SE topics, e.g., a patent on combined code searching and automatic code navigation filed by researchers working in ABB Research (Shepherd and Robinson 2017).…”

Section: Research Relevancementioning

confidence: 99%

Practical relevance of software engineering research: synthesizing the community’s voice

2020

View full text Add to dashboard Cite

Software engineering (SE) research should be relevant to industrial practice. There have been regular discussions in the SE community on this issue since the 1980's, led by pioneers such as Robert Glass. As we recently passed the milestone of "50 years of software engineering", some recent positive efforts have been made in this direction, e.g., establishing "industrial" tracks in several SE conferences. However, many researchers and practitioners believe that we, as a community, are still struggling with research relevance and utility. The goal of this paper is to synthesize the evidence and experience-based opinions shared on this topic so far in the SE community, and to encourage the community to further reflect and act on the research relevance. For this purpose, we have conducted a Multi-vocal Literature Review (MLR) of 54 systematically-selected sources (papers and non peer-reviewed articles). Instead of relying on and considering the individual opinions on research relevance, mentioned in each of the sources, the MLR aims to synthesize and provide the "holistic" view on the topic. The highlights of our MLR findings are as follows. The top three root causes of low relevance, discussed in the community, are: (1) Researchers having simplistic views (or wrong assumptions) about SE in practice; (2) Lack of connection with industry; and (3) Wrong identification of research problems. The top three suggestions for improving research relevance are: (1) Using appropriate research approaches such as action-research; (2) Choosing relevant (practical) research problems; and (3) Collaborating with industry. By synthesizing all the discussions on this important topic so far, this paper aims to encourage further discussions and actions in the community to increase our collective efforts to improve the research relevance. Furthermore, we raise the need for empirically-grounded and rigorous studies on the relevance problem in SE research, as carried out in other fields such as management science.

show abstract

Section: Research Relevancementioning

confidence: 99%

Practical relevance of software engineering research: synthesizing the community’s voice

2020

View full text Add to dashboard Cite

show abstract

“…ExRunner can be extended to other crash reproduction tools 37 for comparison, or assess the development of new ideas in existing tools. Our future work also includes the prioritization of crashes from JCrashPack to allow quick feedback on new ideas in a fast and automated way (Alshahwan et al 2018).…”

Section: Empirical Evaluation For Crash Reproductionmentioning

confidence: 99%

A benchmark-based evaluation of search-based crash reproduction

et al. 2019

View full text Add to dashboard Cite

Crash reproduction approaches help developers during debugging by generating a test case that reproduces a given crash. Several solutions have been proposed to automate this task. However, the proposed solutions have been evaluated on a limited number of projects, making comparison difficult. In this paper, we enhance this line of research by proposing JCrashPack, an extensible benchmark for Java crash reproduction, together with ExRunner, a tool to simply and systematically run evaluations. JCrashPack contains 200 stack traces from various Java projects, including industrial open source ones, on which we run an extensive evaluation of EvoCrash, the state-of-the-art tool for search-based crash reproduction. EvoCrash successfully reproduced 43% of the crashes. Furthermore, we observed that reproducing NullPointerException, IllegalArgumentException, and IllegalStateException is relatively easier than reproducing ClassCastException, ArrayIndexOutOfBoundsException and StringIndexOutOfBoundsException. Our results include a detailed manual analysis of EvoCrash outputs, from which we derive 14 current challenges for crash reproduction, among which the generation of input data and the handling of abstract and anonymous classes are the most frequents. Finally, based on those challenges, we discuss future research directions for search-based crash reproduction for Java.

show abstract

“…We recently deployed the Sapienz search-based software test data generation system at Facebook [1]. Sapienz uses a Search Based Software Testing approach [10] to generate and subsequently select test inputs.…”

Section: Introductionmentioning

confidence: 99%

“…Sapienz uses a Search Based Software Testing approach [10] to generate and subsequently select test inputs. It grew out of research prototype and now runs in continuous production at Facebook, testing changes to mobile apps as they are submitted [1]. Some of the crashes it reveals are also automatically fixed [14].…”

Section: Introductionmentioning

confidence: 99%