Can a Chatbot Support Software Engineers with Load Testing? Approach and Experiences

Okanović, Dušan; Beck, Samuel; Merz, Lasse; Zorn, Christoph; Merino, Leonel; Hoorn, André van; Beck, Fabian

doi:10.1145/3358960.3375792

Cited by 20 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Users get annoyed of this behavior, as bots are insensitive to the expertise of the users. Performobot ( Beck et al, 2020 ; Okanović et al, 2020 ), which runs load testing in Software systems tackled this issue by having intents at various levels. The bot lets expert, novice and intermediate users to execute the same action via different intents.…”

Section: Resultsmentioning

confidence: 99%

“…Feedback from the data exploration chatbot ( Matera & Castaldo, 2019 ) mentioned that the bot should provide hints or help at first and it has to support complex queries. Users who used Performobot ( Okanović et al, 2020 ) also mentioned that the bot should provide information about available commands, hints and available keywords. The bot lacks explanations and is unsuitable for complex load performance analysis for experts.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Bots in software engineering: a systematic mapping study

Santhanam

Hecking

Schreiber

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

Bots have emerged from research prototypes to deployable systems due to the recent developments in machine learning, natural language processing and understanding techniques. In software engineering, bots range from simple automated scripts to decision-making autonomous systems. The spectrum of applications of bots in software engineering is so wide and diverse, that a comprehensive overview and categorization of such bots is needed. Existing works considered selective bots to be analyzed and failed to provide the overall picture. Hence it is significant to categorize bots in software engineering through analyzing why, what and how the bots are applied in software engineering. We approach the problem with a systematic mapping study based on the research articles published in this topic. This study focuses on classification of bots used in software engineering, the various dimensions of the characteristics, the more frequently researched area, potential research spaces to be explored and the perception of bots in the developer community. This study aims to provide an introduction and a broad overview of bots used in software engineering. Discussions of the feedback and results from several studies provide interesting insights and prospective future directions.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Bots in software engineering: a systematic mapping study

Santhanam

Hecking

Schreiber

et al. 2022

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Research in the last years has explored various different dimensions of software engineering where bots may assist developers, including the automated fixing of functional bugs ( Urli et al, 2018 ), bug triaging ( Wessel et al, 2019 ), creating performance tests ( Okanović et al, 2020 ), or source code refactoring ( Wyrich & Bogner, 2019 ). This proliferation of bots is slowly creating demand for coordination between bots in a project, which has recently started to receive attention by Wessel & Steinmacher (2020) through the design of a “meta-bot”.…”

Section: Related Workmentioning

confidence: 99%

Dependency management bots in open-source systems—prevalence and adoption

Erlenhov

Neto

Leitner

2022

PeerJ Computer Science

View full text Add to dashboard Cite

Bots have become active contributors in maintaining open-source repositories. However, the definitions of bot activity in open-source software vary from a more lenient stance encompassing every non-human contributions vs frameworks that cover contributions from tools that have autonomy or human-like traits (i.e., Devbots). Understanding which of those definitions are being used is essential to enable (i) reliable sampling of bots and (ii) fair comparison of their practical impact in, e.g., developers’ productivity. This paper reports on an empirical study composed of both quantitative and qualitative analysis of bot activity. By analysing those two bot definitions in an existing dataset of bot commits, we see that only 10 out of 54 listed tools (mainly dependency management) comply with the characteristics of Devbots. Moreover, five of those Devbots have similar patterns of contributions over 93 projects, such as similar proportions of merged pull-requests and days until issues are closed. Our analysis also reveals that most projects (77%) experiment with more than one bot before deciding to adopt or switch between bots. In fact, a thematic analysis of developers’ comments in those projects reveal factors driving the discussions about Devbot adoption or removal, such as the impact of the generated noise and the needed adaptation in development practices within the project.

show abstract

“…We address these challenges by (i) automatically extracting and evolving performance tests using operational monitoring data and API information [14,17], (ii) the generation and selection of tailored tests based on current test concerns [12,13], as well as (iii) exploiting recommending test strategies suitable for testing in unreliable infrastructures [2,9].…”

Section: Continuous Testingmentioning

confidence: 99%

Performance Engineering for Microservices and Serverless Applications: The RADON Approach

Gias

Hoorn

Zhu

et al. 2020

Companion of the ACM/SPEC International Conference on Performance Engineering

Self Cite

View full text Add to dashboard Cite

Microservices and serverless are becoming integral parts of modern cloud-based applications. Tailored performance engineering is needed for assuring that the applications meet their requirements for quality attributes such as timeliness, resource efficiency, and elasticity. A novel DevOps-based framework for developing microservices and serverless applications is being developed in the RADON project. RADON contributes to performance engineering by including novel approaches for modeling, deployment optimization, testing, and runtime management. This paper summarizes the contents of our tutorial presented at the 11th ACM/SPEC International Conference on Performance Engineering (ICPE).

show abstract

Can a Chatbot Support Software Engineers with Load Testing? Approach and Experiences

Cited by 20 publications

References 30 publications

Bots in software engineering: a systematic mapping study

Bots in software engineering: a systematic mapping study

Dependency management bots in open-source systems—prevalence and adoption

Performance Engineering for Microservices and Serverless Applications: The RADON Approach

Contact Info

Product

Resources

About