What do Programmers Discuss about Deep Learning Frameworks

Han, Junxiao; Shihab, Emad; Wan, Zhiyuan; Deng, Shuiguang; Xia, Xin

doi:10.1007/s10664-020-09819-6

Cited by 74 publications

(59 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We identified all the clients of these 15 libraries (identified by Braiek et al) from our corpus, as shown in Table 1. We found that 95% of the projects in our corpus use the six popular libraries (identified by researchers [56,69,70,125,144]).…”

Section: Librariesmentioning

confidence: 95%

“…While previous researchers 1 The library has only one release in 2013. [56,69,70,125,144,145] have particularly studied the usage of the six most popular libraries (Ten-sorFlow, Scikit-Learn, Keras, PyTorch, Caffe, and Theano). We identified all the clients of these 15 libraries (identified by Braiek et al) from our corpus, as shown in Table 1.…”

Section: Librariesmentioning

confidence: 99%

See 1 more Smart Citation

Understanding Software-2.0

Dilhara

Ketkar

Dig

2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Enabled by a rich ecosystem of Machine Learning (ML) libraries, programming using learned models , i.e., Software-2.0 , has gained substantial adoption. However, we do not know what challenges developers encounter when they use ML libraries. With this knowledge gap, researchers miss opportunities to contribute to new research directions, tool builders do not invest resources where automation is most needed, library designers cannot make informed decisions when releasing ML library versions, and developers fail to use common practices when using ML libraries. We present the first large-scale quantitative and qualitative empirical study to shed light on how developers in Software-2.0 use ML libraries, and how this evolution affects their code. Particularly, using static analysis we perform a longitudinal study of 3,340 top-rated open-source projects with 46,110 contributors. To further understand the challenges of ML library evolution, we survey 109 developers who introduce and evolve ML libraries. Using this rich dataset we reveal several novel findings. Among others, we found an increasing trend of using ML libraries: The ratio of new Python projects that use ML libraries increased from 2% in 2013 to 50% in 2018. We identify several usage patterns including the following: (i) 36% of the projects use multiple ML libraries to implement various stages of the ML workflows, (ii) developers update ML libraries more often than the traditional libraries , (iii) strict upgrades are the most popular for ML libraries among other update kinds, (iv) ML library updates often result in cascading library updates, and (v) ML libraries are often downgraded (22.04% of cases). We also observed unique challenges when evolving and maintaining Software-2.0 such as (i) binary incompatibility of trained ML models and (ii) benchmarking ML models. Finally, we present actionable implications of our findings for researchers, tool builders, developers, educators, library vendors, and hardware vendors.

show abstract

Section: Librariesmentioning

confidence: 95%

Section: Librariesmentioning

confidence: 99%

Understanding Software-2.0

Dilhara

Ketkar

Dig

2021

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

show abstract

“…In recent years, several studies have focused on the challenges in developing DL applications. For example, Han et al [14] applied an automatic topic modelling technique to the SO questions related to three popular DL frameworks and derived the topics contained in these questions. Their results revealed common concerns that developers face when using DL frameworks, such as version problems and model training.…”

Section: III Re L a T E D W O R Kmentioning

confidence: 99%

“…have been well studied in the software engineering (SE) research community [14]- [19]. In particular, researchers [16]- [19] have extensively analyzed faults in the DL programs written based on DL frameworks (e.g., TensorFlow (TF) [20] and Keras [21]), which encode the structure of desirable DL models and the process by which the models are trained using the training data.…”

Section: Introductionmentioning

confidence: 99%

An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

Chen

Yao

Lou

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Deep learning (DL) is moving its step into a growing number of mobile software applications. These software applications, named as DL based mobile applications (abbreviated as mobile DL apps) integrate DL models trained using large-scale data with DL programs. A DL program encodes the structure of a desirable DL model and the process by which the model is trained using training data. Due to the increasing dependency of current mobile apps on DL, software engineering (SE) for mobile DL apps has become important. However, existing efforts in SE research community mainly focus on the development of DL models and extensively analyze faults in DL programs. In contrast, faults related to the deployment of DL models on mobile devices (named as deployment faults of mobile DL apps) have not been well studied. Since mobile DL apps have been used by billions of end users daily for various purposes including for safety-critical scenarios, characterizing their deployment faults is of enormous importance. To fill in the knowledge gap, this paper presents the first comprehensive study to date on the deployment faults of mobile DL apps. We identify 304 real deployment faults from Stack Overflow and GitHub, two commonly used data sources for studying software faults. Based on the identified faults, we construct a fine-granularity taxonomy consisting of 23 categories regarding to fault symptoms and distill common fix strategies for different fault symptoms. Furthermore, we suggest actionable implications and research avenues that can potentially facilitate the deployment of DL models on mobile devices.

show abstract

“…For example, Allamanis and Sutton (Allamanis and Sutton 2013 ) found that software developers discuss topics relating to how and why implementations and technologies do or do not work, while Barua et al (Barua et al 2014 ) analyzed how topics interact and change over time. Other studies analyzed the topics discussed on Stack Overflow about specific software development subjects such as machine learning (Bangash et al 2019 ; Han et al 2020 ), mobile development (Linares-Vásquez et al 2013 ; Rosen and Shihab 2016 ), and security (Yang et al 2016 ). Researchers have also analyzed the trends for topics of different software development disciplines finding that, for example, continuous engineering topics attracted fewer answers and are decreasing over time (Zahedi et al 2020 ), while web development topics increased their share of questions (Bajaj et al 2014 ).…”

Section: Background and Related Workmentioning

confidence: 99%

An empirical study of Q&A websites for game developers

Kamienski

Bezemer

2021

Empir Software Eng

View full text Add to dashboard Cite

The game development industry is growing, and training new developers in game development-specific abilities is essential to satisfying its need for skilled game developers. These developers require effective learning resources to acquire the information they need and improve their game development skills. Question and Answer (Q&A) websites stand out as some of the most used online learning resources in software development. Many studies have investigated how Q&A websites help software developers become more experienced. However, no studies have explored Q&A websites aimed at game development, and there is little information about how game developers use and interact with these websites. In this paper, we study four Q&A communities by analyzing game development data we collected from their websites and the 347 responses received on a survey we ran with game developers. We observe that the communities have declined over the past few years and identify factors that correlate to these changes. Using a Latent Dirichlet Allocation (LDA) model, we characterize the topics discussed in the communities. We also analyze how topics differ across communities and identify the most discussed topics. Furthermore, we find that survey respondents have a mostly negative view of the communities and tended to stop using the websites once they became more experienced. Finally, we provide recommendations on where game developers should post their questions, which can help mitigate the websites’ declines and improve their effectiveness.

show abstract

What do Programmers Discuss about Deep Learning Frameworks

Cited by 74 publications

References 46 publications

Understanding Software-2.0

Understanding Software-2.0

An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications

An empirical study of Q&A websites for game developers

Contact Info

Product

Resources

About