On the Differences between Unit and Integration Testing in the TravisTorrent Dataset

Orellana, Gerardo; Laghari, Gulsher; Murgia, Alessandro; Demeyer, Serge

doi:10.1109/msr.2017.25

Cited by 8 publications

(6 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Integration testing failures are more frequent in ING than in OSS. Instead, and consistently with a previous study by Orellana Cordero et al [14], OSS projects exhibit more unit testing related failures. This indicates a different distribution of testing activities across the development process in the open source and industrial context.…”

Section: Discussionsupporting

confidence: 90%

“…For example, both exhibit a relatively low percentage of compilation failures, and, instead, a high percentage of testing failures. However, while unit testing failures are common in OSS projects (as also discussed in [14]), ING projects have a much higher frequency of integration test failures. ING projects also exhibit a high percentage of build failures related to release preparation, as well as packaging and deployment errors.…”

Section: What Types Of Failures Affect Builds Of Oss and Ing Projects?mentioning

confidence: 99%

“…While our results confirm their finding, we also highlight the importance of failures due to other tasks. Orellana Cordero et al [14] studied test-related build failures in OSS projects. They identified that unit test failures dominate integration test failures.…”

Section: Related Workmentioning

confidence: 99%

“…Our results for OSS projects are similar, yet our findings for ING projects are the opposite. This may be due to more intensive usage of private builds to deal with unit tests in an industrial environment, which is not the case for the OSS projects studied by Orellana Cordero et al [14]. Zampetti et al [12] looked at build failures (mostly related to adherence to coding guidelines) produced by static analysis tools in Javabased OSS projects.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

A Tale of CI Build Failures: An Open Source and a Financial Organization Perspective

Vassallo

Schermann

Zampetti

et al. 2017

2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

Continuous Integration (CI) and Continuous Delivery (CD) are widespread in both industrial and open-source software (OSS) projects. Recent research characterized build failures in CI and identified factors potentially correlated to them. However, most observations and findings of previous work are exclusively based on OSS projects or data from a single industrial organization. This paper provides a first attempt to compare the CI processes and occurrences of build failures in 349 Java OSS projects and 418 projects from a financial organization, ING Nederland. Through the analysis of 34,182 failing builds (26% of the total number of observed builds), we derived a taxonomy of failures that affect the observed CI processes. Using cluster analysis, we observed that in some cases OSS and ING projects share similar build failure patterns (e.g., few compilation failures as compared to frequent testing failures), while in other cases completely different patterns emerge. In short, we explain how OSS and ING CI processes exhibit commonalities, yet are substantially different in their design and in the failures they report.

show abstract

Section: Discussionsupporting

confidence: 90%

Section: What Types Of Failures Affect Builds Of Oss and Ing Projects?mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A Tale of CI Build Failures: An Open Source and a Financial Organization Perspective

Vassallo

Schermann

Zampetti

et al. 2017

2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)

View full text Add to dashboard Cite

show abstract

“…As Trautsch et al (2020) concludes, there is no longer a clear distinction between unit testing and integration testing in modern software testing, and the JUnit framework is often used for both. Interestingly Orellana et al (2017) use naming conventions to distinguish unit and integration tests and Trautsch et al (2020) use coverage information for the same purpose, thus utilising techniques which are similar to those we evaluate. Orellana et al (2017) did find a difference between unit and integration tests with regards to the time and developer coordination needed to fix them but the findings were unintuitive as they found that unit tests took more time and coordination to fix than integration tests.…”

Section: Unit Vs Integration Testingmentioning

confidence: 99%

TCTracer: Establishing test-to-code traceability links using dynamic and static techniques

White

Krinke

2022

Empir Software Eng

View full text Add to dashboard Cite

Test-to-code traceability links model the relationships between test artefacts and code artefacts. When utilised during the development process, these links help developers to keep test code in sync with tested code, reducing the rate of test failures and missed faults. Test-to-code traceability links can also help developers to maintain an accurate mental model of the system, reducing the risk of architectural degradation when making changes. However, establishing and maintaining these links manually places an extra burden on developers and is error-prone. This paper presents TCTracer, an approach and implementation for the automatic establishment of test-to-code traceability links. Unlike existing work, TCTracer operates at both the method level and the class level, allowing us to establish links between tests and functions, as well as between test classes and tested classes. We improve over existing techniques by combining an ensemble of new and existing techniques that utilise both dynamic and static information and exploiting a synergistic flow of information between the method and class levels. An evaluation of TCTracer using five large, well-studied open source systems demonstrates that, on average, we can establish test-to-function links with a mean average precision (MAP) of 85% and test-class-to-class links with an MAP of 92%.

show abstract

An empirical characterization of bad practices in continuous integration

et al. 2020

View full text Add to dashboard Cite

Continuous Integration (CI) has been claimed to introduce several benefits in software development, including high software quality and reliability. However, recent work pointed out challenges, barriers and bad practices characterizing its adoption. This paper empirically investigates what are the bad practices experienced by developers applying CI. The investigation has been conducted by leveraging semi-structured interviews of 13 experts and mining more than 2,300 Stack Overflow posts. As a result, we compiled a catalog of 79 CI bad smells belonging to 7 categories related to different dimensions of a CI pipeline management and process. We have also investigated the perceived importance of the identified bad smells through a survey involving 26 professional developers, and discussed how the results of our study relate to existing knowledge about CI bad practices. Whilst some results, such as the poor usage of branches, confirm existing literature, the study also highlights uncovered bad practices, e.g., related to static analysis tools or the abuse of shell scripts, and contradict knowledge from existing literature, e.g., about avoiding nightly builds. We discuss the implications of our catalog of CI bad smells for (i) practitioners, e.g., favor specific, portable tools over hacking, and do not ignore nor hide build failures, (ii) educators, e.g., teach CI culture, not just technology, and teach CI by providing examples of what not to do, and (iii) researchers, e.g., developing support for failure analysis, as well as automated CI bad smell detectors.

show abstract

On the Differences between Unit and Integration Testing in the TravisTorrent Dataset

Cited by 8 publications

References 4 publications

A Tale of CI Build Failures: An Open Source and a Financial Organization Perspective

A Tale of CI Build Failures: An Open Source and a Financial Organization Perspective

TCTracer: Establishing test-to-code traceability links using dynamic and static techniques

An empirical characterization of bad practices in continuous integration

Contact Info

Product

Resources

About