A Method-Level Test Generation Framework for Debugging Big Data Applications

Feng, Huadong; Chandrasekaran, Jaganmohan; Lei, Yu; Kacker, Raghu N.; Kuhn, D. Richard

doi:10.1109/bigdata.2018.8622248

Cited by 2 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feng et al [54] proposed a method-level test execution framework to solve the Big Data applications debugging issues arising from a large volume of data. Their method consists of two steps.…”

Section: Methods Level Test Execution Frameworkmentioning

confidence: 99%

“…Challenges during validation of pre-Hadoop, data processing phase and while generating reports [9] ETL testing problems [41,46,53] Transformation of data problems [42] Performance, scalability, continuous availability and data security, meeting speed of data, understanding it and addressing data quality and node failure [48,51] Debugging and testing at module level is time consuming phases [54] Vulnerabilities challenges…”

Section: Testing Challenges Studiesmentioning

confidence: 99%

“…Feng et al [54] highlighted that debugging and testing Big Data applications is challenging at the module/unit level due to the large volume of data. They proposed a framework for effectively testing method-level tests to facilitate the debugging of Big Data.…”

Section: Unit Level Debugging Challengementioning

confidence: 99%

See 2 more Smart Citations

Big Data Testing Techniques: Taxonomy, Challenges and Future Trends

Arshad¹,

Alsamhi²

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Big Data is reforming many industrial domains by providing decision support through analyzing large data volumes. Big Data testing aims to ensure that Big Data systems run smoothly and error-free while maintaining the performance and quality of data. However, because of the diversity and complexity of data, testing Big Data is challenging. Though numerous research efforts deal with Big Data testing, a comprehensive review to address testing techniques and challenges of Big Data is not available as yet. Therefore, we have systematically reviewed the Big Data testing techniques' evidence occurring in the period 2010-2021. This paper discusses testing data processing by highlighting the techniques used in every processing phase. Furthermore, we discuss the challenges and future directions. Our findings show that diverse functional, non-functional and combined (functional and non-functional) testing techniques have been used to solve specific problems related to Big Data. At the same time, most of the testing challenges have been faced during the MapReduce validation phase. In addition, the combinatorial testing technique is one of the most applied techniques in combination with other techniques (i.e., random testing, mutation testing, input space partitioning and equivalence testing) to find various functional faults through Big Data testing.

show abstract

Section: Methods Level Test Execution Frameworkmentioning

confidence: 99%

Section: Testing Challenges Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Big Data Testing Techniques: Taxonomy, Challenges and Future Trends

Arshad¹,

Alsamhi²

2023

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…Feng et al [73] propose to debug the Big Data applications by means of a binary search aimed to reduce the input data while at the same time preserving the test coverage. MRDebug is not focused on general-domain failures as the previous technique, but on the design faults of the MapReduce applications.…”

Section: Related Workmentioning

confidence: 99%

Automatic Debugging of Design Faults in MapReduce Applications

Morán,

Bertolino,

de la Riva

et al. 2024

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Among the current technologies to analyse large data, the MapReduce processing model stands out in Big Data. MapReduce is implemented in frameworks such as Hadoop, Spark or Flink that are able to manage the program executions according to the resources available at runtime. The developer should design the program in order to support all possible nondeterministic executions. However, the program may fail due to a design fault. Debugging these kinds of faults is difficult because the data are executed non-deterministically in parallel and the fault is not caused directly by the code, but by its design. This paper presents a framework called MRDebug which includes two debugging techniques focused on the MapReduce design faults. A spectrum-based fault localization technique locates the root cause of these faults analysing several executions of the test case, and a Delta Debugging technique isolates the data relevant to trigger the failure. An empirical evaluation with 13 programs shows that MRDebug is effective in debugging the faults, especially when the localization is done with the reduced data. In summary, MRDebug automatically provides valuable information to understand MapReduce design faults as it helps locate their root cause and obtains a minimal data that triggers the failure.

show abstract

A Method-Level Test Generation Framework for Debugging Big Data Applications

Cited by 2 publications

References 15 publications

Big Data Testing Techniques: Taxonomy, Challenges and Future Trends

Big Data Testing Techniques: Taxonomy, Challenges and Future Trends

Automatic Debugging of Design Faults in MapReduce Applications

Contact Info

Product

Resources

About