Metamorphic Testing for Hybrid Simulation Validation

Mohammed, Farhan; Krejci, Caroline C.; Olsen, Megan; Raunak, Mohammad

doi:10.23919/annsim52504.2021.9552058

Cited by 2 publications

(1 citation statement)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several studies have been conducted to use MT to test the software for scientific computations and simulations. For instances, in testing simulations, MT has been successfully adopted to test agent-based (ABM) and discrete-event (DES) simulations [26], hybrid ABM and DES systems [11], health care simulation [25], webenabled simulation [1], and simulator platform for self-driving cars [36,41]. In testing scientific software, MT is an effective technique to detect faults in simulation programs for designing nuclear power plants [14], bioinformatics programs [4], epidemiological models [31,33], chemical reaction networks for prototyping nano-scale molecular devices [13], matrix calculation programs [32], solvers for partial differential equations [3], multiple linear regression software [22], ocean modelling [16], storm water management model systems [19], machine learning-based hydro-logical models [40], Monte-Carlo computational programs [10,30], serverless scientific applications [20], as well as other types of scientific software [9,18,19,29].…”

Section: Introductionmentioning

confidence: 99%

Testing Ocean Software with Metamorphic Testing

Luu,

Liu,

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

Advancing ocean science has a significant impact to the development of the world, from operating a safe navigation for vessels to maintaining a healthy and diverse ocean ecosystem. Various ocean software systems have been extensively adopted for different purposes, for instance, predicting hourly sea level elevation across shorelines, simulating large-scale ocean circulations, as well as integrating into Earth system models for weather forecasts and climate projections. Regardless of their significance, guaranteeing the trustworthiness of ocean software and modelling systems is a long-standing challenge. The testing of ocean software suffers a lot from the so-called oracle problem, which refers to the absence of test oracles mainly due to the nonlinear interactions of multiple physical variables and the high complexity in computation. In the ocean, observed tidal signals are distorted by non-deterministic physical variables, hindering us from knowing the "true" astronomical tidal constituents existing in the timeseries. In this paper, we present how to test tidal analysis and prediction (TAP) software based on metamorphic testing (MT), a simple yet effective testing approach to the oracle problem. In particular, we construct metamorphic relations from the periodic property of astronomical tide, and then use them to successfully detect a real-life defect in an open-source TAP software. We also conduct a series of experiments to further demonstrate the applicability and effectiveness of MT in the testing of TAP software. Our study not only justifies the potential of MT in testing more complex ocean software and modelling systems, but also can be expanded to assess and improve the quality of a broader range of scientific simulation software systems. CCS CONCEPTS• Applied computing → Earth and atmospheric sciences; • Software and its engineering → Software testing and debugging.

show abstract