Validation, Verification, and Testing of Computer Software

Adrion, W. Richards; Branstad, Martha; Cherniavsky, John C.

doi:10.1145/356876.356879

Cited by 334 publications

(111 citation statements)

References 32 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…In Knupp et al (2007), it is implicitly argued that code verification takes too much effort to perform on the very latest version, and is thus relegated to release candidates; however, by automating the process, we have achieved great success in applying code verification to a codebase that changes daily. This philosophy is widespread in the software engineering community (Adrion et al, 1982), but is not yet the generally accepted practice among the scientific modelling community. We discuss a framework for automated continuous verification with particular suitability to numerical models.…”

Section: Verification and Validationmentioning

confidence: 99%

Automated continuous verification for numerical simulation

et al. 2011

View full text Add to dashboard Cite

Abstract. Verification is a process crucially important for the final users of a computational model: code is useless if its results cannot be relied upon. Typically, verification is seen as a discrete event, performed once and for all after development is complete. However, this does not reflect the reality that many geoscientific codes undergo continuous development of the mathematical model, discretisation and software implementation. Therefore, we advocate that in such cases verification must be continuous and happen in parallel with development: the desirability of their automation follows immediately. This paper discusses a framework for automated continuous verification of wide applicability to any kind of numerical simulation. It also documents a range of test cases to show the possibilities of the framework.

show abstract

Section: Verification and Validationmentioning

confidence: 99%

Automated continuous verification for numerical simulation

et al. 2011

View full text Add to dashboard Cite

show abstract

“…Schlesinger, 1979;Adrion et al, 1982;Balci, 1997;Lipaev, 2003;Dasso and Funes, 2006) the same standards and techniques cannot be used for models of natural systems as outlined by Oreskes et al (1994). This is because natural systems are never closed and because model results are always non-unique.…”

Section: Introductionmentioning

confidence: 99%

A general Cluster data and global MHD simulation comparison

et al. 2008

View full text Add to dashboard Cite

Abstract. Among the many challenges facing the space weather modelling community today, is the need for validation and verification methods of the numerical models available describing the complex nonlinear Sun-Earth system. Magnetohydrodynamic (MHD) models represent the latest numerical models of this environment and have the unique ability to span the enormous distances present in the magnetosphere, from several hundred kilometres to several thousand kilometres above the Earth's surface. This makes it especially difficult to develop verification and validation methods which posses the same range spans as the models. In this paper we present a first general large-scale comparison between four years (2001)(2002)(2003)(2004) worth of in situ Cluster plasma observations and the corresponding simulated predictions from the coupled Block-Adaptive-Tree-SolarwindRoe-Upwind-Scheme (BATS-R-US) MHD code. The comparison between the in situ measurements and the model predictions reveals that by systematically constraining the MHD model inflow boundary conditions a good correlation between the in situ observations and the modeled data can be found. These results have an implication for modelling studies addressing also smaller scale features of the magnetosphere. The global MHD simulation can therefore be used to place localised satellite and/or ground-based observations into a global context and fill the gaps left by measurements.

show abstract

“…Representative test cases are constructed during optimization procedure using the model (1). The test selection is combined with the test execution and these two activities are conducted at the same time.…”

Section: A Test Selection Methods For Negative Testingmentioning

confidence: 99%

“…If the tests fail then the system needs to be redesigned. The physical system that is being tested for the correct operation is often referred as system under test (SUT).The term test oracle describes a source to determine expected results to compare with the actual result of the SUT [1]. The role of such source in the model-based approach is often played by the model (see Fig.…”

mentioning

confidence: 99%

Mathematical methods for verification of microprocessor-based PID controllers for improving their reliability

Skruch¹,

Długosz²,

Mitkowski³

2015

EiN

View full text Add to dashboard Cite

Proportional-Integral-Derivative (PID) control is the most common control algorithm used in industry. The extensive use of electronics and software has resulted in the situation where the digital PID controller using a microprocessor as well as its software implementation replaces existing pneumatic, mechanical and electromechanical solutions. The reliability of the software system is assured by detection and removal of errors that can lead to failures. The paper presents mathematical methods for verification and testing of microprocessor-based PID controllers that can be used to increase the reliability of the system. The presented methodology explores the concept of testing with a model as an oracle. Keywords: controller, PID, testing, reliability.Regulator PID (regulator proporcjonalno-całkująco-różniczkujący) IntroductionProportional-Integral-Derivative (PID) control is the most common control algorithm used in industry. It has been in use for over a century in various forms: as a purely mechanical device, as a pneumatic device and as an electronic device.Modern digital PID controller is a system that can be considered as a combination of computer hardware and software designed to perform a dedicated control function. The control is implemented on a custom hardware platform, which is often designed and configured for the particular application. Such systems are called embedded systems [29,30]. Embedded systems may be observed in common devices employed in everyday living (e.g., coffee machines, washing machines, cell phones) as well as in sophisticated engineering systems (e.g. cars [4,29], planes, spacecrafts).PID controllers are also often safety critical systems. Due to the area of application, the PID controller must have high reliability as unexpected failures can be fatal. Ensuring the reliability of embedded software systems based on the detection and removal of errors that can lead to system failure. The process to verify that the system meets the specified requirements is referred to as testing. Testing is also the process of trying to discover every conceivable fault or weakness in a work product [12,14].The most common errors that can lead to improper operation of control devices equipped with the software include functional errors in the code, arithmetic errors associated with the use of fixed-point arithmetic, communication and task management errors, lack of robustness to different types of disturbances and work outside the scope of the variability of input signals. There are several facts that show clearly possible consequences of poorly tested systems. On February 25, 1991, an Iraqi Scud hit the barracks in Dhahran in Saudi Arabia, killing 28 soldiers from the US Army. This accident was caused by software error in the system's clock [24]. The PATRIOT missile battery has been in operation for 100 hours, by which time the system's internal clock had drifted by one third of a second. For a target moving as fast as Scud, this was equivalent to a position error of 600 meters. Another example i...

show abstract

Validation, Verification, and Testing of Computer Software

Cited by 334 publications

References 32 publications

Automated continuous verification for numerical simulation

Automated continuous verification for numerical simulation

A general Cluster data and global MHD simulation comparison

Mathematical methods for verification of microprocessor-based PID controllers for improving their reliability

Contact Info

Product

Resources

About