Evaluating the suitability of state‐based formal methods for industrial deployment

Abstract

After a number of success stories in safety-critical domains, we are starting to witness applications of formal methods in contemporary systems and software engineering. However, one thing that is still missing is the evaluation criteria that help software practitioners choose the right formal method for the problem at hand. In this paper, we present the criteria for evaluating and comparing different formal methods. The criteria were chosen through a literature review, discussions with experts from academia and practitioners from industry, and decade-long personal experience with the application of formal methods in industrial and academic projects. The criteria were then evaluated on several model-oriented state-based formal methods. Our research shows that besides technical grounds (eg, modeling capabilities and supported development phases), formal methods should also be evaluated from social and industrial perspectives. We also found out that it is not possible to generate a matrix that renders the selection of the right formal method an automatic process. However, we can generate several pointers, which make this selection process a lot less cumbersome.

Automatically learning formal models

“…Such automated methods will significantly help find potential errors as they can automatically generate and verify production code at regular intervals. This will further strengthen the suitability of formal methods for industrial deployment [38,41]. Automatically constructing formal models can also help understand and reason about ill-documented legacy systems and black-box systems which is crucial for the quality assurance in large-scale and complex automotive systems.…”

“…Admittedly, formal methods can be relatively more beneficial if introduced during the early stages of the software development workflow rather than being used for post-hoc verification after development. However, there are several obstacles that impede the widespread adoption of formal methods [39,41]. Significant trade-offs (e.g.…”

IVOIRE Deliverable 1.1: Classification of existing VOs & tools and Formalization of VOs semantics

“…Whenever something is marked with , we did not find referable evidence for the existence of the respective tool support. A more comprehensive evaluation of state-based formal methods is provided by Mashkoor et al [69]. One can see that tool support is widely spread.…”

Formal methods by stealth: The INSPEX experience

“…We can cite literature. [51][52][53][54][55][56][57] Among the approaches that are perhaps closest to ours here is the so-called 'lightweight formal methods' approach, dating back to the mid 1990s. [58][59][60][61] In these, the 'standard' way of deploying formal techniques is replaced by a less comprehensive approach.…”