Nowadays, there exists an increasing demand for reliable software systems able to fulfill their requirements in different operational environments and to cope with uncertainty that can be introduced both at design-time and at runtime because of the lack of control over third-party system components and complex interactions among software, hardware infrastructures and physical phenomena. This article addresses the problem of the discrepancy between measured data at runtime and the design-time formal specification by using an inverse uncertainty quantification approach. Namely, we introduce a methodology called METRIC and its supporting toolchain to quantify and mitigate software system uncertainty during testing by combining (on-the-fly) model-based testing and Bayesian inference. Our approach connects probabilistic input/output conformance theory with statistical hypothesis testing in order to assess if the behaviour of the system under test corresponds to its probabilistic formal specification provided in terms of a Markov decision process. An uncertainty-aware model-based test case generation strategy is used as a means to collect evidence from software components affected by sources of uncertainty. Test results serve as input to a Bayesian inference process that updates beliefs on model parameters encoding uncertain quality attributes of the system under test. This article describes our approach from both theoretical and practical perspectives. An extensive empirical evaluation activity has been conducted in order to assess the cost-effectiveness of our approach. We show that, under same effort constraints, our uncertainty-aware testing strategy increases the accuracy of the uncertainty quantification process up to 50 times with respect to traditional model-based testing methods. of uncertainty away because of insufficient understanding and unavailability of adequate methods and techniques. Today, endowing conventional software engineering methodologies with techniques and practices able to model, quantify, and manage uncertainty explicitly is becoming increasingly crucial [1,2]. In particular, techniques that explicitly consider uncertainty in software testing is an emerging research direction.This article focuses on the problem of uncertainty quantification and introduces a methodology able to deal with it by means of a model-based hypothesis testing approach. Namely, we use statistical hypothesis testing to reduce the discrepancy between delivered products and initial design-time uncertainty assumptions. As described by Broy at al.[3], model-based testing (MBT) is a software testing technique where runtime behaviour of a software system under test (SUT) is checked against predictions made by a model description of the system's behaviour. In our approach, the design-time model description contains uncertain aspects explicitly specified by means of probability. Uncertainty must be mitigated accounting for evidence during the actual system's execution. Thus, we make use of (on-the-fly) MBT as a means to extrac...