Model based testing of an interactive music system

Abstract

The role of an interactive music system (IMS) is to accompany musicians during live performances, like a real musician. It reacts in realtime to audio signals from musicians, according to a timed specification called mixed score, written in a domain specific language. Such goals imply strong requirements of temporal reliability and robustness to unforeseen errors in input, yet not so much studied in the computer music community.We present the application of model-based testing techniques and tools to a state-of-the-art IMS, including the following tasks: generation of relevant input data for testing (including timing values) following coverage criteria, computation of the corresponding expected output, according to the semantics of a given mixed score, black-box execution of the test data and verdict. Our method is based on formal models compiled directly from mixed scores, and passed, after conversion to timed automata, to the model-checker Uppaal. This fully automatic approach has been applied to real mixed scores used in concerts and the results obtained have permitted to identify bugs in the target IMS.

Authoring and Automatic Verification of Interactive Multimedia Scores

“…In (Poncelet & Jacquemard, 2015), the authors present the application of model base testing techniques to verify interactive music systems. They translate scores into a Timed Automata network, which is then processed by tools from UPPAAL in order to generate covering suites of test cases.…”

“…In the last decade, there have been a number of extensions to the original TA. In the following we shall give a brief introduction to the UPPAAL (David et al, 2015) tool and its modeling language, which has been used to model and analyze many real-time systems, e.g., audio protocols (Bengtsson et al, 2002), automotive systems (Kim, Larsen, Nielsen, Mikučionis, & Olsen, 2015;Lindahl, Pettersson, & Yi, 2001), orchestration systems (Dong, Liu, Sun, & Zhang, 2014), and multimedia systems (Echeveste, Cont, Giavitto, & Jacquemard, 2013;Poncelet & Jacquemard, 2015).…”

Model-based testing for building reliable realtime interactive music systems

“…But they generally do not offer procedures for generating test data, hence the user must compute some input test data and the expected corresponding output by other means. Our approach ( [29,30]) in contrast focus on the generation of test data, based on formal models, and in this respect the two approaches can be seen as complementary.…”

A Visual Framework For Dynamic Mixed Music Notation

“…Much like a traditional score, electronic action staves would mark the firing of messages or evolution of continuous value streams along a common timeline. We now present a perfected simulation model, to be implemented into the next version of AscoGraph, that more robustly handles the dynamic aspects of the score language and also supports the recently developed Antescofo test framework (Poncelet and Jacquemard, 2015).…”

“…Beyond the ideal trace produced by executing a score with all events and actions occurring as scripted in the score, the simulation view extends to support the Model-Based Testing workflow (Poncelet and Jacquemard, 2015), which builds a test case by receiving a timed input trace, executing the piece accordingly and outputting a verdict. Such an input trace describes the behaviour of the listening machine, by way of the deviations of the detected musician activity from the ideal score.…”