Rapid Composition for Networked Devices: HappyBrackets

Abstract

This article introduces an open-source Java-based programming environment for creative coding of agglomerative systems using Internet-of-Things (IoT) technologies. Our software originally focused on digital signal processing of audio—including synthesis, sampling, granular sample playback, and a suite of basic effects—but composers now use it to interface with sensors and peripherals through general-purpose input/output and external networked systems. This article examines and addresses the strategies required to integrate novel embedded musical interfaces and creative coding paradigms through an IoT infrastructure. These include: the use of advanced tooling features of a professional integrated development environment as a composition or performance interface rather than just as a compiler; techniques to create media works using features such as autodetection of sensors; seamless and serverless communication among devices on the network; and uploading, updating, and running of new compositions to the device without interruption. Furthermore, we examined the difficulties many novice programmers experience when learning to write code, and we developed strategies to address these difficulties without restricting the potential available in the coding environment. We also examined and developed methods to monitor and debug devices over the network, allowing artists and programmers to set and retrieve current variable values to or from these devices during the performance and composition stages. Finally, we describe three types of art work that demonstrate how the software, called HappyBrackets, is being used in live-coding and dance performances, in interactive sound installations, and as an advanced composition and performance tool for multimedia works.

Sustainable Internet of Musical Things: Strategies to Account for Environmental and Social Sustainability in Network-Based Interactive Music Systems

“…Moreover, the attention of researchers has focused on the development of several frameworks to connect Musical Things [46,83,132,34,126], as well as networked music performance system aiming at interconnecting geographically displaced musicians [39,62,23,19,127,25]. In particular, recent years have witnessed an increasing use of 5G technologies in musical settings [turchet2023latency, 24,40,28].…”

The Internet of Sounds: Convergent Trends, Insights, and Future Directions

“…2) Musical Things: Various types of Musical Things have been proposed by the IoMusT research community (e.g., [3], [6], [34], [78], [79]), along with frameworks to connect them (e.g., [80]- [82]).…”

On the relation between the fields of Networked Music Performances, Ubiquitous Music, and Internet of Musical Things

“…Different kinds of Musical Things prototypes have been developed by the IoMusT community (see, e.g., [43,92]), along with frameworks to connect them (see, e.g., [26,31,50,87]).…”