CATtalk: An IoT-Based Interactive Art Development Platform

Abstract

Interactive art has been significant advanced by the Internet of Things (IoT) and cyber physical interaction technologies, which enables the participants to engage with the art devices. Several tools and platforms have been proposed to create the art devices. However, the interactive artworks are typically developed with these art devices in ad hoc approaches, and the artists need to spend significant programming efforts to integrate the art devices. This paper proposes CATtalk, a platform to create and maintain interactive artworks. The novel idea is to treat all art devices in an interactive artwork as IoT devices that can be transparently reused by reconfiguration in CATtalk. Therefore, the artworks developed independently by individual artists can be quickly integrated to create new interactive applications. Through CATtalk's no-code and low-code mechanisms, the artists can manipulate CATtalk with little or no programing efforts. CATtalk offers a built-in mechanism so that any person in the audience can play with an interactive artwork with his/her smartphone. We also conduct analytic analysis, simulation and measurements to ensure that the interactive art performance in cross-country remote stages are not affected by the communications delays. In our measurements, the average local and remote communication delays are about 0.01 and 0.05 seconds, respectively. If the art performance is designed such that the average delay between two actions of a local (remote) performer is longer than 0.1 seconds, then the probability of out-of-sequence actions is less than 0.01%. That is, the local dancer should perform slower than the remote dancer. Such delay analysis for remote interactive art performance has not been conducted in the literature.

ViolinTalk: Violin Robots as Internet of Things Devices

“…Based on the CATtalk IoT application creation platform [3], we develop the ViolinTalk server (Figure 7 (1)). The ViolinTalk server is installed on an industrial Raspberry Pi4 (Figure 8 (1)).…”

“…Therefore, configuring the IoT devices and their connections in Figure 7 can be easily accomplished in the ViolinTalk GUI using Joins 1-5 in Figure 8. For more details, please refer to [3]. ViolinTalk also installs multiple cameras (see Figure 7 (11)) to observe the robot's operation, with the video streams directed in realtime to the video viewer (see Figure 9 (5)).…”

“…All IoT devices involved in the project, such as the Control Board, Calibration, and Performance components, can be selected from the "Model" dropdown list (Figure 7 (7)). More details can be found in [3].…”

“…This paper proposes ViolinTalk, an IoT-based control system for XY-type robots. By connecting the XY-type robot to the CATtalk IoT development platform [3], ViolinTalk provides a powerful environment for enhancing the performance of XY-type robots through calibration and testing. This paper is organized as follows: Section 2 overviews the related studies; Section 3 describes the design and implementation of the XY-type violin robot; Section 4 proposes ViolinTalk based on the CATtalk IoT development platform; Sections 5-7 elaborate on the design and implementation of the bowing, string-changing, and fingerpressing software modules and their integration with the mechanical mechanisms; Section 8 shows how ViolinTalk effectively calibrates and plays the XY-type violin robots; Section 9 discusses the interaction among multiple music instruments.…”

Practitioners’ Perceptions on the Adoption of Low Code Development Platforms

“…An option to react to these challenges is to use Low Code Development Platforms (LCDP), which are promoted to increase efficiency, effectiveness, reduce costs, and empower users [4]- [8]. Moreover, LCDP vendors advertise their products as being capable of supporting professional software developers but also developers in the business department or regular business employees (often referred to as citizen developers) who develop applications with little to no programming experience [4]- [6], [9].…”