Musical Vision: an interactive bio-inspired sonification tool to convert images into music

Polo, Antonio J.; Sevillano, Xavier

doi:10.1007/s12193-018-0280-4

Cited by 15 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…erefore, the flexible data gloves made are all made with some traditional and easy to replace electrical components, and some classical filtering methods are chosen to process the data, to better show the sensing performance of flexible sensors in practical applications. Commonly used wearable sensing units include the following: accelerometers-accelerometers are the basic sensing units commonly used in motion monitoring, measuring acceleration rates, and determining whether the device is in a horizontal or vertical position; gyroscope-a gyroscope is a device that detects angular motion and measures the angle of rotation of an object [18]; magnetometer-a magnetometer is a compass that can be used to improve the accuracy of motion tracking. Barometric sensing unit-a barometer is used to measure atmospheric pressure and can be used to determine changes in altitude by atmospheric pressure.…”

Section: Related Workmentioning

confidence: 99%

Design and Application of Interactive Music Equipment Based on Wireless Wearable Sensors

Wang

2022

Scientific Programming

View full text Add to dashboard Cite

The fusion of emerging technology, means, and music has provided a diversified development direction for the performance form of electronic music. Interactive electronic music has become a hot form of music performance in recent years and is widely used in various art exhibitions. In this paper, we will briefly explain the basic principles and practical applications of wireless wearable sensing devices, sort out the development of interactive platforms and sensors, and study the application value of wireless wearable sensing devices in interactive electronic music works. Near-field communication (NFC), as a short-range radio technology, can realize wireless energy and data transmission simultaneously. By integrating NFC modules, electrochemical sensing circuits, and flexible electrodes, it is possible to ensure complete electrochemical sensing functions while maximizing the miniaturization, flexibility, and integration of the system. Therefore, this paper constructs a series of wireless passive flexible electrochemical sensing systems based on NFC technology, electrochemical sensing technology, and flexible electronics, and uses these systems to realize a series of innovative applications, aiming to guide the creation of higher quality interactive music artworks as well as explore the artistic and esthetic experience brought by interactive music functions, and provide theoretical references for virtual art interactive music creation and practice.

show abstract

Section: Related Workmentioning

confidence: 99%

Design and Application of Interactive Music Equipment Based on Wireless Wearable Sensors

Wang

2022

Scientific Programming

View full text Add to dashboard Cite

show abstract

“…Musical Vision [17] is an image sonification system that uses an RGB color model. Color saturation is coded as volume, and pitch changes represent the spatial location of the pixels.…”

Section: Existing Color-to-sound Ssdsmentioning

confidence: 99%

“…However, the color-to-sound coding devices seem relatively unexplored. To the authors' knowledge, only 11 systems realize color sonification [15][16][17][18][19][20][21][22][23][24][25], while only six provide real-time sonification of the acquired color information [18,19,[22][23][24][25] (see Section 4-Existing Color-to-Sound SSDs for details). One of these was the first version of the Colorophone system [25].…”

Section: Introductionmentioning

confidence: 99%

Colorophone 2.0: A Wearable Color Sonification Device Generating Live Stereo-Soundscapes—Design, Implementation, and Usability Audit

Osinski

Łukowska

Hjelme

et al. 2021

Sensors

View full text Add to dashboard Cite

The successful development of a system realizing color sonification would enable auditory representation of the visual environment. The primary beneficiary of such a system would be people that cannot directly access visual information—the visually impaired community. Despite the plethora of sensory substitution devices, developing systems that provide intuitive color sonification remains a challenge. This paper presents design considerations, development, and the usability audit of a sensory substitution device that converts spatial color information into soundscapes. The implemented wearable system uses a dedicated color space and continuously generates natural, spatialized sounds based on the information acquired from a camera. We developed two head-mounted prototype devices and two graphical user interface (GUI) versions. The first GUI is dedicated to researchers, and the second has been designed to be easily accessible for visually impaired persons. Finally, we ran fundamental usability tests to evaluate the new spatial color sonification algorithm and to compare the two prototypes. Furthermore, we propose recommendations for the development of the next iteration of the system.

show abstract

“…Strong associations were also found between music-color mappings and emotions, as different emotions can be evoked through them [16], and the study also found other musical parameters like tempo and mode can play an important yet unclear role. Prior research on photographic sonification has also yielded several sonification methods and algorithms, adhering to a musical approach focused on musical structure [17] or a naturalistic one following viewing tendencies [18]. Additionally, research on visual saliency in paintings has yielded several algorithms that can imitate human gaze perception and fixations, regardless of art movement the visual artwork belongs to [19].…”

Section: Related Workmentioning

confidence: 99%

Preliminary Guidelines on the Sonification of Visual Artworks: Linking Music, Sonification & Visual Arts

Nadri¹,

Anaya²,

Yuan³

et al. 2019

Proceedings of the 25th International Conference on Auditory Display (ICAD 2019)

View full text Add to dashboard Cite

Sonification and data processing algorithms have advanced over the years to reach practical applications in our everyday life. Similarly, image processing techniques have improved over time. While a number of image sonification methods have already been developed, few have delved into potential synergies through the combined use of multiple data and image processing techniques. Additionally, little has been done on the use of image sonification for artworks, as most research has been focused on the transcription of visual data for people with visual impairments. Our goal is to sonify paintings reflecting their art style and genre to improve the experience of both sighted and visually impaired individuals. To this end, we have designed initial sonifications for paintings of abstractionism and realism, and conducted interviews with visual and auditory experts to improve our mappings. We believe the recommendations and design directions we have received will help develop a multidimensional sonification algorithm that can better transcribe visual art into appropriate music.

show abstract

Musical Vision: an interactive bio-inspired sonification tool to convert images into music

Cited by 15 publications

References 18 publications

Design and Application of Interactive Music Equipment Based on Wireless Wearable Sensors

Design and Application of Interactive Music Equipment Based on Wireless Wearable Sensors

Colorophone 2.0: A Wearable Color Sonification Device Generating Live Stereo-Soundscapes—Design, Implementation, and Usability Audit

Preliminary Guidelines on the Sonification of Visual Artworks: Linking Music, Sonification & Visual Arts

Contact Info

Product

Resources

About