Patch-based visual microphone for improving quality of sound

Ahn, Juhyun; Kim, Yong-Joong; Kim, Daijin

doi:10.1109/icpr.2016.7900248

Cited by 2 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, future work may be directed at coming up with methods for choosing the best region of interest to perform sound recovery, like those suggested by previous works. 27,34,35 However, this is not in the scope of current work, which is to investigate the impact of color to grayscale conversion and visual enhancement on sound recovery.…”

Section: Discussionmentioning

confidence: 99%

“…Additionally, the 240 and 360 Hz harmonics from the photographic lamp flickering effect at 120 Hz (removed by the bandstop filter) are erroneously considered as a valid signal by all the conversion algorithms. Thus, future work may be directed at coming up with methods for choosing the best region of interest to perform sound recovery, like those suggested by previous works 27,34,35 . However, this is not in the scope of current work, which is to investigate the impact of color to grayscale conversion and visual enhancement on sound recovery.…”

Section: Discussionmentioning

confidence: 99%

“…19 Research has also shown that different methods to convert color images to grayscale and visual enhancement significantly affects the performance of edge detection algorithms. 11 Although the visual microphone has been successful in recovering sound, 2,5,[23][24][25][26][27][28][29] the choice of the most suitable color to grayscale conversion and visual enhancement for extracting local motion and subsequently recovering sound with the visual microphone has not been explored. This work aims to study the influence that various algorithms for converting color to grayscale and performing visual enhancement have on the recovered sound intelligibility using the visual microphone.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Impact of visual enhancement and color conversion algorithms on remote sound recovery from silent videos

Choong,

Yap,

Hum

et al. 2024

J Soc Info Display

View full text Add to dashboard Cite

The visual microphone is a technique for remote sound recovery that extracts sound information from tiny pixel‐scale vibrations in a video. Despite having demonstrated success in sound recovery, the impact of various visual enhancement and color conversion algorithms applied on the video before the sound recovery process has not been explored. Thus, it is important to investigate these effects have on the recovered sound quality, as the vibrations are so small the effects play an important role. This work experimented with different color to grayscale conversions and visual enhancement algorithms on 576 videos, and found that the recovered sound quality is indeed greatly affected by the choice of algorithms. The best conversion algorithms were found to be the average of the red, green and blue color channels and the perceptual lightness in the CIELAB color space, improving the recovered sound quality by up to 23.22%. Furthermore, visual enhancement techniques such as gamma correction have been found to corrupt vibration information, leading to a 22.47% drop in recovered sound quality in one of the tested videos. Therefore, it is advisable to avoid or minimize the use of visual enhancement techniques for remote sound recovery to prevent the elimination of useful subtle vibrations.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%