This study presents the initial study for a new approach to visualize an acoustic sound aimed at mimicking the traveling wave propagation of the basilar membrane within the human cochlea. Typically, a fast Fourier transform (FFT) is required to extract the frequency information from acoustic sound (i.e., voice) for speech recognition. Although this algorithm ensures real-time frequency extraction due to the inherent fast recursive structure, it is necessary to develop a new frequency selectivity technique for advanced speech recognition. We explore the potential of the cochlea-inspired sound visualization to deliver new frequency selectivity by using an image sensor. The experimental prototyping model is fabricated, and we capture images of frequency dependent wave propagation motion using a camera and reproduce 2D images through motion magnification. This approach offers a promising application for speech recognition systems because no FFT is required to extract the frequency information, although there are outstanding technical problems that need to be further examined.

In this study, we present initial efforts for a new speech recognition approach aimed at producing different input images for convolutional neural network (CNN)-based speech recognition. We explored the potential of the tympanic membrane (eardrum)-inspired viscoelastic membrane-type diaphragms to deliver audio visualization images using a cross-recurrence plot (CRP). These images were formed by the two phase-shifted vibration responses of viscoelastic diaphragms. We expect this technique to replace the fast Fourier transform (FFT) spectrum currently used for speech recognition. Herein, we report that the new creation method of color images enabled by combining two phase-shifted vibration responses of viscoelastic diaphragms with CRP shows a lower computation burden and a promising potential alternative way to STFT (conventional spectrogram) when the image resolution (pixel size) is below critical resolution.