Semantic translation of face image with limited pixels for simulated prosthetic vision

Xia, Xuan; He, Xiangming; Liu, Feng; Pan, Xizhou; Li, Nan; Zhang, Jingfei; Pang, Xufang; Yu, Fengqi; Ding, Ning

doi:10.1016/j.ins.2022.07.094

Cited by 6 publications

(2 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then the researchers built a pixel face database designed for visual prostheses. The experiment showcased that the model has better recognizability in features and expressions than existing technologies [15]. The optic nerve was the first choice for the placement of visual prostheses, and the effectiveness of visual prostheses was closely related to the combination of stimulus parameters.…”

Section: Related Workmentioning

confidence: 94%

Improved VGG Algorithm for Visual Prosthesis Image Recognition

Li,

Jahng

et al. 2024

IEEE Access

View full text Add to dashboard Cite

With the development of bio-informatics, visual prostheses have become an effective method for blind people to restore visual function. To meet the visual needs of visual implant recipients, this study explored the problem of in vitro image processor image recognition and classification. It selected the convolutional neural network framework VGG as the technical core, introduced the fruit fly optimization algorithm for optimizing the VGG recognition model, and constructed a visual prosthesis image recognition model on the ground of improved VGG. The experiment demonstrated that the improved fruit fly search algorithm had an average absolute error and root mean square error values lower than 0.4, which was superior to other intelligent optimization algorithms. The performance of the improved image recognition model has been significantly improved, with a maximum AUC value of 0.942, a recognition accuracy range of 68.29%-97.23%, and a stable fitness curve of around 97.00. The maximum F1 value of the image recognition model designed in the study reached 91.47%, and the loss function curve converged to the minimum value. In the application of visual prostheses, the recognition accuracy and R-squared performance of this model were both the best. Compared with natural human vision, the contrast and functional visual effects matched well, the processing speed was faster, and the delay time did not affect the actual application of visual prostheses, achieving high user satisfaction. This study can enrich and improve the theoretical foundation of visual image analysis and visual prosthetics, and help visually impaired groups improve their lives and quality of life.

show abstract

Section: Related Workmentioning

confidence: 94%

Improved VGG Algorithm for Visual Prosthesis Image Recognition

Li,

Jahng

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…image [39], end-to-end optimization [40], semantic face image translation [41], and electrode-retina distance [6] to supplement the previously un-simulated features or to add new features to current prosthetic vision techniques. This kind of also happened in other prosthetic fields, e.g., prosthetic arm and hand [42].…”

Section: B Neuroprothesis Simulatorsmentioning

confidence: 99%

Comparable Encoding, Comparable Perceptual Pattern: Acoustic and Electric Hearing

Kong

Zhou

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Perception with electric neuroprostheses is sometimes expected to be simulated using properly designed physical stimuli. Here, we examined a new acoustic vocoder model for electric hearing with cochlear implants (CIs) and hypothesized that comparable speech encoding can lead to comparable perceptual patterns for CI and normal hearing (NH) listeners. Speech signals were encoded using FFT-based signal processing stages including band-pass filtering, temporal envelope extraction, maxima selection, and amplitude compression and quantization. These stages were specifically implemented in the same manner by an Advanced Combination Encoder (ACE) strategy in CI processors and Gaussian-enveloped Tones (GET) or Noise (GEN) vocoders for NH. Adaptive speech reception thresholds (SRTs) in noise were measured using four Mandarin sentence corpora. Initial consonant (11 monosyllables) and final vowel (20 monosyllables) recognition were also measured. NaÏve NH listeners were tested using vocoded speech with the proposed GET/GEN vocoders as well as conventional vocoders (controls).Experienced CI listeners were tested using their daily-used processors. Results showed that: 1) there was a significant training effect on GET vocoded speech perception; 2) the GEN vocoded scores (SRTs with four corpora and consonant and vowel recognition scores) as well as the phoneme-level confusion pattern matched with the CI scores better than controls. The findings suggest that the same signal encoding implementations may lead to similar perceptual patterns simultaneously in multiple perception Manuscript

show abstract