Reading Imagined Letter Shapes from the Mind’s Eye Using Real-time 7 Tesla fMRI

Goebel, Rainer; Hoof, Rick van; Bhat, Salil; Lührs, Michael; Senden, Mario

doi:10.1109/bci53720.2022.9735031

Cited by 6 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the explicit link between the modeled area of cortical activation and the simulated phosphene sizes and locations makes our software very suitable for including new receptive field modeling results. Collaborative international projects such as the PRIMatE Resource Exchange (PRIME-RE) offer advanced tools which allow to fit probabilistic retinotopic maps generated from large samples to any individual NHP brain (61, 62) and it is currently possible to accurately predict human cortical receptive field mapping based on anatomical scans (63, 64), or other retinotopic mapping strategies that do not rely on visual input (65, 66). This opens new doors for future research into the functionality of visual prostheses with limited visual field coverage.…”

Section: Discussionmentioning

confidence: 99%

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Grinten¹,

Steveninck²,

Lozano

et al. 2022

Preprint

View full text Add to dashboard Cite

Blindness affects millions of people around the world, and is expected to become increasingly prevalent in the years to come. For some blind individuals, a promising solution to restore a form of vision are cortical visual prostheses, which convert camera input to electrical stimulation of the cortex to bypass part of the impaired visual system. Due to the constrained number of electrodes that can be implanted, the artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') is of limited resolution, and a great portion of the field's research attention is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is the non-invasive functional evaluation in sighted subjects or with computational models by making use of simulated prosthetic vision (SPV) pipelines. Although the SPV literature has provided us with some fundamental insights, an important drawback that researchers and clinicians may encounter is the lack of realism in the simulation of cortical prosthetic vision, which limits the validity for real-life applications. Moreover, none of the existing simulators address the specific practical requirements for the electrical stimulation parameters. In this study, we developed a PyTorch-based, fast and fully differentiable phosphene simulator. Our simulator transforms specific electrode stimulation patterns into biologically plausible representations of the artificial visual percepts that the prosthesis wearer is expected to see. The simulator integrates a wide range of both classical and recent clinical results with neurophysiological evidence in humans and non-human primates. The implemented pipeline includes a model of the retinotopic organisation and cortical magnification of the visual cortex. Moreover, the quantitative effect of stimulation strength, duration, and frequency on phosphene size and brightness as well as the temporal characteristics of phosphenes are incorporated in the simulator. Our results demonstrate the suitability of the simulator for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioural experiments. The modular approach of our work makes it ideal for further integrating new insights in artificial vision as well as for hypothesis testing. In summary, we present an open-source, fully differentiable, biologically plausible phosphene simulator as a tool for computational, clinical and behavioural neuroscientists working on visual neuroprosthetics.

show abstract

Section: Discussionmentioning

confidence: 99%

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Grinten¹,

Steveninck²,

Lozano

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Furthermore, the explicit link between the modeled area of cortical activation and the simulated phosphene sizes and locations makes our software very suitable for including new receptive field modeling results. Collaborative international projects such as the PRI-MatE Resource Exchange (PRIME-RE) offer advanced tools which allow to fit probabilistic retinotopic maps generated from large samples to any individual NHP brain (61,62) and it is currently possible to accurately predict human cortical receptive field mapping based on anatomical scans (63,64), or other retinotopic mapping strategies that do not rely on visual input (65,66). This opens new doors for future research into the functionality of visual prostheses with limited visual field coverage.…”

Section: K1 Visuotopic Mappingmentioning

confidence: 99%

Identification of novel coloboma candidate genes through conserved gene expression analyses across four vertebrate species

Trejo-Reveles

Owen²,

Chan

et al. 2022

Preprint

View full text Add to dashboard Cite

Ocular coloboma (OC) is a failure of complete optic fissure closure during embryonic development and presents as a tissue defect along the proximal distal axis of the ventral eye. It is classed as part of the clinical spectrum of structural eye malformations with microphthalmia and anophthalmia, collectively abbreviated to MAC. Despite deliberate attempts to identify causative variants in MAC, many patients remain without a genetic diagnosis. To reveal potential candidate genes, we utilised transcriptomes experimentally generated from embryonic eye tissues derived from human, mouse, zebrafish, and chicken at stages coincident with optic fissure closure. Our in-silico analyses found 10 genes with optic fissure specific enriched expression: ALDH1A3, BMPR1B, EMX2, EPHB3, NID1, NTN1, PAX2, SMOC1, TENM3, and VAX1. In situ hybridization revealed that all 10 genes were broadly expressed ventrally in the developing eye, but that only PAX2 and NTN1 were expressed in cells at the edges of the optic fissure margin. Of these conserved optic fissure genes, EMX2, NID1, and EPHB3 have not previously been associated with human MAC cases. Targeted genetic manipulation in zebrafish embryos using CRISPR/Cas9 caused the developmental MAC phenotype for emx2 and ephb3. We scrutined available whole genome sequencing datasets from MAC patients and identified a range of variants with plausible causality. In combination our data suggest that expression of genes involved in ventral eye development are conserved across a range of vertebrate species, and that EMX2, NID1, and EPHB3 are candidate loci that should be adopted into clinical diagnostic screens for patients with structural eye malformations.

show abstract

“…This online mapping combined with reconstruction of perceived and mental imagery (Thirion et al, 2006;Senden et al, 2019) can pave the way for brain computer interfaces (BCIs). Some work has already been done towards such BCIs by Goebel et al (2022). Such a BCI can be, in future, helpful for locked-in patients.…”

Section: Population Receptive Fields (Prf) Mappingmentioning

confidence: 99%

“…We show that, on a regular workstation, it is possible to map millions of voxels within seconds. In a proof-of-concept study (Goebel et al, 2022), our approach was used to map pRFs in real-time, followed by their inversion for reconstructing imagined stimuli. In terms b 75 b of estimating the pRFs and their parameters, we validate our approach on both simulated data (with ground-truth) and empirical data.…”

Section: Extremely Fast Prf Mappingmentioning

confidence: 99%

“…This can largely impact development of brain-computer interfaces (BCIs) for helping lockedin patients. In a recent proof-of-concept work by Goebel et al (2022), it was demonstrated that such an online pRF mapping can be used to reconstruct imaged letters in real-time. This can pave a way for developing a letter speller BCI which can, in realtime, reconstruct imagined letters and display them back to the participants, in MRI scanners, as feedback.…”

Section: Connective Field Modellingmentioning

confidence: 99%

See 1 more Smart Citation

The efficient cause of science

Bhat

View full text Add to dashboard Cite

published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the "Taverne" license above, please follow below link for the End User

show abstract

Reading Imagined Letter Shapes from the Mind’s Eye Using Real-time 7 Tesla fMRI

Cited by 6 publications

References 5 publications

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses

Identification of novel coloboma candidate genes through conserved gene expression analyses across four vertebrate species

The efficient cause of science

Contact Info

Product

Resources

About