Phylter: A System for Modulating Notifications in Wearables Using Physiological Sensing

Afergan, Daniel; Hincks, Samuel W.; Shibata, Tomoki; Jacob, Robert J. K.

doi:10.1007/978-3-319-20816-9_17

Cited by 11 publications

(6 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent work has shown that EEG-based BCIs can successfully be used in combination with new technological developments such as AR to improve real-world practicality by offering a richer, more direct, and intuitive interface (Kansaku et al, 2010;Takano et al, 2011;Borges et al, 2016;Faller et al, 2017). However, very few fNIRS applications have explored this option (Afergan et al, 2015;McKendrick et al, 2016;Si-Mohammed et al, 2018;Hu et al, 2019). In the present study, we employed an AR cube to guide the temporal-encoding approach and to display the decoded answer of participants' intentions.…”

Section: Using Ar In Bcis Offers a Great Flexibilitymentioning

confidence: 99%

“…The cortical activity was superimposed onto a participant's head in the real world in real-time through an OST-HMD (HoloLens) device the clinician was wearing. Afergan et al (2015) developed an fNIRSbased BCI using OST-HMD called Phylter. They developed a control system connected to Google Glass that helped preventing the user from getting flooded by notifications.…”

Section: Introductionmentioning

confidence: 99%

“…Although the application of BCIs has been limited primarily to a laboratory setting, some of the studies mentioned above have examined the possibility of using BCI in everyday-life settings in different contexts (Takano et al, 2011;Blum et al, 2012;Afergan et al, 2015;Hu et al, 2019;Park et al, 2019). However, ecologically valid approaches are challenging to develop as, among other reasons, they should be as efficient, accurate and reliable as possible, but also easy to use, intuitive, and simple to (dis)assemble.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

et al. 2020

View full text Add to dashboard Cite

Augmented reality (AR) enhances the user's environment by projecting virtual objects into the real world in real-time. Brain-computer interfaces (BCIs) are systems that enable users to control external devices with their brain signals. BCIs can exploit AR technology to interact with the physical and virtual world and to explore new ways of displaying feedback. This is important for users to perceive and regulate their brain activity or shape their communication intentions while operating in the physical world. In this study, twelve healthy participants were introduced to and asked to choose between two motorimagery tasks: mental drawing and interacting with a virtual cube. Participants first performed a functional localizer run, which was used to select a single fNIRS channel for decoding their intentions in eight subsequent choice-encoding runs. In each run participants were asked to select one choice of a six-item list. A rotating AR cube was displayed on a computer screen as the main stimulus, where each face of the cube was presented for 6 s and represented one choice of the six-item list. For five consecutive trials, participants were instructed to perform the motor-imagery task when the face of the cube that represented their choice was facing them (therewith temporally encoding the selected choice). In the end of each run, participants were provided with the decoded choice based on a joint analysis of all five trials. If the decoded choice was incorrect, an active error-correction procedure was applied by the participant. The choice list provided in each run was based on the decoded choice of the previous run. The experimental design allowed participants to navigate twice through a virtual menu that consisted of four levels if all choices were correctly decoded. Here we demonstrate for the first time that by using AR feedback and flexible choice encoding in form of search trees, we can increase the degrees of freedom of a BCI system. We also show that participants can successfully navigate through a nested menu and achieve a mean accuracy of 74% using a single motor-imagery task and a single fNIRS channel.

show abstract

Section: Using Ar In Bcis Offers a Great Flexibilitymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In high-consequence settings, physiological-based interventions such as biofeedback often rely on HRV and are associated with improved performance [ 27 ]. Furthermore, the sensitivity of NIRS data affords the opportunity to determine the optimal time to provide notifications or interruptions along the course of a primary task [ 28 ], which has otherwise been shown to increase error, time to completion, annoyance, and anxiety [ 29 ]. This has similarly been demonstrating using HRV data [ 30 ] during real-world cardiac surgeries.…”

Section: Discussionmentioning

confidence: 99%

Sensors for Continuous Monitoring of Surgeon’s Cognitive Workload in the Cardiac Operating Room

Kennedy-Metz

Dias

Srey

et al. 2020

Sensors

View full text Add to dashboard Cite

Monitoring healthcare providers’ cognitive workload during surgical procedures can provide insight into the dynamic changes of mental states that may affect patient clinical outcomes. The role of cognitive factors influencing both technical and non-technical skill are increasingly being recognized, especially as the opportunities to unobtrusively collect accurate and sensitive data are improving. Applying sensors to capture these data in a complex real-world setting such as the cardiac surgery operating room, however, is accompanied by myriad social, physical, and procedural constraints. The goal of this study was to investigate the feasibility of overcoming logistical barriers in order to effectively collect multi-modal psychophysiological inputs via heart rate (HR) and near-infrared spectroscopy (NIRS) acquisition in the real-world setting of the operating room. The surgeon was outfitted with HR and NIRS sensors during aortic valve surgery, and validation analysis was performed to detect the influence of intra-operative events on cardiovascular and prefrontal cortex changes. Signals collected were significantly correlated and noted intra-operative events and subjective self-reports coincided with observable correlations among cardiovascular and cerebral activity across surgical phases. The primary novelty and contribution of this work is in demonstrating the feasibility of collecting continuous sensor data from a surgical team member in a real-world setting.

show abstract

“…Therefore, with respect to other functional neuroimaging methods such as fMRI, fNIRS represents an useful tool for neuroergonomics research (for review, see Ayaz et al, 2013 ; Derosière et al, 2013 ), for studies in other fields of neuroscience such as brain-computer interface (for review, see Naseer and Hong, 2015 ), human-robot interaction (for review, see Canning and Scheutz, 2013 ), and cognitive states measurements (for review, see Strait and Scheutz, 2014 ). In addition, very recently the integration of fNIRS with a wearable technology, such as Google Glass, has been demonstrated (Afergan et al, 2015 ). For an adequate understanding of the current findings, some limitations should be pointed out: (1) this study has been conducted in a small sample of healthy young male adults subjects and the subjective cognitive load was not tested by NASA Task Load Index; (2) the duration of the adopted task and the length of the route were relatively short, hence, the effect of a longer duration of the VR HCT on the PFC hemodynamic response remains unknown; (3) this study did not imply a control session for example including/not including motor task with/without VR; (4) this study did not contemplate repeated trials on separate days in order to verify the reproducibility and the potential learning effect of the HCT; (5) the limited number of measurement points (16) made possible by the utilized fNIRS system did not allow the investigation of the supposed connectivity between the PFC and other cortical areas (e.g., premotor and motor cortices) likely involved in performing HCT; and (6) this study did not take into account the impact of the variability of the skull thickness amongst the 16 measurement points within subject and amongst subjects.…”

Section: Discussionmentioning

confidence: 99%

Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics

Carrieri

Petracca

Lancia

et al. 2016

Front. Hum. Neurosci.

View full text Add to dashboard Cite

Functional near-infrared spectroscopy (fNIRS) is a non-invasive vascular-based functional neuroimaging technology that can assess, simultaneously from multiple cortical areas, concentration changes in oxygenated-deoxygenated hemoglobin at the level of the cortical microcirculation blood vessels. fNIRS, with its high degree of ecological validity and its very limited requirement of physical constraints to subjects, could represent a valid tool for monitoring cortical responses in the research field of neuroergonomics. In virtual reality (VR) real situations can be replicated with greater control than those obtainable in the real world. Therefore, VR is the ideal setting where studies about neuroergonomics applications can be performed. The aim of the present study was to investigate, by a 20-channel fNIRS system, the dorsolateral/ventrolateral prefrontal cortex (DLPFC/VLPFC) in subjects while performing a demanding VR hand-controlled task (HCT). Considering the complexity of the HCT, its execution should require the attentional resources allocation and the integration of different executive functions. The HCT simulates the interaction with a real, remotely-driven, system operating in a critical environment. The hand movements were captured by a high spatial and temporal resolution 3-dimensional (3D) hand-sensing device, the LEAP motion controller, a gesture-based control interface that could be used in VR for tele-operated applications. Fifteen University students were asked to guide, with their right hand/forearm, a virtual ball (VB) over a virtual route (VROU) reproducing a 42 m narrow road including some critical points. The subjects tried to travel as long as possible without making VB fall. The distance traveled by the guided VB was 70.2 ± 37.2 m. The less skilled subjects failed several times in guiding the VB over the VROU. Nevertheless, a bilateral VLPFC activation, in response to the HCT execution, was observed in all the subjects. No correlation was found between the distance traveled by the guided VB and the corresponding cortical activation. These results confirm the suitability of fNIRS technology to objectively evaluate cortical hemodynamic changes occurring in VR environments. Future studies could give a contribution to a better understanding of the cognitive mechanisms underlying human performance either in expert or non-expert operators during the simulation of different demanding/fatiguing activities.

show abstract

Phylter: A System for Modulating Notifications in Wearables Using Physiological Sensing

Cited by 11 publications

References 37 publications

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

An Augmented-Reality fNIRS-Based Brain-Computer Interface: A Proof-of-Concept Study

Sensors for Continuous Monitoring of Surgeon’s Cognitive Workload in the Cardiac Operating Room

Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics

Contact Info

Product

Resources

About