Learning to deal with delayed outcomes: EEG oscillatory and slow potentials during the prefeedback interval

Bhangal, Sabrina; Sharma, Shashikant; Valle-Inclán, Fernando; Ren, Xi; Hackley, Steven A.

doi:10.1111/psyp.13853

Cited by 4 publications

(5 citation statements)

References 58 publications

(114 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The half-sessions can be considered as two smaller sessions performed one immediately after another. Such treatment of half-sessions as consecutive measurements has been used in EEG analysis in various contexts [56][57][58][59]. Repeatability between such measurements can be considered as the minimum requirement, indicating that the frequencies of maximal coupling are stable at least at the time scale of the session, and could be meaningfully used to guide neuromodulation performed shortly after the EEG measurement.…”

Section: Reproducibility Between Measurements: Characteristics and St...mentioning

confidence: 99%

Estimating the Frequencies of Maximal Theta-Gamma Coupling in EEG during the N-Back Task: Sensitivity to Methodology and Temporal Instability

Sinitsyn,

Poydasheva,

Bakulin

et al. 2023

Algorithms

View full text Add to dashboard Cite

Phase-amplitude coupling (PAC) of theta and gamma rhythms of the brain has been observed in animals and humans, with evidence of its involvement in cognitive functions and brain disorders. This motivates finding individual frequencies of maximal theta-gamma coupling (TGC) and using them to adjust brain stimulation. This use implies the stability of the frequencies at least during the investigation, which has not been sufficiently studied. Meanwhile, there is a range of available algorithms for PAC estimation in the literature. We explored several options at different steps of the calculation, applying the resulting algorithms to the EEG data of 16 healthy subjects performing the n-back working memory task, as well as a benchmark recording with previously reported strong PAC. By comparing the results for the two halves of each session, we estimated reproducibility at a time scale of a few minutes. For the benchmark data, the results were largely similar between the algorithms and stable over time. However, for the EEG, the results depended substantially on the algorithm, while also showing poor reproducibility, challenging the validity of using them for personalizing brain stimulation. Further research is needed on the PAC estimation algorithms, cognitive tasks, and other aspects to reliably determine and effectively use TGC parameters in neuromodulation.

show abstract

Section: Reproducibility Between Measurements: Characteristics and St...mentioning

confidence: 99%

Estimating the Frequencies of Maximal Theta-Gamma Coupling in EEG during the N-Back Task: Sensitivity to Methodology and Temporal Instability

Sinitsyn,

Poydasheva,

Bakulin

et al. 2023

Algorithms

View full text Add to dashboard Cite

show abstract

“…Because the SPN is produced by the anticipation of motivationally relevant stimuli, such as feedback on one's performance, it may be a useful tool in understanding feedback learning (Brunia et al, 2011), including credit assignment (Bhangal et al, 2021;Ren et al, 2017). A trait of the SPN that is particularly relevant to this thesis is that it has larger amplitudes if participants believe they have more control or involvement over their outcomes (Kotani et al, 2017;Masaki et al, 2010).…”

Section: The Stimulus-preceding Negativity (Spn)mentioning

confidence: 99%

“…Note. Time-frequency analysis example taken from Bhangal et al (2021). During the time when a subject was making a movement, alpha was suppressed (blue), followed by alpha activation (red) immediately after the cessation of the movement while the subject awaited feedback on their performance.…”

Section: Example Of Time-frequency Analysis Plotmentioning

confidence: 99%

Perceived agency and the pre-feedback delay: a time-frequency analysis

Hicks¹

View full text Add to dashboard Cite

In everyday life, it is often the case that an individual must reflect on which, if any, of their actions led to a favorable or unfavorable outcome, especially if a sizeable period of time passed prior to knowing the outcome. If they attribute these outcomes to themselves, rather than to some external source, they would be said to have a "sense of agency." As of this paper, only event-related potentials and fMRI methods have been used to assess this phenomenon. In this experiment, we re-analyzed the data from a previous SPN study using a more powerful analytic method, time-frequency analysis, and tested a variety of hypotheses concerning previously established relationships between oscillatory activity and brain regions involved in feedback anticipation. Participants took part in a gambling task where their "sense of agency" and the valence of the expected outcome (potential loss versus potential gain) were manipulated. In choice trials, it was implied that participants' decisions influenced whether they won or lost money, while in no-choice trials, participants were correctly informed that the computer determined these outcomes. We found that pre-feedback alpha suppression in sites overlying right frontal and occipital areas was stronger on choice trials. Pre-feedback beta suppression in sites overlying left frontal sites was also greater on these trials. Both of these effects appeared to be modulated by contextual valence--effects were significantly larger in conditions with potential gain than those involving potential loss. These findings highlight the importance of selective attention and reward anticipation when the outcome of one's own action is awaited.||In everyday life, it is often the case that an individual must reflect on which, if any, of their actions led to a favorable or unfavorable outcome, especially if a sizeable period of time passed prior to knowing the outcome. If they attribute these outcomes to themselves, rather than to some external source, they would be said to have a "sense of agency." As of this paper, only event-related potentials and fMRI methods have been used to assess this phenomenon. In this experiment, we re-analyzed the data from a previous SPN study using a more powerful analytic method, time-frequency analysis, and tested a variety of hypotheses concerning previously established relationships between oscillatory activity and brain regions involved in feedback anticipation. Participants took part in a gambling task where their "sense of agency" and the valence of the expected outcome (potential loss versus potential gain) were manipulated. In choice trials, it was implied that participants' decisions influenced whether they won or lost money, while in no-choice trials, participants were correctly informed that the computer determined these outcomes. We found that pre-feedback alpha suppression in sites overlying right frontal and occipital areas was stronger on choice trials. Pre-feedback beta suppression in sites overlying left frontal sites was also greater on these trials. Both of these effects appeared to be modulated by contextual valence--effects were significantly larger in conditions with potential gain than those involving potential loss. These findings highlight the importance of selective attention and reward anticipation when the outcome of one's own action is awaited.

show abstract

“…Using the data from the published analysis (Bhangal et al, 2021), Figure 7 shows that there was no decline found in the P3 amplitude across either subblocks or tasks. We expected that aggregating trials with correct and incorrect responses would distort feedback-related components.…”

Section: Feedback-p3mentioning

confidence: 99%

“…The experiment summarized in Chapter 2 (Bhangal et al, 2021), attempted to determine whether subjects hold in mind a representation of a sequence of actions as they await feedback. If this representation is held in WM, its neural signature should diminish as the sequence is reduced toward a single chunk during automatization.…”

Section: Introductionmentioning

confidence: 99%

Working memory in reinforcement learning

Bhangal¹

View full text Add to dashboard Cite

The ability of an individual to learn from delayed action outcomes helps them to better adapt to their environment. In order for learning to take place, they might utilize working memory to hold in mind a response they have just made and compare it to feedback received. This dissertation encompasses three experiments that test several putative neural correlates of working memory during the prefeedback delay. In the first experiment, I show that two EEG measures, the stimulus-preceding negativity (SPN) and lateralized theta, decline in amplitude over the course of learning a new motor skill. In the second experiment, I show that SPN and frontocentral beta are of greater amplitude in a condition that imposes a high load on working memory for actions. In the third experiment, I show that SPN and frontocentral beta increased progressively across conditions with one, two, or three cues in a probabilistic categorization task. This effect was greatly reduced in participants whose ability to develop complex hypotheses regarding the relation of stimuli, responses, and outcomes was impaired by Parkinson's Disease. By adopting three different approaches to testing the relationship of EEG measures to the brain's ability to bridge the gap between responses and outcomes, this dissertation provides converging evidence regarding the role of working memory.

show abstract

Learning to deal with delayed outcomes: EEG oscillatory and slow potentials during the prefeedback interval

Cited by 4 publications

References 58 publications

Estimating the Frequencies of Maximal Theta-Gamma Coupling in EEG during the N-Back Task: Sensitivity to Methodology and Temporal Instability

Estimating the Frequencies of Maximal Theta-Gamma Coupling in EEG during the N-Back Task: Sensitivity to Methodology and Temporal Instability

Perceived agency and the pre-feedback delay: a time-frequency analysis

Working memory in reinforcement learning

Contact Info

Product

Resources

About