Making ERP research more transparent: Guidelines for preregistration

Paul, Mariella; Govaart, Gisela; Schettino, Antonio

doi:10.1016/j.ijpsycho.2021.02.016

Cited by 50 publications

(54 citation statements)

References 150 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, we calculated development of the N400 using pre-defined pre-processing steps, recording montages, and researchers' choice of time windows and distribution factors. There are various researchers' degrees of freedom in selecting ways to analyze the data, where researchers often follow lab traditions (Paul et al, 2021 ). Consequently, the results from our systematic review could be biased, as studies did not (nor were they designed to) agree on paradigms, recording settings, pre-processing steps, and quantification of the N400, all of which could affect the physical characteristics of the N400.…”

Section: Discussionmentioning

confidence: 99%

“…For adults, guidelines have been suggested to improve reproducibility and consistency in EEG recordings (Keil et al, 2014 ). For EEG studies it has also been recommended that prior to data collection, researchers preregister their study, in which they outline their hypotheses and proposed analyses plan (Paul et al, 2021 ). A recent set of research standards particularly focused at the N400 component, but this was again aimed at adults, not infants (Šoškić et al, 2020 ).…”

Section: Discussionmentioning

confidence: 99%

“…Moreover, it prevents the field from making clear predictions on how to analyze the infant N400. This would benefit future studies that would like to use pre-registrations or registered reports in which analysis plans need to be thoroughly described prior to data collection (as suggested in Frank et al, 2017a ; Paul et al, 2021 ).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Development of the N400 for Word Learning in the First 2 Years of Life: A Systematic Review

et al. 2021

Self Cite

View full text Add to dashboard Cite

The N400 ERP component is a direct neural index of word meaning. Studies show that the N400 component is already present in early infancy, albeit often delayed. Many researchers capitalize on this finding, using the N400 component to better understand how early language acquisition unfolds. However, variability in how researchers quantify the N400 makes it difficult to set clear predictions or build theory. Not much is known about how the N400 component develops in the first 2 years of life in terms of its latency and topographical distributions, nor do we know how task parameters affect its appearance. In the current paper we carry out a systematic review, comparing over 30 studies that report the N400 component as a proxy of semantic processing elicited in infants between 0 and 24 months old who listened to linguistic stimuli. Our main finding is that there is large heterogeneity across semantic-priming studies in reported characteristics of the N400, both with respect to latency and to distributions. With age, the onset of the N400 insignificantly decreases, while its offset slightly increases. We also examined whether the N400 appears different for recently-acquired novel words vs. existing words: both situations reveal heterogeneity across studies. Finally, we inspected whether the N400 was modulated differently with studies using a between-subject design. In infants with more proficient language skills the N400 was more often present or showed itself here with earlier latency, compared to their peers; but no consistent patterns were observed for distribution characteristics of the N400. One limitation of the current review is that we compared studies that widely differed in choice of EEG recordings, pre-processing steps and quantification of the N400, all of which could affect the characteristics of the infant N400. The field is still missing research that systematically tests development of the N400 using the same paradigm across infancy.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Development of the N400 for Word Learning in the First 2 Years of Life: A Systematic Review

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…What is the statistical problem that preregistration aims to address? Statistically, preregistration is offered as a solution to the problem of using data more than once and issues of validity of statistical procedures resulting from this usage [ 13 , 22 , 92 , 93 ]. Once a hypothesis and an analytical plan is preregistered, the idea is that researchers would be prevented from performing analyses that were not preregistered and subsequently, from presenting them as ‘confirmatory’.…”

Section: Claim 2: Using Data More Than Once Invalidates Statistical Inferencementioning

confidence: 99%

The case for formal methodology in scientific reform

et al. 2021

View full text Add to dashboard Cite

Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (i) that reproducibility is the cornerstone of science; (ii) that data must not be used twice in any analysis; and (iii) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.

show abstract

“…Such an approach can help to reduce researcher degrees of freedom by specifying a researcher's proposed 'path through the garden'. We hope to see ARTEM-IS template complementing other endeavours towards pre-registration in this field (e.g., Paul et al, 2021).…”

Section: From Principles To Action: Artem-is Draft Template For Erpmentioning

confidence: 99%

Towards ARTEM-IS: Design Guidelines for evidencebased EEG methodology reporting tools

Styles¹,

Kovic²,

Ke³

et al. 2021

Preprint

View full text Add to dashboard Cite

As the number of EEG papers increases, so too do the number of guidelines for how to report what has been done. However, current guidelines and checklists appear to have limited adoption, as systematic reviews have shown the journal article format is highly prone to errors, ambiguities and omissions of methodological details. This is a problem for transparency in the scientific record, along with reproducibility, replicability and meta-science. Following lessons learned in the high complexity fields of aviation and surgery, we conclude that new tools are needed to overcome the limitations of current prose methodology descriptions, and that these tools should be developed through community consultation to ensure that they have the best utility for EEG stakeholders. We discuss key features that can support adoption of such tools, and as a first step, we present the ARTEM-IS Statement describing what action will be needed to create an Agreed Reporting Template for EEG Methodology - International Standard (ARTEM-IS). We first launched the statement at the LiveMEEG conference in 2020 along with an evidence-based draft of an ARTEM-IS template for public consultation. Members of the EEG community are invited to join this collective effort to create evidence-based tools that will help make the process of reporting methodology EEG easier, faster and un-get-wrong-able.

show abstract

Making ERP research more transparent: Guidelines for preregistration

Cited by 50 publications

References 150 publications

Development of the N400 for Word Learning in the First 2 Years of Life: A Systematic Review

Development of the N400 for Word Learning in the First 2 Years of Life: A Systematic Review

The case for formal methodology in scientific reform

Towards ARTEM-IS: Design Guidelines for evidencebased EEG methodology reporting tools

Contact Info

Product

Resources

About