Big Data, Exploratory Data Analyses and Questionable Research Practices: Suggestion for a Foundational Principle

Bissonette, John A.

doi:10.1002/wsb.1201

Cited by 3 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HARKing. —A “reproducibility crisis” in science (O'Boyle et al 2017; Fraser et al 2018; Hutson 2018; Parker et al 2019; Bissonette 2021; Nelson et al 2021; O'Dea et al 2021) has reinforced the need for caution surrounding practices such as multiple comparisons and hypothesizing after the results are known (HARKing). When the popular R package bipartite is used with its default settings to study plant–herbivore interactions, the networklevel function calculates 47 bipartite network metrics and the grouplevel function calculates 30 metrics: 15 for each host plant taxon and 15 for each herbivore taxon (Dormann et al 2008)—77 metrics despite few studies addressing 77 distinct questions.…”

Section: Introductionmentioning

confidence: 99%

Linking host plants to damage types in the fossil record of insect herbivory

Schachat¹,

Payne²,

Boyce³

2023

Paleobiology

View full text Add to dashboard Cite

Studies of insect herbivory on fossilized leaves tend to focus on a few, relatively simple metrics that are agnostic to the distribution of insect damage types among host plants. More complex metrics that link particular damage types to particular host plants have the potential to address additional ecological questions, but such metrics can be biased by sampling incompleteness due to the difficulty of distinguishing the true absence of a particular interaction from the failure to detect it—a challenge that has been raised in the ecological literature. We evaluate a range of methods for characterizing the relationships between damage types and host plants by performing resampling and subsampling exercises on a variety of datasets. We found that the components of beta diversity provide a more valid, reliable, and interpretable method for comparing component communities than do bipartite network metrics and that the rarefaction of interactions represent a valid, reliable, and interpretable method for comparing compound communities. Both beta diversity and rarefaction of interactions avoid the potential pitfalls of multiple comparisons. Finally, we found that the host specificity of individual damage types is challenging to assess. Whereas bipartite network metrics are sufficiently biased by sampling incompleteness to be inappropriate for fossil herbivory data, alternatives exist that are perfectly suitable for fossil datasets with sufficient sample coverage.

show abstract

Section: Introductionmentioning

confidence: 99%

Linking host plants to damage types in the fossil record of insect herbivory

Schachat¹,

Payne²,

Boyce³

2023

Paleobiology

View full text Add to dashboard Cite

show abstract

Linking host plants to damage types in the fossil record of insect herbivory

Schachat

Payne

Boyce

2021

Preprint

View full text Add to dashboard Cite

Studies of insect herbivory on fossilized leaves tend to focus on a few, relatively simple metrics that are agnostic to the distribution of insect damage types among host plants. More complex metrics that link particular damage types to particular host plants have the potential to address additional ecological questions, but such metrics can be biased by sampling incompleteness due to the difficulty of distinguishing the true absence of a particular interaction from the failure to detect it—a challenge that has been raised in the ecological literature. We evaluate a range of methods for characterizing the relationships between damage types and host plants by performing resampling and subsampling exercises on a variety of datasets. We found that the components of beta diversity provide a more valid, reliable, and interpretable method for comparing component communities than do bipartite network metrics. We found the rarefaction of interactions to be a valid, reliable, and interpretable method for comparing compound communities. Both beta diversity and rarefaction of interactions avoid the potential pitfalls of multiple comparisons. Lastly, we found that the host specificity of individual damage types is challenging to assess. Whereas some previously used methods are sufficiently biased by sampling incompleteness to be inappropriate for fossil herbivory data, alternatives exist that are perfectly suitable for fossil datasets with sufficient sample coverage.

show abstract

Initial data analysis: A new technology not yet ready to use

Vach

2022

Open Statistics

View full text Add to dashboard Cite

In the ongoing discussion on improving quality and reproducibility of research, the data pipeline had been identified as one crucial element. Everything happening with the data already prior to the final analysis may have a highly relevant impact on the final analysis itself and its interpretation. Consequently, there is an increased focus on these processes. Sound practices for initial data analyses (IDA) have been described and the rigorous use of IDA has been advocated. However, IDA can only have a positive impact on research quality if findings from a systematic data screening lead to reasonable changes in the final analysis or its interpretation. In this communication some general challenges in finding adequate responses to findings from data screening are identified. They reflect a lack of knowledge and consensus. Consequently, rigorous future use of IDA can only be advocated if guidance for responding to data screening findings is further developed. With the knowledge present today, IDA may do more harm than good.

show abstract

Big Data, Exploratory Data Analyses and Questionable Research Practices: Suggestion for a Foundational Principle

Cited by 3 publications

References 39 publications

Linking host plants to damage types in the fossil record of insect herbivory

Linking host plants to damage types in the fossil record of insect herbivory

Linking host plants to damage types in the fossil record of insect herbivory

Initial data analysis: A new technology not yet ready to use

Contact Info

Product

Resources

About