Ten Simple Rules for Digital Data Storage

Hart, Edmund; Barmby, P.; LeBauer, David; Michonneau, François; Mount, Sarah; Mulrooney, Patrick; Poisot, Timothée; Woo, Kara; Zimmerman, Naupaka; Hollister, Jeffrey W.

doi:10.1371/journal.pcbi.1005097

Cited by 84 publications

(69 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Such a "build and smoke test" is particularly helpful when supposedly-innocent changes are being made to the program, or when it has to run on several different machines, e.g., the developer's laptop and the department's cluster. 8. Submit code to a reputable DOI-issuing repository (2j) upon submission of paper, just as you do with data.…”

mentioning

confidence: 99%

Good enough practices in scientific computing

et al. 2017

View full text Add to dashboard Cite

Author summaryComputers are now essential in all branches of science, but most researchers are never taught the equivalent of basic lab skills for research computing. As a result, data can get lost, analyses can take much longer than necessary, and researchers are limited in how effectively they can work with software and data. Computing workflows need to follow the same practices as lab projects and notebooks, with organized data, documented steps, and the project structured for reproducibility, but researchers new to computing often don't know where to start. This paper presents a set of good computing practices that every researcher can adopt, regardless of their current level of computational skill. These practices, which encompass data management, programming, collaborating with colleagues, organizing projects, tracking work, and writing manuscripts, are drawn from a wide variety of published sources from our daily lives and from our work with volunteer organizations that have delivered workshops to over 11,000 people since 2010.

show abstract

mentioning

confidence: 99%

Good enough practices in scientific computing

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Moreover, digital raw data in conjunction with adequate meta-information and appropriate storage is less susceptible to entropy (i.e. decay or degradation Michener 2006;Hart et al 2016) and falsification than data condensations (e.g. spreadsheets) while maintaining data dimensionality.…”

Section: Discussionmentioning

confidence: 99%

A computer vision approach for studying fossorial and cryptic crabs

Herrera

Baker

Sheaves

et al. 2020

Preprint

View full text Add to dashboard Cite

Despite the increasing need to catalogue and describe biodiversity and the ecosystem processes it underpins, these tasks remain inherently challenging. This is particularly true for species that are difficult to observe in their natural environment, such as fossorial and cryptic crabs that inhabit intertidal sediments. Traditional sampling techniques for intertidal crabs are often invasive, labour intensive and/or inconsistent. These factors can limit the amount and type of data that can be collected which in turn hinders our ability to obtain reliable ecological estimates and compare findings between studies. Computer vision and machine learning algorithms present an opportunity to innovate and improve sampling approaches. Moreover, cheaper and tougher recording devices and the diversity of open source software further boost the possibilities of achieving rigorous image-based sampling, which can broaden the range of questions that can be addressed from the data collected. Despite its significant potential, the software and algorithms associated with image-based sampling may be daunting to researchers without expertise in computer vision. Therefore, there is a need to develop protocols and data processing workflows to showcase the value of embracing new technologies. This paper presents a non-invasive computer vision and learning protocol for sampling fossorial and cryptic crabs in their natural environment. The image-based protocol is underpinned by fit-for-purpose and off-the-shelf software. We demonstrate this approach using fiddler crab and sediment recordings to study and quantify crab abundance, motion patterns, behaviour, intraspecific interactions, and estimate bioturbation rates. We discuss current limitations in this protocol and identify opportunities for improvement and additional data stream options that can be obtained from this approach. We conclude that this protocol can overcome some of the limitations associated with traditional techniques for sampling intertidal crabs, and could be applied to other taxa or ecosystems that present similar challenges. We believe this sampling and analytical framework represents an important step forward in understanding the ecology of species and their functional role within ecosystems.

show abstract

“…The size of the raw IMD files often deter operators from storing these data after converting to FCS. However, it is always advisable to keep and organize raw data from experiments , and the files are highly compressible (usually >97% reduction in size). Additionally, the IMD file is typically required for reprocessing of the data.…”

Section: Running the Cytof® Systemmentioning

confidence: 99%

The anatomy of single cell mass cytometry data

et al. 2018

View full text Add to dashboard Cite

Mass cytometry enables the measurement of up to 50 features on single cell. This has catalyzed a shift toward multidimensional data analysis methods, rather than the manual gating strategies as traditionally for in flow cytometry data. This shift means that data scientists are involved in the analysis process to an increasing degree. As the data is analyzed in a more unbiased fashion, where noisy or uninformative observations are not easily excluded, a deeper knowledge of the origin, noise, and modalities of the data is therefore needed to embark on useful data analysis. In this primer, we introduce the idiosyncrasies of mass cytometry data with a focus on the technical properties of how data generated with the CyTOF® system, and the characteristics of protein expression in the cells of the hematopoietic continuum, specifically targeted toward data scientists. We also provide a comprehensive online repository of scripts, tutorials, and example data. © 2018 International Society for Advancement of Cytometry

show abstract

Ten Simple Rules for Digital Data Storage

Cited by 84 publications

References 32 publications

Good enough practices in scientific computing

Good enough practices in scientific computing

A computer vision approach for studying fossorial and cryptic crabs

The anatomy of single cell mass cytometry data

Contact Info

Product

Resources

About