biogitflow: development workflow protocols for bioinformatics pipelines with git and GitLab

Cited by 3 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All DICs validate their data querying scripts. Evidence for validation is provided by manually documenting the queries in a structured and permanently accessible way on GitLab (GitLab Inc) [28] or JIRA [24] (n=5). Unstructured evidence such as the four-eyes principle was practiced in 4 DICs.…”

Section: Documentation Artifacts Of Testing Procedures and Script Val...mentioning

confidence: 99%

The Status of Data Management Practices Across German Medical Data Integration Centers: Mixed Methods Study

Gierend,

Freiesleben,

Kadioglu

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

Background In the context of the Medical Informatics Initiative, medical data integration centers (DICs) have implemented complex data flows to transfer routine health care data into research data repositories for secondary use. Data management practices are of importance throughout these processes, and special attention should be given to provenance aspects. Insufficient knowledge can lead to validity risks and reduce the confidence and quality of the processed data. The need to implement maintainable data management practices is undisputed, but there is a great lack of clarity on the status. Objective Our study examines the current data management practices throughout the data life cycle within the Medical Informatics in Research and Care in University Medicine (MIRACUM) consortium. We present a framework for the maturity status of data management practices and present recommendations to enable a trustful dissemination and reuse of routine health care data. Methods In this mixed methods study, we conducted semistructured interviews with stakeholders from 10 DICs between July and September 2021. We used a self-designed questionnaire that we tailored to the MIRACUM DICs, to collect qualitative and quantitative data. Our study method is compliant with the Good Reporting of a Mixed Methods Study (GRAMMS) checklist. Results Our study provides insights into the data management practices at the MIRACUM DICs. We identify several traceability issues that can be partially explained with a lack of contextual information within nonharmonized workflow steps, unclear responsibilities, missing or incomplete data elements, and incomplete information about the computational environment information. Based on the identified shortcomings, we suggest a data management maturity framework to reach more clarity and to help define enhanced data management strategies. Conclusions The data management maturity framework supports the production and dissemination of accurate and provenance-enriched data for secondary use. Our work serves as a catalyst for the derivation of an overarching data management strategy, abiding data integrity and provenance characteristics as key factors. We envision that this work will lead to the generation of fairer and maintained health research data of high quality.

show abstract

Section: Documentation Artifacts Of Testing Procedures and Script Val...mentioning

confidence: 99%

The Status of Data Management Practices Across German Medical Data Integration Centers: Mixed Methods Study

Gierend,

Freiesleben,

Kadioglu

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…This way, it is easier to maintain many different pipelines on one hand and it simplifies the skill transfer from one person to another on the other hand. We developed the biogitflow protocols which cover the code versioning using git and GitLab ( Kamoun et al, 2020 ). However, these protocols do not address how to write the code itself.…”

Section: Introductionmentioning

confidence: 99%

Geniac: Automatic Configuration GENerator and Installer for nextflow pipelines

Allain¹,

Roméjon²,

Rosa³

et al. 2022

Open Res Europe

Self Cite

View full text Add to dashboard Cite

With the advent of high-throughput biotechnological platforms and their ever-growing capacity, life science has turned into a digitized, computational and data-intensive discipline. As a consequence, standard analysis with a bioinformatics pipeline in the context of routine production has become a challenge such that the data can be processed in real-time and delivered to the end-users as fast as possible. The usage of workflow management systems along with packaging systems and containerization technologies offer an opportunity to tackle this challenge. While very powerful, they can be used and combined in many multiple ways which may differ from one developer to another. Therefore, promoting the homogeneity of the workflow implementation requires guidelines and protocols which detail how the source code of the bioinformatics pipeline should be written and organized to ensure its usability, maintainability, interoperability, sustainability, portability, reproducibility, scalability and efficiency. Capitalizing on Nextflow, Conda, Docker, Singularity and the nf-core initiative, we propose a set of best practices along the development life cycle of the bioinformatics pipeline and deployment for production operations which target different expert communities including i) the bioinformaticians and statisticians ii) the software engineers and iii) the data managers and core facility engineers. We implemented Geniac (Automatic Configuration GENerator and Installer for nextflow pipelines) which consists of a toolbox with three components: i) a technical documentation available at https://geniac.readthedocs.io to detail coding guidelines for the bioinformatics pipeline with Nextflow, ii) a command line interface with a linter to check that the code respects the guidelines, and iii) an add-on to generate configuration files, build the containers and deploy the pipeline. The Geniac toolbox aims at the harmonization of development practices across developers and automation of the generation of configuration files and containers by parsing the source code of the Nextflow pipeline.

show abstract

Geniac: Automatic Configuration GENerator and Installer for nextflow pipelines

Allain¹,

Roméjon²,

Rosa³

et al. 2021

Open Res Europe

View full text Add to dashboard Cite

With the advent of high-throughput biotechnological platforms and their ever-growing capacity, life science has turned into a digitized, computational and data-intensive discipline. As a consequence, standard analysis with a bioinformatics pipeline in the context of routine production has become a challenge such that the data can be processed in real-time and delivered to the end-users as fast as possible. The usage of workflow management systems along with packaging systems and containerization technologies offer an opportunity to tackle this challenge. While very powerful, they can be used and combined in multiple ways thus increasing their usage complexity. Therefore, guidelines and protocols are required in order to detail how the source code of the bioinformatics pipeline should be written and organized to ensure its usability, maintainability, interoperability, sustainability, portability, reproducibility, scalability and efficiency. Capitalizing on Nextflow, Conda, Docker, Singularity and the nf-core initiative, we propose a set of best practices along the development life cycle of the bioinformatics pipeline and deployment for production operations which address different expert communities including i) the bioinformaticians and statisticians ii) the software engineers and iii) the data managers and core facility engineers. We implemented Geniac (Automatic Configuration GENerator and Installer for nextflow pipelines) which consists of a toolbox with three components: i) a technical documentation available at https://geniac.readthedocs.io to detail coding guidelines for the bioinformatics pipeline with Nextflow, ii) a linter to check that the code respects the guidelines, and iii) an add-on to generate configuration files, build the containers and deploy the pipeline. The Geniac toolbox aims at the harmonization of development practices across developers and automation of the generation of configuration files and containers by parsing the source code of the Nextflow pipeline. The Geniac toolbox and two demo pipelines are available on GitHub. This article presents the main functionalities of Geniac.

show abstract

biogitflow: development workflow protocols for bioinformatics pipelines with git and GitLab

Cited by 3 publications

References 13 publications

The Status of Data Management Practices Across German Medical Data Integration Centers: Mixed Methods Study

The Status of Data Management Practices Across German Medical Data Integration Centers: Mixed Methods Study

Geniac: Automatic Configuration GENerator and Installer for nextflow pipelines

Geniac: Automatic Configuration GENerator and Installer for nextflow pipelines

Contact Info

Product

Resources

About