Scalability and cost-effectiveness analysis of whole genome-wide association studies on Google Cloud Platform and Amazon Web Services

Krissaane, Inès; Niz, Carlos De; Gutiérrez-Sacristán, Alba; Korodi, Gabor; Ede, Nneka; Kumar, Ranjay; Lyons, Jessica B.; Manrai, Arjun K.; Patel, Chirag; Kohane, Isaac S.; Avillach, Paul

doi:10.1093/jamia/ocaa068

Cited by 18 publications

(14 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hence, we have used technologies like Terraform (for cloud infrastructure provisioning, Modi, 2021), Helm (for deploying applications on Kubernetes clusters, Shah and Dubaria, 2019) and Docker (for application code packaging and shipment, Boettiger, 2015). SeQuiLa has been successfully deployed to both popular managed Hadoop services like Google Dataproc (utilized also in Krissaane et al ., 2020 and managed Kubernetes services like Google Kubernetes Engine (GKE) or Azure Kubernetes Service. Figure 3 presents an exemplary setup on GKE using the spark-on-k8s-operator and SeQuiLa application defined as a Kubernetes Custom Resource Definition.…”

Section: Methodsmentioning

confidence: 99%

Cloud-native distributed genomic pileup operations

Wiewiórka

Szmurło

Stankiewicz

et al. 2022

Preprint

View full text Add to dashboard Cite

Motivation: Pileup analysis is a building block of many bioinformatics pipelines, including variant calling and genotyping. This step tends to become a bottleneck of the entire assay since the straightforward pileup implementations involve processing of all base calls from all alignments sequentially. On the other hand, a distributed version of the algorithm faces the intrinsic challenge of splitting reads-oriented file formats into self-contained partitions to avoid costly data exchange between computation nodes. Results: Here, we present a scalable, distributed, and efficient implementation of a pileup algorithm that is suitable for deploying in cloud computing environments. In particular, we implemented: (i) our custom data-partitioning algorithm optimized to work with the alignment reads, (ii) a novel and unique approach to process alignment events from sequencing reads using the MD tags, (iii) the source code micro-optimizations for recurrent operations, and (iv) a modular structure of the algorithm. We have proven that our novel approach consistently and significantly outperforms other state-of-the-art distributed tools in terms of execution time (up to 6.5x faster) and memory usage (up to 2x less), resulting in a substantial cloud cost reduction. SeQuiLa is a cloud-native solution that can be easily deployed using any managed Kubernetes and Hadoop services available in public clouds, like Microsoft Azure Cloud, Google Cloud Platform, or Amazon Web Services. Together with the already implemented distributed range joins and coverage calculations, our package provides end-users with an unified SQL interface for convenient analyzing of population-scale genomic data in an interactive way. See https://biodatageeks.github.io/sequila/ for details.

show abstract

Section: Methodsmentioning

confidence: 99%

Cloud-native distributed genomic pileup operations

Wiewiórka

Szmurło

Stankiewicz

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The UK Biobank shows how standard data use agreements that set minimum data management and usage requirements can be developed to build trust and facilitate data exchange between data owners and utilizers. 14 Cloud-based solutions can also facilitate greater access to and usage of data, 15 and platforms and tools such as the National Institute of Allergy and Infectious Diseases TB Portals, GenTB (Translational Genomics platform for TB), and ReSeqTB (Relational Sequencing TB Data Platform) share data and tools that support translational clinical research. 16 – 18 Furthermore, digitalization should link already existing contributions, such as the Global TB Network (GTN) (active in both Brazil and South Africa), who go on to collect missing data on active TB drug safety monitoring (aDSM), 19 , 20 and to investigate possible interaction between COVID-19 and TB.…”

Section: Moving Aheadmentioning

confidence: 99%

Advancing TB research using digitized programmatic data

Taaffe

Croda

Moultrie

et al. 2021

int j tuberc lung dis

View full text Add to dashboard Cite

The use of real-world data from national TB care programs has great potential to answer key research questions in TB control and is now opportune due to increasing digital data collection and storage. We summarize an expert stakeholder workshop conducted on this topic in October 2019, with perspectives from academics, national TB program officers, and data managers. We discuss challenges and opportunities in the use of TB programmatic data for research and describe digital data availability in two large, high TB burden countries, Brazil and South Africa. From this, we posit that with a standardized data collection set, improved data management, and greater collaboration, more TB programmatic data can be used for research with measurable public health impact.

show abstract

“…Equipos en la Nube • Equipo servidor para el soporte del sistema operativo y el software PBX. Configurado en la nube aprovechando los servicios con que cuenta la UFPS con Amazon Cloud AWS [28]. La principal ventaja del software PBX Asterisk, es su bajo costo.…”

Section: Propuesta De Solución O Mejoraunclassified

PBX IP como herramienta de integración de servicios de comunicaciones para la UFPS basado en las mejores prácticas de ITIL.

García

Diaz-Leal

Llanez

2020

AiBi Revista de Investigación, Administración e Ingeniería

View full text Add to dashboard Cite

La telefonía de voz sobre el protocolo de Internet IP (VoIP) contempla beneficios en la implementación, permitiendo la comunicación entre usuarios de la empresa, sucursales y demás interesados representando un ahorro; siendo esta tecnología tendencia en comunicación telefónica empresarial y posibilitando la transmisión de voz humana a través de redes de transmisión de datos. Profundizando en esta tecnología y en sus beneficios, se diseña una central telefónica para el Departamento de Sistemas e Informática como modelo piloto en la Universidad Francisco de Paula Santander, ofreciendo servicios VoIP. Este modelo es factible de extender a la sede Cúcuta y demás sedes y dependencias dispersas geográficamente como; Los Patios, los CREAD (Centros Regionales de Educación Abierta y a Distancia) y la Seccional Ocaña. En esta implementación de la Central Privada Automática PBX (Private Branch Exchange), se considera aplicar las buenas prácticas para la gestión de servicios informáticos ITIL (Information Technology Infrastructure Library) que permita lograr la calidad y eficiencia en las operaciones asociadas a la solución de comunicaciones propuesta. La ventaja de este tipo de tecnología es el ahorro para la organización que la implemente; debido a que la VoIP, se transmite a través de las redes de datos de la organización y es posible enrutar a través de la red Internet servicios estratégicos que se Integren con sistemas de atención a cliente y centros de llamadas de call center, vía soluciones abiertas o propietarias de la institución. Además, el servicio se optimiza con alojamiento en la nube, centralizando la administración, el mantenimiento y la gestión lo cual redundará en los beneficios económicos de bajos costos y fácil escalabilidad. El proyecto desde la perspectiva institucional es importante, puesto que es un sistema de comunicaciones el cual busca mejorar el servicio al cliente y a las personas que intervienen en los procesos organizacionales; dejando de utilizar medios de comunicación personal en actividades laborales.

show abstract

Scalability and cost-effectiveness analysis of whole genome-wide association studies on Google Cloud Platform and Amazon Web Services

Cited by 18 publications

References 29 publications

Cloud-native distributed genomic pileup operations

Cloud-native distributed genomic pileup operations

Advancing TB research using digitized programmatic data

PBX IP como herramienta de integración de servicios de comunicaciones para la UFPS basado en las mejores prácticas de ITIL.

Contact Info

Product

Resources

About