Data quality for federated medical data lakes

Abstract

Purpose Medical research requires biological material and data collected through biobanks in reliable processes with quality assurance. Medical studies based on data with unknown or questionable quality are useless or even dangerous, as evidenced by recent examples of withdrawn studies. Medical data sets consist of highly sensitive personal data, which has to be protected carefully and is available for research only after the approval of ethics committees. The purpose of this research is to propose an architecture to support researchers to efficiently and effectively identify relevant collections of material and data with documented quality for their research projects while observing strict privacy rules. Design/methodology/approach Following a design science approach, this paper develops a conceptual model for capturing and relating metadata of medical data in biobanks to support medical research. Findings This study describes the landscape of biobanks as federated medical data lakes such as the collections of samples and their annotations in the European federation of biobanks (Biobanking and Biomolecular Resources Research Infrastructure – European Research Infrastructure Consortium, BBMRI-ERIC) and develops a conceptual model capturing schema information with quality annotation. This paper discusses the quality dimensions for data sets for medical research in-depth and proposes representations of both the metadata and data quality documentation with the aim to support researchers to effectively and efficiently identify suitable data sets for medical studies. Originality/value This novel conceptual model for metadata for medical data lakes has a unique focus on the high privacy requirements of the data sets contained in medical data lakes and also stands out in the detailed representation of data quality and metadata quality of medical data sets.

Deep Learning-Based Attack Detection and Classification in Android Devices

“…This, combined with the dynamic evolutionary capacity of malware, allows it to assume semantically similar but structurally dissimilar forms, thereby causing novel damages such as connection blockages, system corruption, password theft, and more, posing threats to users, organizations, and systems. Recent history demonstrates how malware has become the primary tool for initiating large-scale attacks, resulting in severe damages and economic losses [11][12][13][14][15][16]. Traditionally, various techniques have been used for malware detection, including signature-based approaches (utilizing regular expressions, file names, etc.…”

Retos y oportunidades en la accesibilidad de datos

“…Los almacenes de datos hacen estas funciones a través de procesos de ETL (Extract, Transform, Load), lo que implica altos costos en la creación, mantenimiento y gestión de estos entornos. Este es uno de los motivos por el que los lagos de datos se están posicionando como una alternativa para almacenar grandes cantidades de información en su forma y formato original sin costos altos por la gestión de los datos (Eder & Shekhovtsov, 2021).…”

Data Lakes: A Survey of Functions and Systems

“…In essence, a data lake is a flexible, scalable data storage and management system, which ingests and stores raw data from heterogeneous sources in their original format, and provides maintenance, query processing and data analytics in an on-the-fly manner, with the help of rich metadata [116], [138], [142], [143]. Data lakes are proposed to store and manage data in many real-life use cases: Internet of things (IoT) and smart city [99], manufacturing [112], medicine [42], [55], [114], mobility service (e.g., Uber) [50], biology [23], smart grids [20], [103], air quality control [145], flights data [96], disease control, labor markets and products [13].…”