Automatic Versioning of Time Series Datasets: a FAIR Algorithmic Approach

Gonzalez-Cebrian, Alba; McGuinness, Luke A; Bradford, Michael J.; Chis, Adriana E.; González–Vélez, Horacio

doi:10.1109/escience55777.2022.00034

Cited by 3 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We empirically evaluated our approach using time series data in three common scenarios of data versioning, namely i) data (cells) with missing values, ii) data with the row-wise transformation of values (values expressed as percentages) and data with column-wise transformation (values expressed on a logarithmic scale), and iii) sample size reduction by sub-setting rows. Initial results presented in [19] show that our proposed approach successfully detects different versions of a dataset, for up to 60% of cell changes, the deletion of up to 60% of rows, and column-wise transformation. However, there are minimal similarities detected for row-wise transformations.…”

Section: A Dimensionality Reductionmentioning

confidence: 98%

“…Data provenance and lineage play a key role in fostering reusability. In this context, we propose a novel approach for automatic data versioning with the goal to automatically generate FAIR-compliant provenance metadata [19]. We systematically detect and measure changes in datasets by using dimensionality reduction techniques.…”

Section: A Dimensionality Reductionmentioning

confidence: 99%

“…These parameters are then included in the metadata to enable data versioning and comparison. [source: [19]].…”

Section: B Data Fingerprinting and Watermarkingmentioning

confidence: 99%

See 2 more Smart Citations

Smardy: Zero-Trust FAIR Marketplace for Research Data

Filip

Ionite

Gonzalez-Cebrian

et al. 2022

2022 IEEE International Conference on Big Data (Big Data)

Self Cite

View full text Add to dashboard Cite

Over the past five years, different organisations have increasingly called for science to become more open and reproducible. They have endorsed a set of data-management principles known as the FAIR (Findable, Accessible, Interoperable, Reusable) principles. As such, there is a growing trend towards the open availability of research data, as researchers continue to enhance reproducibility by enabling sharing and opening of their findings and datasets. However, there is not yet a standardised way to openly enable access to datasets while keeping control of their final use, potentially obtaining benefits from their utilisation. This paper introduces SMARDY, an EUfunded project which is deploying a traceable FAIR-compliant open innovation marketplace for data. Its innovative method for data exchange consists of the use of blockchain for controlling access rights to data, with data models able to grant access according to policies completely kept under the control of the data owner/producer. We also describe how SMARDY employs dimensionality reduction techniques to automatically generate FAIR-compliant metadata, statistical fingerprinting to identify derivated datasets, and watermarking to help data owners trace the distribution of multiple copies of a dataset.

show abstract

Section: A Dimensionality Reductionmentioning

confidence: 98%

Section: A Dimensionality Reductionmentioning

confidence: 99%

See 1 more Smart Citation

Smardy: Zero-Trust FAIR Marketplace for Research Data

Filip

Ionite

Gonzalez-Cebrian

et al. 2022

2022 IEEE International Conference on Big Data (Big Data)

Self Cite

View full text Add to dashboard Cite

show abstract

Standardised Versioning of Datasets: a FAIR–compliant Proposal

González–Cebrián,

Bradford,

Chis

et al. 2024

Sci Data

Self Cite

View full text Add to dashboard Cite

This paper presents a standardised dataset versioning framework for improved reusability, recognition and data version tracking, facilitating comparisons and informed decision-making for data usability and workflow integration. The framework adopts a software engineering-like data versioning nomenclature (“major.minor.patch”) and incorporates data schema principles to promote reproducibility and collaboration. To quantify changes in statistical properties over time, the concept of data drift metrics (d) is introduced. Three metrics (dP, dE,PCA, and dE,AE) based on unsupervised Machine Learning techniques (Principal Component Analysis and Autoencoders) are evaluated for dataset creation, update, and deletion. The optimal choice is the dE,PCA metric, combining PCA models with splines. It exhibits efficient computational time, with values below 50 for new dataset batches and values consistent with seasonal or trend variations. Major updates (i.e., values of 100) occur when scaling transformations are applied to over 30% of variables while efficiently handling information loss, yielding values close to 0. This metric achieved a favourable trade-off between interpretability, robustness against information loss, and computation time.

show abstract