A design framework and exemplar metrics for FAIRness

Wilkinson, Mark D.; Sansone, Susanna-Assunta; Schultes, Erik; Doorn, Peter; Santos, Luiz Olavo Bonino da Silva; Dumontier, Michel

doi:10.1101/225490

Cited by 27 publications

(33 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For example, a FAIR Metric group created ways to measure the principles. After reviewing results from focus group interviews, the FAIR Metric group discovered that both their metrics and even some principles were misunderstood (Wilkinson et al, 2018). GO FAIR, n.d.).…”

Section: Fair Data Principlesmentioning

confidence: 99%

Scientists' data discovery and reuse behavior: (Meta)data fitness for use and the FAIR data principles

Bishop

Hank

Webster

et al. 2019

Proceedings of the Association for Information Science and Tech

View full text Add to dashboard Cite

Environmental science researchers frequently must work at regional and global scales that require them to search for, evaluate, and reuse data collected by others. This study assesses science data reusers' information‐seeking behavior to determine fitness for use, a concept from consumer behavior research that explains how consumers define and assess product quality and compatibility to satisfy their specific needs. Using a critical incident technique, 22 researchers in earth science described their most recent data discovery resulting in reuse. An interview schedule derived from the FAIR Data Principles framed participants' information‐seeking behavior along a sequence of actions, which included finding, accessing, making interoperable, and reusing data. Researchers' perspectives on how they discovered and evaluated data for reuse gives new insights into fitness for use of data by human consumers, and contributes to considerations for furthering work into machine‐actionable data and learning.

show abstract

Section: Fair Data Principlesmentioning

confidence: 99%

Scientists' data discovery and reuse behavior: (Meta)data fitness for use and the FAIR data principles

Bishop

Hank

Webster

et al. 2019

Proceedings of the Association for Information Science and Tech

View full text Add to dashboard Cite

show abstract

“…A FAIR metric is directly related to one of the FAIR guiding principles. FAIRshake adopts the concept of a FAIR metric from the FAIRmetrics effort (6). In order to make FAIR metrics reusable, FAIRshake collects information about each metric, and when users attempt to associate a digital resource with metrics and rubrics, existing metrics are provided as a first choice.…”

Section: Registering Digital Resourcesmentioning

confidence: 99%

“…This approach is utilized by major search engines to index websites and bind information together. Using this RDF-expressed metadata alone, some FAIR metrics may be automatically resolved; including most of the exemplar universal metrics that were designed with RDF in mind (6). As schema.org expands its vocabulary through initiatives such as Bioschemas (15), RDF will enable these automatic mechanisms to scale FAIRshake assessments with the annotations present in public resources.…”

Section: Rubricsmentioning

confidence: 99%

“…Several initiatives have already developed their own understandings of FAIRness and developed methods of assessing FAIRness by self-and peer-reviewed manual question-answer approaches (4,5). Because the different strategies for asserting FAIRness to date have been independent of one another, and as such not comparable, a template was created for constructing FAIR metrics around the original FAIR guiding principles (6). This FAIR metrics template is provided through GitHub in a format that can become community driven.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FAIRshake: toolkit to evaluate the findability, accessibility, interoperability, and reusability of research digital resources

Clarke

Wang

Jones

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

As more datasets, tools, workflows, APIs, and other digital resources are produced by the research community, it is becoming increasingly difficult to harmonize and organize these efforts for maximal synergistic integrated utilization. The Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles have prompted many stakeholders to consider strategies for tackling this challenge by making these digital resources follow common standards and best practices so that they can become more integrated and organized. Faced with the question of how to make digital resources more FAIR, it has become imperative to measure what it means to be FAIR. The diversity of resources, communities, and stakeholders have different goals and use cases and this makes assessment of FAIRness particularly challenging. To begin resolving this challenge, the FAIRshake toolkit was developed to enable the establishment of community-driven FAIR metrics and rubrics paired with manual, semi-and fully-automated FAIR assessment capabilities. The FAIRshake toolkit contains a database that lists registered digital resources, with their associated metrics, rubrics, and assessments. The FAIRshake toolkit also has a browser extension and a bookmarklet that enables viewing and submitting assessments from any website. The FAIR assessment results are visualized as an insignia that can be viewed on the FAIRshake website, or embedded within hosting websites. Using FAIRshake, a variety of bioinformatics tools, datasets listed on dbGaP, APIs registered in SmartAPI, workflows in Dockstore, and other biomedical digital resources were manually and automatically assessed for FAIRness. In each case, the assessments revealed room for improvement, which prompted enhancements that significantly upgraded FAIRness scores of several digital resources.

show abstract

“…Recently, the FAIRmetrics.org working group [ http://fairmetrics.org ] has published an open framework [3] for the creation and publication of metrics to objectively evaluate the many possible dimensions of FAIR that have been published. The framework was used to generate an initial set of 14 FAIR Metrics.…”

Section: Introductionmentioning

confidence: 99%

Evaluating FAIR-Compliance Through an Objective, Automated, Community-Governed Framework

Wilkinson

Dumontier

Sansone

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

With the increased adoption of the FAIR Principles, a wide range of stakeholders, from scientists to publishers, funding agencies and policy makers, are seeking ways to transparently evaluate resource FAIRness. We describe the FAIR Evaluator, a software infrastructure to register and execute tests of compliance with the recently published FAIR Metrics. The Evaluator enables digital resources to be assessed objectively and transparently. We illustrate its application to three widely used generalist repositories -Dataverse, Dryad, and Zenodo -and report their feedback. Evaluations allow communities to select relevant Metric subsets to deliver FAIRness measurements in diverse and specialized applications. Evaluations are executed in a semi-automated manner through Web Forms filled-in by a user, or through a JSON-based API. A comparison of manual vs automated evaluation reveals that automated evaluations are generally stricter, resulting in lower, though more accurate, FAIRness scores. Finally, we highlight the need for enhanced infrastructure such as standards registries, like FAIRsharing, as well as additional community involvement in domain-specific data infrastructure creation.

show abstract

A design framework and exemplar metrics for FAIRness

Cited by 27 publications

References 5 publications

Scientists' data discovery and reuse behavior: (Meta)data fitness for use and the FAIR data principles

Scientists' data discovery and reuse behavior: (Meta)data fitness for use and the FAIR data principles

FAIRshake: toolkit to evaluate the findability, accessibility, interoperability, and reusability of research digital resources

Evaluating FAIR-Compliance Through an Objective, Automated, Community-Governed Framework

Contact Info

Product

Resources

About