New Trends in Databases and Information Systems

Abstract

material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

On data lake architectures and metadata management

“…Oram's metadata classification is the most cited, especially in the industrial literature [9,28,46,47], presumably because it is inspired from metadata categories from data warehouses [45]. Thus, its adoption seems easier and more natural for practitioners who are already working with it.…”

“…Moreover, Oram's classification is quite fuzzy when applied in the context of data lakes. Diamantini et al indeed show that functional metadata intersect (Section 4.1.1) [9]. Therefore, practitioners who do not know this typology may be confused when using it to identify and organize metadata in a data lake.…”

“…They are commonly obtained from a DBMS for structured data, or via custom techniques during the data maturation stage. Diamantini et al enhance this typology with a generic metadata model [9] and show that business, operational and technical metadata sometimes intersect. For instance, data fields relate both to business and technical metadata, since they are defined in data schemas by business users.…”

“…Composition-centered graph models help decompose each data object into several inherent elements. The lake is viewed as a DAG where nodes represent objects or attributes, e.g., columns, tags, etc., and edges from any node A to any node B express the constraint B ⊆ A [9,20,38]. This organization helps users navigate through the data [38].…”

“…It can also be used as a basis to detect connections between objects. For instance, [9] used a simple string measure to detect links between heterogeneous objects by comparing their respective tags.…”