Searching heterogeneous personal digital traces

Abstract

Digital traces of our lives are now constantly produced by various connected devices, internet services and interactions. Our actions result in a multitude of heterogeneous data objects, or traces, kept in various locations in the cloud or on local devices. Users have very few tools to organize, understand, and search the digital traces they produce. We propose a simple but flexible data model to aggregate, organize, and find personal information within a collection of a user's personal digital traces. Our model uses as basic dimensions the six questions: what, when, where, who, why, and how. These natural questions model universal aspects of a personal data collection and serve as unifying features of each personal data object, regardless of its source. We propose indexing and search techniques to aid users in searching for their past information in their unified personal digital data sets using our model. Experiments performed over real user data from a variety of data sources such as Facebook, Dropbox, and Gmail show that our approach significantly improves search accuracy when compared with traditional search tools.

A Frequency-Based Learning-To-Rank Approach for Personal Digital Traces

“…Based on the observation that contextual cues are strong trigger for autobiographical memories ( [4, 5, 6, 7]), and that personal data is rich in contextual information, in the form of metadata, application data, or environment knowledge, we can represent personal digital traces using a combination of dimensions that naturally summarize various aspects of the data collection: who, when, where, what, why and how. Our work uses an intuitive multidimensional data model that relies on these six dimensions as the unifying features of each personal digital trace object, regardless of its source [8]: what: content such as messages, messages subjects, description of events, list of interests of a user who: user names, senders, recipients, event owners where: physical or logical, in the real-world and in the system. For instance, hometown, location, event venues, URLs, file/folder paths when: time and date, but also what was happening concurrently.…”

“…A simple approach would be to use ETL rules, but this proved unpractical as new sources of data were added, and the schemas of the existing sources were modified by the third-party apps. Therefore, we opted for a machine learning multi-class classifier using a combination of LSTM (Long Short-Term Memory) and Dense layers [8]. Given a sentence, the classifier will output a label (who, when, where, what, why and how).…”

A Frequency-Based Learning-To-Rank Approach for Personal Digital Traces

“…Considering the fact that contextual cues are strong trigger for autobiographical memories ( [6, 16, 23, 28]), and that personal data is rich in contextual information, in the form of metadata, application data, or environment knowledge, personal digital traces can be represented following a combination of dimensions that naturally summarize various aspects of the data collection: who, when, where, what, why and how. In this work, we use an intuitive multidimensional data model that relies on these six dimensions as the unifying features of each personal digital trace object, regardless of its source [26]. Below is a list of dimensional data that can be extracted from a user's personal digital traces:…”

“…Having defined the multidimensional data model, it is still necessary to find an effective mechanism to automatically translate the heterogeneous set of personal data into the six dimensions. To this end, we use a machine learning multi-class classifier using a combination of LSTM (Long Short-Term Memory) and Dense layers [26]. Given a sentence, the classifier will output a label (who, when, where, what, why and how).…”