A Framework for Evaluating Privacy Preserving Data Mining Algorithms*

Abstract

Recently, a new class of data mining methods, known as privacy preserving data mining (PPDM) algorithms, has been developed by the research community working on security and knowledge discovery. The aim of these algorithms is the extraction of relevant knowledge from large amount of data, while protecting at the same time sensitive information. Several data mining techniques, incorporating privacy protection mechanisms, have been developed that allow one to hide sensitive itemsets or patterns, before the data mining process is executed. Privacy preserving classification methods, instead, prevent a miner from building a classifier which is able to predict sensitive data. Additionally, privacy preserving clustering techniques have been recently proposed, which distort sensitive numerical attributes, while preserving general features for clustering analysis. A crucial issue is to determine which ones among these privacy-preserving techniques better protect sensitive information. However, this is not the only criteria with respect to which these algorithms can be evaluated. It is also important to assess the quality of the data resulting from the modifications applied by each algorithm, as well as the performance of the algorithms. There is thus the need of identifying a comprehensive set of criteria with respect to which to assess the existing PPDM algorithms and determine which algorithm meets specific requirements.In this paper, we present a first evaluation framework for estimating and comparing different kinds of PPDM algorithms. Then, we apply our criteria to a specific set of algorithms and discuss the evaluation results we obtain. Finally, some considerations about future work and promising directions in the context of privacy preservation in data mining are discussed.

Privacy Preserving Data Mining Research: Current Status and Key Issues

“…They also suggested a set of metrics for assessing PPDM performance. Bertino et al [1] proposed a taxonomy for classifying existing PPDM algorithms. They also developed a framework and based upon which to evaluate the relative performance of selected heuristic-based hiding algorithms.…”

“…They also developed a framework and based upon which to evaluate the relative performance of selected heuristic-based hiding algorithms. In this paper, we propose to consolidate and simplify the taxonomy brought by [1]. We have also attempted to examine the relative performance of PPDM from individual component level instead of the complete PPDM algorithms.…”

“…The proposed taxonomy contains four levels (see Figure 1). This scheme differs from existing studies in two ways: (1) we treat data modification methods as part of privacy preserving techniques, and (2) we consider the purpose of hiding as a key classifier and place it before data mining task/algorithm level. The number appears below each category indicated its percentage of usage (popularity).…”

“…Moreover, data mining techniques can reveal critical information about business transactions, compromising the free competition in a business setting [1]. Therefore, privacy preserving data mining (PPDM) has becoming an increasingly important field of study.…”

“…Given the number of different algorithms have been developed over the past years, there is an emerging need of synthesizing literature to understand the nature of problems, identify potential research issues, standardize new research area, and evaluate the relative performance of different approaches [1] [28].…”