Maximizing Accuracy of Shared Databases when Concealing Sensitive Patterns

Abstract

The sharing of databases either within or across organizations raises the possibility of unintentionally revealing sensitive relationships contained in them. Recent advances in data-mining technology have increased the chances of such disclosure. Consequently, firms that share their databases might choose to hide these sensitive relationships prior to sharing. Ideally, the approach used to hide relationships should be impervious to as many data-mining techniques as possible, while minimizing the resulting distortion to the database. This paper focuses on frequent item sets, the identification of which forms a critical initial step in a variety of data-mining tasks. It presents an optimal approach for hiding sensitive item sets, while keeping the number of modified transactions to a minimum. The approach is particularly attractive as it easily handles databases with millions of transactions. Results from extensive tests conducted on publicly available real data and data generated using IBM’s synthetic data generator indicate that the approach presented is very effective, optimally solving problems involving millions of transactions in a few seconds.

The challenges of personal data markets and privacy

“…Systems enabling a flow control for accountable information would have to be designed, implemented, tested, and standardized along with the underlying data models, audit processes, and controls. Quality metrics could adapt existing paradigms for privacy metrics, such as k-anonymity for database privacy (Sweeney 2002 effectiveness of data perturbation approaches (Menon et al 2005 ;; Li and Sarkar 2006 ), information-theoretic metrics of data leakage, or randomization-based proof techniques inspired by differential privacy (Dwork 2011 ).…”

Full-Exact Approach for Frequent Itemset Hiding

“…The Full-Exact method is performed together with a popular existing algorithm chosen from the literature for comparison purposes. The reference method is Menon's exact approach (Menon et al, 2005). The first part of the method proposed in Menon (Menon et al, 2005) decides about the minimum number of transactions that have to be sanitized, with the objective of minimizing accuracy using linear programming.…”

“…The reference method is Menon's exact approach (Menon et al, 2005). The first part of the method proposed in Menon (Menon et al, 2005) decides about the minimum number of transactions that have to be sanitized, with the objective of minimizing accuracy using linear programming. The results are then used by the heuristic part for actual sanitization trying to make minimum harm on actual database.…”

Manage the Tradeoff in Data Sanitization

“…The user often needs to choose a preferred one by using highlevel domain experience. Generally, the receiver of shared data would likely be interested in a sharing agreement only when the data accuracy is reasonable [5]. In other words, the degree of data distortion cannot exceed a maximum level.…”

“…A variety of other solutions have been proposed [5], [6]. Generally, the difficulty of data sanitization is not in concealing sensitive knowledge, but in reducing the accompanying side effects.…”