Using Metrics Suites to Improve the Measurement of Privacy in Graphs

Abstract

Social graphs are widely used in research (e.g., epidemiology) and business (e.g., recommender systems). However, sharing these graphs poses privacy risks because they contain sensitive information about individuals. Graph anonymization techniques aim to protect individual users in a graph, while graph de-anonymization aims to re-identify users. The effectiveness of anonymization and de-anonymization algorithms is usually evaluated with privacy metrics. However, it is unclear how strong existing privacy metrics are when they are used in graph privacy. In this paper, we study 26 privacy metrics for graph anonymization and de-anonymization and evaluate their strength in terms of three criteria: monotonicity indicates whether the metric indicates lower privacy for stronger adversaries; for within-scenario comparisons, evenness indicates whether metric values are spread evenly; and for between-scenario comparisons, shared value range indicates whether metrics use a consistent value range across scenarios. Our extensive experiments indicate that no single metric fulfills all three criteria perfectly. We therefore use methods from multi-criteria decision analysis to aggregate multiple metrics in a metrics suite, and we show that these metrics suites improve monotonicity compared to the best individual metric. This important result enables more monotonic, and thus more accurate, evaluations of new graph anonymization and de-anonymization algorithms.

Anonymization Techniques for Privacy Preserving Data Publishing: A Comprehensive Survey

“…However, in most cases, a single metric cannot capture the entire concept of privacy, therefore, two or more metrics are jointly used to measure the level of privacy offered by an anonymization algorithm/model. Recently, Zhao et al [125] analyzed and discussed twenty-six different metrics used for privacy evaluation in anonymized graph. The authors suggested that no single metric is effective to evaluate privacy protection in anonymous graphs.…”

Toward Privacy Preservation Using Clustering Based Anonymization: Recent Advances and Future Research Outlook

“…In many real-world cases, the privacy measured by one evaluation metric may not be monotonic. Hence, some approaches have suggested employing a metrics suit (i.e., multiple metrics), rather than relying on one or two metrics, while evaluating performance of anonymization methods [138], [139]. With the rapid increase in the diversity of privacy threats, and the ever-changing landscape of attacker capabilities, the development of accurate privacy and utility evaluation metrics has become more urgent than ever.…”

Designing Strong Privacy Metrics Suites Using Evolutionary Optimization

“…In this paper, we focus on five objectives: monotonicity, diversity, evenness, shared value range, and number of metrics. We selected these five objectives because they have been used as criteria for the strength of privacy metrics in the literature [59,60], and because they could be expressed formally and implemented in our optimization.…”

“…In addition, combinations of metrics can make the ranking of alternative PETs more robust. Recent work has also shown that combinations of metrics, or metrics suites, can improve the monotonicity of privacy measurement [60], which is a key requirement for consistent measurement.…”

“…However, there are two open issues: first, how the metrics should be combined, and second, which metrics should be selected. To address the first issue, the use of multi-criteria decision analysis methods (MCDA) was recently proposed [60]. The second issue has so far been addressed in a rather ad-hoc fashion, either relying on convenience, i.e.…”