DP-Finder: Finding differential privacy violations by sampling and optimization

Benjamin Bichsel, Timon Gehr, Dana Drachsler-Cohen, Petar Tsankov, Martin Vechev

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We present DP-Finder, a novel approach and system that automatically derives lower bounds on the differential privacy enforced by algorithms. Lower bounds are practically useful as they can show tightness of existing upper bounds or even identify incorrect upper bounds. Computing a lower bound involves searching for a counterexample, defined by two neighboring inputs and a set of outputs, that identifies a large privacy violation. This is an inherently hard problem as finding such a counterexample involves inspecting a large (usually infinite) and sparse search space. To address this challenge, DP-Finder relies on two key insights. First, we introduce an effective and precise correlated sampling method to estimate the privacy violation of a counterexample. Second, we show how to obtain a differentiable version of the problem, enabling us to phrase the search task as an optimization objective to be maximized with state-of-the-art numerical optimizers. This allows us to systematically search for large privacy violations. Our experimental results indicate that DP-Finder is effective in computing differential privacy lower bounds for a number of randomized algorithms. For instance, it finds tight lower bounds in algorithms that obfuscate their input in a non-trivial fashion.

Original languageEnglish
Title of host publicationCCS 2018 - Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security
Pages508-524
Number of pages17
ISBN (Electronic)9781450356930
DOIs
StatePublished - 15 Oct 2018
Externally publishedYes
Event25th ACM Conference on Computer and Communications Security, CCS 2018 - Toronto, Canada
Duration: 15 Oct 2018 → …

Publication series

NameProceedings of the ACM Conference on Computer and Communications Security

Conference

Conference25th ACM Conference on Computer and Communications Security, CCS 2018
Country/TerritoryCanada
CityToronto
Period15/10/18 → …

Keywords

  • Differential privacy
  • Lower bounds
  • Optimization
  • Sampling

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'DP-Finder: Finding differential privacy violations by sampling and optimization'. Together they form a unique fingerprint.

Cite this