This area is a stub, you can help by improving it.
In recent years, the debate around data ownership in Machine Learning has intensified. Do individuals who contribute data to datasets have the "right to be forgotten"? What would it mean to fully delete someone's data in the context of machine learning, where data lives on inside models, even after the data has been removed from the training dataset? The EU has introduced legislation safeguarding the right to be forgotten and many public data resources include terms whereby individuals may ask for their data to be deleted. For example, in the UK Biobank, one of the most valuable biomedical research databases, participants may request at any time that their data no longer be used.
In modern machine learning, these requirements pose significant technical challenges. In most settings, retraining the model from scratch on the remaining data may be prohibitively expensive, especially since the requests for deletion may come at any time. Is it possible to efficiently remove datapoints from a trained model without retraining? Ginart et al. propose a framework for studying this problem. They outline four principles that can guide the development of data deletion methods (e.g. modularity) and propose some methods for k-means clustering that are based on them. For more complex models, exact data deletion is challenging, so a number of approximate data deletion methods have been proposed. Influence functions (also proposed by Monari and Dreyfus) provide a natural method for approximate data deletion and rely only on the computation of the Hessian-gradient product. For linear and logistic models, we can sidstep this expensive computation, instead using the projective residual update, which is linear in the feature dimensionality. Evaluating the effectiveness of approximate data deletion methods poses yet another challenge. The feature injection test is one approach. Others have tackled the data deletion problem under a slightly different guise, Cauwenberghs and Poggio refer to it as decremental learning and present an incremental procedure for training SVMs that are amenable to data deletion. Tsai et al. propose a decremental learning method based on retraining with warm starts (which is also similar to the "machine unlearning" approach proposed by Cao and Yang).
It's worth noting that data deletion has applications beyond data ownershp and privacy. In the context of data selection, data deletion methods provide an efficient way to debug models by removing the influence of problematic datapoints. In model evaluation, they provide a means for efficiently performing leave-one-out cross validation.
In addition to adversarial attacks aiming to mislead machine learning systems, the attacks against privacy and confidentiality have also received great attention in recent years. In these attacks, the adversary aims to recover various types of private sensitive information about the training data. Concretely, we discuss three categories of such privacy attacks.
-
Membership Inference Attacks The goal of the membership inference attack (MIA) is to infer whether a given record is in the model’s training dataset or not. Assuming black-box access, they train an attack model to distinguish the target model’s behavior on the inputs that are involved or not involved in training. Observing the impact of overfitting in membership leakage, the generalized membership inference attack (GMIA) focuses on the well-generalized models and aims at detecting and analyzing the membership of vulnerable target records (outliers). As opposed to these MIAs against supervised models, LOGAN reveals the membership of instances in training SOTA generative models via leveraging the generative adversarial networks (GANs), and the Secret Sharer investigates the unintended memorization of unique, secret training data sequences in generative sequence models via quantitative risk assessment.
-
Reconstruction Attacks We discuss two types of reconstruction attacks with different reconstruction goals. In the first type, the attacker aims to recover semantically meaningful and representative data for each training class, e.g., recover an image of a person given their name (the label) Fredrikson et al.. The other type of attack aims to reconstruct the exact unknown data given the classifier’s prediction. The Secret Revealer successfully inverts the deep neural networks by learning a distributional prior via the GANs to assist the inversion, assuming access to partial public information.
Apart from the privacy attacks on these traditional data types, we briefly outline the privacy attacks on more complicated and heterogeneous data types such as graphs. Duddu et al. show that one can conduct various types of privacy attacks on graphs given node embeddings or graph embeddings. LinkTeller studies the edge privacy in the vertical data partition scenario via inter-node influence analysis.
Differential Privacy (DP) proposed by Dwork et al. is a standard way to preserve privacy. It bounds the change in output distribution caused by a small input difference for a randomized algorithm.
DP-SGD is a popular way to train DP models in supervised learning, via performing gradient perturbation. It clips the norm of per-sample gradient to bound sensitivity and adds Guassian noise on the summed gradients. In DP-SGD, Moments Accountant is proposed to compute the privacy budget which provides a tighter privacy guarantee compared with standard composition theorems.
McMahan et al. extend DP-SGD to Federated Learning, and propose the notion of user-level DP, i.e., bounding the change of global model caused by one user difference. At each round, the server clips the norm of each local update, adds Gaussian noise on the summed update, and characterizes its privacy guarantee via moments accountant. In CpSGD, each user clips and quantizes the model update, and adds noise drawn from Binomial distribution, achieving both communication efficiency and DP.
In Reinforcement Learning, different DP methods are proposed for different privacy protection purposes. For example, Balle et al. propose differentially private on-policy evaluation via output perturbation, i.e., adding Gaussian noise to the parameters vector for the value functions. Xie et al. propose a differentially private approach for off-policy evaluation via gradient perturbation. Wang et al. propose a differentially private Q-learning algorithm to protect the privacy of value function approximator by adding functional noise to the value function iteratively in the training.
Differentially private generative models are trained on privacy-sensitive data and aim to generate differentially private data with rigorous privacy guarantees to avoid leaking private information. The generated differentially private data is model or downstream task agnostic, and thus can be published and applied to various applications.
Several studies apply differential privacy to traditional data generation algorithms such as Bayesian networks, synthetic data generation from marginal distributions, and the multiplicative weights approach.
A recent line of work adapts differential privacy to neural-based generative models, such as Generative Adversarial Networks (GAN). DP-GAN achieves differential privacy by adding Gaussian noise to the discriminator gradients during the training process. DP-CGAN uses a similar approach to guarantee DP and trains a conditional GAN to generate both synthetic data and labels. GS-WGAN uses the Wasserstein loss and sanitizes the data-dependent gradients of the generator to improve data utility.
In addition, for more complicated data generation, PATE-GAN combines the PATE framework with GAN, specifically the discriminator within a GAN, which achieves high performance on low dimensional tabular data. G-PATE improves upon PATE-GAN by directly training a student generator using the teacher discriminators. In particular, it uses the random projection algorithm to reduce the gradient dimension during training for generating high-dimensional data. DataLens proposes a novel DP gradient compression and aggregation approach, which combines top-$k$ dimension compression with a corresponding noise injection mechanism, which provides detailed theoretical analysis on the convergence and demonstrates a significant utility improvement upon PATE-GAN and G-PATE on high dimensional datasets empirically.
A few recent works focus on synthetic data generation using private embeddings. For example, DP-MERF and PEARL use differentially private embedding to train generative models on the embedding space. Concretely, DP-MERF uses random feature representations of kernel mean embeddings, and PEARL improves upon DP-MERF by incorporating characteristic function that improves the generator’s learning capability. Both DP-MERF and PEARL focus on generating a private embedding space on which the distance metric between synthetic and real data is computed.