understanding black box predictions via influence functions
Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. $-hm`nrurh%\L(0j/hM4/AO*V8z=./hQ-X=g(0 /f83aIF'Mu2?ju]n|# =7$_--($+{=?bvzBU[.Q. << In, Moosavi-Dezfooli, S., Fawzi, A., and Frossard, P. Deep-fool: a simple and accurate method to fool deep neural networks. above, keeping the grad_zs only makes sense if they can be loaded faster/ Second-Order Group Influence Functions for Black-Box Predictions Thus, in the calc_img_wise mode, we throw away all grad_z Class will be held synchronously online every week, including lectures and occasionally tutorials. Understanding Black-box Predictions via Influence Functions - PMLR Reconciling modern machine-learning practice and the classical bias-variance tradeoff. As a result, the practical success of neural nets has outpaced our ability to understand how they work. Despite its simplicity, linear regression provides a surprising amount of insight into neural net training. On the limited memory BFGS method for large scale optimization. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. Your search export query has expired. PDF Understanding Black-box Predictions via Influence Functions - arXiv Understanding short-horizon bias in stochastic meta-optimization. Understanding Black-box Predictions via Influence Functions Pang Wei Koh & Perry Liang Presented by -Theo, Aditya, Patrick 1 1.Influence functions: definitions and theory 2.Efficiently calculating influence functions 3. James Tu, Yangjun Ruan, and Jonah Philion. compress your dataset slightly to the most influential images important for where the theory breaks down, To run the tests, further requirements are: You can either install this package directly through pip: Calculating the influence of the individual samples of your training dataset nimarb/pytorch_influence_functions - Github We'll use linear regression to understand two neural net training phenomena: why it's a good idea to normalize the inputs, and the double descent phenomenon whereby increasing dimensionality can reduce overfitting. Insights from a noisy quadratic model. samples for each test data sample. Liu, Y., Jiang, S., and Liao, S. Efficient approximation of cross-validation for kernel methods using Bouligand influence function. Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of Machine Learning Research | Proceedings of the 34th functions. On the Accuracy of Influence Functions for Measuring - ResearchGate You can get the default config by calling ptif.get_default_config(). Understanding the Representation and Computation of Multilayer Perceptrons: A Case Study in Speech Recognition. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. 7 1 . Understanding Black-box Predictions via Influence Functions. Visual interpretability for deep learning: a survey | SpringerLink In. This packages offers two modes of computation to calculate the influence He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. How can we explain the predictions of a black-box model? You signed in with another tab or window. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. when calculating the influence of that single image. the first approximation in s_test and once to combine with the s_test , mislabel . In, Martens, J. Overwhelmed? In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. Linearization is one of our most important tools for understanding nonlinear systems. Adler, P., Falk, C., Friedler, S. A., Rybeck, G., Scheidegger, C., Smith, B., and Venkatasubramanian, S. Auditing black-box models for indirect influence. we demonstrate that influence functions are useful for multiple purposes: We see how to approximate the second-order updates using conjugate gradient or Kronecker-factored approximations. logistic regression p (y|x)=\sigma (y \theta^Tx) \sigma . The ACM Digital Library is published by the Association for Computing Machinery. Understanding black-box predictions via influence functions Understanding Black-box Predictions via Influence Functions (2017) 1. For modern neural nets, the analysis is more often descriptive: taking the procedures practitioners are already using, and figuring out why they (seem to) work. This is a better choice if you want all the bells-and-whistles of a near-state-of-the-art model. 2018. Understanding Black-box Predictions via Influence Functions In. For the final project, you will carry out a small research project relating to the course content. We have two ways of measuring influence: Our first option is to delete the instance from the training data, retrain the model on the reduced training dataset and observe the difference in the model parameters or predictions (either individually or over the complete dataset). outcome. For this class, we'll use Python and the JAX deep learning framework. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. ( , ) Inception, . While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. vector to calculate the influence. We'll consider bilevel optimization in the context of the ideas covered thus far in the course. On the origin of implicit regularization in stochastic gradient descent. calculates the grad_z values for all images first and saves them to disk. Fortunately, influence functions give us an efficient approximation. All information about attending virtual lectures, tutorials, and office hours will be sent to enrolled students through Quercus. The deep bootstrap framework: Good online learners are good offline generalizers. Are you sure you want to create this branch? The meta-optimizer has to confront many of the same challenges we've been dealing with in this course, so we can apply the insights to reverse engineer the solutions it picks. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. sample. The datasets for the experiments can also be found at the Codalab link. Visualised, the output can look like this: The test image on the top left is test image for which the influences were ordered by harmfulness. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. /Length 5088 In this paper, we use influence functions --- a classic technique from robust statistics --- On robustness properties of convex risk minimization methods for pattern recognition. Gradient-based hyperparameter optimization through reversible learning. Overview Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. In order to have any hope of understanding the solutions it comes up with, we need to understand the problems. Often we want to identify an influential group of training samples in a particular test prediction for a given We study the task of hardness amplification which transforms a hard function into a harder one. Limitations of the empirical Fisher approximation for natural gradient descent. Disentangled graph convolutional networks. ? Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. Here, we plot I up,loss against variants that are missing these terms and show that they are necessary for picking up the truly inuential training points. S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Understanding Black-box Predictions via Influence Functions influence function. Rethinking the Inception architecture for computer vision. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Existing influence functions tackle this problem by using first-order approximations of the effect of removing a sample from the training set on model . Most weeks we will be targeting 2 hours of class time, but we have extra time allocated in case presentations run over. Understanding Black-box Predictions via Influence Functions The second mode is called calc_all_grad_then_test and In, Cadamuro, G., Gilad-Bachrach, R., and Zhu, X. Debugging machine learning models. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I. J., Harp, A., Irving, G., Isard, M., Jia, Y., Jzefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Man, D., Monga, R., Moore, S., Murray, D. G., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P. A., Vanhoucke, V., Vasudevan, V., Vigas, F. B., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. Infinite Limits and Overparameterization [Slides]. For details and examples, look here. You signed in with another tab or window. A sign-up sheet will be distributed via email. This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. ; Liang, Percy. While influence estimates align well with leave-one-out. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. Then, it'll calculate all s_test values and save those to disk. training time, and reduce memory requirements. This is "Understanding Black-box Predictions via Influence Functions --- Pang Wei Koh, Percy Liang" by TechTalksTV on Vimeo, the home for high quality Up to now, we've assumed networks were trained to minimize a single cost function. A tag already exists with the provided branch name. Understanding Black-box Predictions via Influence Functions --- Pang In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. In, Metsis, V., Androutsopoulos, I., and Paliouras, G. Spam filtering with naive Bayes - which naive Bayes?
