Sample efficiency is a major challenge of applying deep reinforcement learning (RL) techniques to robotics tasks --- existing
algorithms often require a massive amount of interactions with the environment (samples). Promising solutions include model-based
reinforcement learning and imitation learning (IL). However, the theoretical understanding of them is mostly missing in the setting of
continuous and high-dimensional state space and neural network function approximators.

In this talk, I will present recent work on designing principled model-based RL and IL algorithms with theoretical analyses. These
algorithms also empirically outperform prior works on benchmark tasks in sample efficiency.

No prior knowledge of deep reinforcement learning and imitation learning is required. Based on joint works with Nick Landolfi, Yuping Luo, Garrett Thomas, Huazhe Xu, Trevor Darrell, and Yuandong Tian.

A model with zero training error is overfit to the training data and will typically generalize poorly" goes statistical textbook wisdom. Yet, in modern practice, over-parametrized deep networks with near perfect fit on training data still show excellent test performance. As I will discuss in the talk, this apparent contradiction is key to understanding the practice of modern machine learning. 

While classical methods rely on a trade-off balancing the complexity of predictors with training error, modern models are best described by interpolation, where a predictor is chosen among functions that fit the training data exactly, according to a certain (implicit or explicit) inductive bias. Furthermore, classical and modern models can be unified within a single "double descent" risk curve, which extends the classical U-shaped bias-variance curve beyond the point of interpolation. This understanding of model performance delineates the limits of the usual ''what you see is what you get" generalization bounds in machine learning and points to new analyses required to understand computational, statistical, and mathematical properties of modern models. 

I will proceed to discuss some important implications of interpolation for optimization, both in terms of "easy" optimization (due to the scarcity of non-global minima), and to fast convergence of small mini-batch SGD with fixed step size.

This talk will survey old connections and also shed new light on the ability of margin maximization methods to not only minimize certain
losses like he exponential loss, but moreover output margin-mazimizing solutions.

Joint work with Ziwei Ji.

Conventional wisdom in machine learning taboos training on the test set, interpolating the training data, and optimizing to high
precision. This talk will present evidence demonstrating that this conventional wisdom is wrong. I will additionally highlight commonly
overlooked phenomena imperil the reliability of current learning systems: surprising sensitivity to how data is generated and significant diminishing returns in model accuracies given increased compute resources. I will close with a discussion of how new best practices to mitigate these effects are critical for truly robust and reliable machine learning.

We build a rigorous bridge between deep networks (DNs) and approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs), which provide a powerful portal through which to view and analyze their inner workings. For instance, conditioned on the input signal, the output of a MASO DN can be written as a simple affine transformation of the input. This implies that a DN constructs a set of signal-dependent, class-specific templates against which the signal is compared via a simple inner product; we explore the links to the classical theory of optimal classification via matched filters and the effects of data memorization. Going further, we propose a simple penalty term that can be added to the cost function of any DN learning algorithm to force the templates to be orthogonal with each other; this leads to significantly improved classification performance and reduced overfitting with no change to the DN architecture. The spline partition of the input signal space that is implicitly induced by a MASO directly links DNs to the theory of vector quantization (VQ) and K-means clustering, which opens up new geometric avenue to study how DNs organize signals in a hierarchical fashion. To validate the utility of the VQ interpretation, we develop and validate a new distance metric for signals and images that quantifies the difference between their VQ encodings.

The widespread susceptibility of the current ML models to adversarial perturbations is an intensely studied but still mystifying phenomenon. A popular view is that these perturbations are aberrations that arise due to statistical fluctuations in the training data and/or high-dimensional nature of our inputs.

But is this really the case?

In this talk, I will present a new perspective on the phenomenon of adversarial perturbations. This perspective ties this phenomenon to the existence of "non-robust" features: features derived from patterns in the data distribution that are highly predictive, yet brittle and incomprehensible to humans. Such patterns turn out to be prevalent in our real-world datasets and also shed light on previously observed phenomena in adversarial robustness, including transferability of adversarial examples and properties of robust models. Finally, this perspective suggests that we may need to recalibrate our expectations in terms of how models should make their decisions, and how we should interpret them.
 

In this talk, I'd like to discuss two projects interpreting DNNs. The first project proposes the DeepTune framework as a way to elicit
interpretations of DNN-based models of single neurons in the difficult primate visual cortex area V4. Using DNN-based features, DeepTune images combine 18 accurately predictive regression models through a stability criterion. They provide characterizations of 71 V4 neurons and data-driven stimuli for closed-loop experiments.

The second project introduces agglomerative contextual decomposition (ACD) for hierarchical interpretations of DNN predictions. Using examples from Stanford Sentiment Treebank and ImageNet, we show that ACD is effective at diagnosing incorrect predictions and identifying dataset bias. We also find that ACD's hierarchy is largely robust to adversarial perturbations, implying that it captures fundamental aspects of the input and ignores spurious noise.

Several hundred papers have been written over the last few years proposing defenses to adversarial examples. Unfortunately, most proposed defenses to adversarial examples are quickly broken.

This talk surveys the ways in which defenses to adversarial examples have been broken in the past, and what lessons we can learn from these breaks. Beginning with a discussion of common evaluation pitfalls when performing the initial analysis, I then provide recommendations for how we can perform more thorough defense evaluations. I conclude with a discussion on recent directions in adversarial robustness research and promising future directions for defenses.

Model-based control is a popular paradigm for robot navigation because it can leverage a known dynamics model to efficiently plan robust robot trajectories. The key challenge in robot navigation is safely and efficiently operating in environments which are unknown ahead of time, and can only be partially observed through sensors on the robot. In this talk, we present our work in coupling learning-based perception with model-based control. The learning-based perception module is trained using optimal control, and it produces a series of waypoints that guides the robot to the goal via a collision free path. These waypoints are then used by a model-based planner to generate a smooth and dynamically feasible trajectory that is executed on the physical system using feedback control. Our experiments in simulated real-world cluttered environments and on an actual ground vehicle demonstrate that the proposed approach can reach goal locations more reliably and efficiently in novel environments as compared to purely mapping-based or end-to-end learning-based alternatives. We conclude with some lessons learned in using learning-based perception in a feedback control loop.

This is joint work with Somil Bansal, Varun Tolani, Saurabh Gupta, Andrea Bajcsy, and Jitendra Malik.

In almost all applications of deep learning, transfer learning -- where a deep neural network is pretrained on a first task, and then
finetuned on a different, target task -- plays a crucial role. This is especially the case in medical imaging applications, where the de-facto method is to use a standard large model from natural image datasets (e.g. ImageNet) and corresponding pretrained weights. However, the effects of transfer learning are poorly understood, with several recent papers in the natural image setting challenging commonly believed intuitions. For medical applications, fundamental differences between data and task specifications mean that these questions and many others remain unexplored. In this talk, I survey some of the recent results on understanding transfer learning in natural images, as well as findings on transfer learning for medical tasks. In this latter setting, we observe a number of counter-intuitive results, including connections between feature reuse and overparametrization, surprisingly strong performance of lightweight non-standard architectures, and even feature independent effects of transfer.

I will present my recent work on constructing generalization bounds for deep neural networks in order to understand existing learning algorithms and propose new ones. The tightness of generalization bounds varies widely, and depends on the complexity of the learning task and the amount of data available, but also on how much information the bounds take into consideration. My work is particularly concerned with data and algorithm-dependent bounds that are numerically nonvacuous. I will first present computational techniques that use PAC-Bayes bounds built from parameters obtained by stochastic gradient descent (SGD) and discuss the limitations of these bounds for particular choices of prior and posterior distributions on the parameters. I will then talk about my recent progress on tightening these bounds by constructing data-distribution dependent priors using the training data.

Joint work with Daniel M. Roy (University of Toronto, Vector Institute) and Waseem Gharbieh (Element AI), Alexander Lacoste (Element AI), Chin Wei (MILA and Element AI).