Date

Jan 21, 2020

3:30 PM
— 5:00 PM

Location

University of California Berkeley

Course Description:

This course covers topics related to the interplay between optimization and statistical learning. The first part of the course will cover the fundamentals, methods, and algorithms for dynamic programming, and optimal control. We will study approximate dynamic programming and the formulation and numerical implementation of several different algorithms and reinforcement learning methods. In addition we will study learning-based model predictive control (LBMPC), which is a method for robust adaptive optimization that can use machine learning to provide the adaptation online. The second part of the course will deal with inverse decision-making problems, which are problems where an agent’s decisions are observed and used to infer properties about the agent, such as preferences, utility functions, etc.

Course Syllabus - Spring 2020

Spring 2020 course evaluation

Course Material:

Lecture 1: Introduction to Dynamic Programming lecture notes - chess match example

Lecture 2: Deterministic Dynamic Programming lecture notes - lecture slides

Lecture 3: Hidden Markov Models lecture notes - lecture slides

Lecture 4: Stochastic DP and LQR model lecture notes - lecture slides

Lecture 5: Approximate DP and Reinforcement Learning lecture slides

Lecture 6: Approximation in Value Space lecture notes - lecture slides

Lecture 7: Approximation Architectures and (Deep) Neural Networks lecture notes - lecture slides

Lecture 8: Value Iteration and the DQN Algorithm lecture notes - lecture slides

Lecture 9: DQN Algorithm implementation lecture slides

Lecture 10: Policy Iteration lecture notes - lecture slides

Lecture 11: Approximate PI, Critic Algorithm and Policy Gradient lecture notes - lecture slides

Lecture 12: The Actor-Critic Algorithm lecture notes - lecture slides

Lecture 13: AlphaGo Implementation and Analysis lecture slides

Lecture 14: Monte-Carlo Tree Search lecture slides

Lecture 15: Model Predictive Control (MPC) lecture slides

Lecture 16: Establishing MPC Properties lecture slides

Lecture 17: Invariant Sets and Polyhedral Operations lecture slides

Lecture 18: Robust Model Predictive Control lecture slides

Lecture 19: Learning-based Model Predictive Control (LBMPC) lecture slides

Lecture 20: Recap of Approximate DP and Reinforcement Learning lecture slides

Lecture 21: Inverse Decision Making Problems lecture slides

Lecture 22: Inverse Optimization and Variational Inequalities lecture slides