Subtitles section Play video Print subtitles [MUSIC PLAYING] AKSHAY AGRAWAL: Hi, my name is Akshay. I'm a PhD candidate at Stanford. And today, I'm going to talk about some recent research that makes it possible to embed convex optimization problems into TensorFlow. This is an optimization problem. And in an optimization problem, the goal is to find a value for a variable x that minimizes the cost function while also satisfying some constraints. So here, the variable might represent a decision, or it might represent weights in a machine learning model, or it could be the design of a physical device. And one thing to notice is that an optimization problem is a declarative object. It's not a procedural method. So basically, all you have to do is to articulate the objective function f, which measures how much you dislike any given value of x. And then you specify the constraints that your variable has to satisfy. And also notice that the optimization problem here is parameterized by a vector of theta. And this vector determines the shape of the objective function and the constraints. And it's going to be important later. Unfortunately, most optimization problems are computationally intractable, but convex optimization problems are the subset that we can solve easily. Convex optimization has a lot of properties that make a useful tool. So for one, convex optimization problems can be solved exactly and very quickly, up to thousands or even millions of times a second. Convex optimization is also easy to use in practice, thanks to software libraries like CVXPY. And to make things really concrete, here's a five-line snippet of code that constructs and solves a convex optimization problem, using CVXPY. So first, you construct a variable. Then you construct the objective function and a list of constraints. Then the objective and the constraints are used to construct a problem object, and you can call the solve method on this object to produce a value of x that solves the problem, i.e., it minimizes the objective function and satisfies the constraints. And I guess the only reason that any of this matters is that convex optimization has tons of applications. It's got many more applications than people once thought actually. Now I'll really quickly give you guys a few examples. Convex optimization is used on board self-driving cars for tasks like generating trajectories and tracking paths in real-time. It's also used to control actuators in spacecrafts and to generate landing trajectories for rockets. Convex optimization has even been successfully used to design airplanes and other physical structures. Until now, it was very difficult, if not impossible, to use convex optimization problems in TensorFlow pipelines. And as a result, the parameters theta in the optimization problem were chosen and tuned by hand. So this means that the structure of the problem was often painstakingly crafted by a human. This process of choosing parameters in a problem is carried out in basically every field. And in different fields, it goes by different names. So in machine learning, we know it as hyperparameter tuning. It's an essential part of controls engineering. In finance, it's called back testing. But if machine learning has taught us anything, it's that we should prefer gradient-based tuning of parameters over manual tuning whenever possible. And this observation gets to the heart of this talk. So last year, my lab mates and I figured out how to differentiate through convex optimization problems. This makes it possible for the first time to tune the parameters in a convex optimization problem using gradient descent. Once we figured out the math, we wrote a software library called CVXPY Layers that makes the math actionable. Our library turns a CVXPY problem into a differential convex optimization layer that you can use in TensorFlow. And the basic idea is to view a convex optimization problem as a function mapping parameters to an optimal solution. So on this slide, the layer is represented by the function x star of theta. And x star of theta is the argmin of a convex optimization problem parameterized by theta. Different values of theta lead to different convex optimization problems and different solutions. And just like any other layer in TensorFlow, a CVXPY layer has a forward pass and a backward pass. In the forward pass of the layer, the parameters are taken as input. Then the forward pass solves the CVXPY problem and outputs a solution. The backward pass computes the gradient of the solution with respect to the parameters. And here's the upshot. Our software lets you learn the parameter's theta in a convex optimization problem using gradient descent. And this automates what has traditionally been a very manual process. So let's walk through a short code snippet that shows how to use a CVXPY layer in TensorFlow. In the first three lines highlighted here, we import three packages. We import CVXPY to construct the problem, TensorFlow. And then we import CVXPY layers. In this next highlighted block, we construct a simple convex optimization problem using CVXPY. In line four, we constructed variable x. In line five, we declared the parameters in the optimization problem, which are A and b. So A and b here correspond to the vector of theta that we saw in the previous slides. In line six, we constrain x to be non-negative. In line seven, we construct the objective, which, here, is to minimize the one norm of Ax minus b. And in line eight, we construct the problem. In line nine, we construct a CVXPY layer. And all we have to do in order to do this is to pass in the problem, the parameters, and the variable to the CVXPY layer constructor. And we can now use this layer in TensorFlow. In this last highlighted block in lines 10 and 11, we construct TensorFlow variables that represent the parameters A and b in the problem. And line 13 is where the magic starts to happen. So in line 13, we solve the convex optimization problem. And notice that the CVXPY layer is a callable object. So you pass it the parameters, and it returns a solution to the corresponding convex optimization problems. And the details of the solution algorithm are completely abstracted away from you, the user. A lot of stuff happens behind the scenes, but we abstract all that away. In line 15, we compute the gradient of the solution with respect to A and b, the parameters. And again, the details of the derivative are completely abstracted away from you. And that's it. So in just 15 lines of code, we constructed a convex optimization layer, solved the convex optimization problem, and then we differentiated through it. I'll end by showing just one example of the things that you can do with CVXPY layers. And in this example, we're going to learn how to control a self-driving car. So the setting is we've got a car, and we want to learn a control policy that allows it to track a given path at a given speed. We can control the car by choosing its acceleration and its steering angle at each time step. And these two things are the control inputs. So today, in the real world, you might choose the controls using something like PID, or you might use a painstakingly hand-tuned controller. So in this example, we use a CVXPY layer as the policy that chooses the control inputs. And this allows us to naturally incorporate constraints on the acceleration and the steering angle in the policy itself. In order to learn the policy, we randomly