AKSHAY AGRAWAL: Hi, my name is Akshay.

I'm a PhD candidate at Stanford.

And today, I'm going to talk about some recent research that

makes it possible to embed convex optimization

problems into TensorFlow.

This is an optimization problem.

And in an optimization problem, the goal

is to find a value for a variable x that

minimizes the cost function while also satisfying

some constraints.

So here, the variable might represent a decision,

or it might represent weights in a machine learning model,

or it could be the design of a physical device.

And one thing to notice is that an optimization problem

is a declarative object.

It's not a procedural method.

So basically, all you have to do is

to articulate the objective function

f, which measures how much you dislike any given value of x.

And then you specify the constraints

that your variable has to satisfy.

And also notice that the optimization problem here

is parameterized by a vector of theta.

And this vector determines the shape of the objective function

and the constraints.

And it's going to be important later.

Unfortunately, most optimization problems

are computationally intractable, but convex optimization

problems are the subset that we can solve easily.

Convex optimization has a lot of properties

that make a useful tool.

So for one, convex optimization problems

can be solved exactly and very quickly,

up to thousands or even millions of times a second.

Convex optimization is also easy to use in practice, thanks

to software libraries like CVXPY.

And to make things really concrete,

here's a five-line snippet of code that constructs and solves

a convex optimization problem, using CVXPY.

So first, you construct a variable.

Then you construct the objective function

and a list of constraints.

Then the objective and the constraints

are used to construct a problem object,

and you can call the solve method on this object

to produce a value of x that solves the problem, i.e.,

it minimizes the objective function

and satisfies the constraints.

And I guess the only reason that any of this matters

is that convex optimization has tons of applications.

It's got many more applications than people once

thought actually.

Now I'll really quickly give you guys a few examples.

Convex optimization is used on board self-driving cars

for tasks like generating trajectories and tracking

paths in real-time.

It's also used to control actuators in spacecrafts

and to generate landing trajectories for rockets.

Convex optimization has even been successfully used

to design airplanes and other physical structures.

Until now, it was very difficult, if not impossible,

to use convex optimization problems

in TensorFlow pipelines.

And as a result, the parameters theta

in the optimization problem were chosen and tuned by hand.

So this means that the structure of the problem

was often painstakingly crafted by a human.

This process of choosing parameters in a problem

is carried out in basically every field.

And in different fields, it goes by different names.

So in machine learning, we know it as hyperparameter tuning.

It's an essential part of controls engineering.

In finance, it's called back testing.

But if machine learning has taught us anything,

it's that we should prefer gradient-based tuning

of parameters over manual tuning whenever possible.

And this observation gets to the heart of this talk.

So last year, my lab mates and I figured out

how to differentiate through convex optimization problems.

This makes it possible for the first time

to tune the parameters in a convex optimization

problem using gradient descent.

Once we figured out the math, we wrote

a software library called CVXPY Layers that

makes the math actionable.

Our library turns a CVXPY problem

into a differential convex optimization layer

that you can use in TensorFlow.

And the basic idea is to view a convex optimization

problem as a function mapping parameters

to an optimal solution.

So on this slide, the layer is represented by the function x

star of theta.

And x star of theta is the argmin of a convex optimization

problem parameterized by theta.

Different values of theta lead to different convex

optimization problems and different solutions.

And just like any other layer in TensorFlow,

a CVXPY layer has a forward pass and a backward pass.

In the forward pass of the layer,

the parameters are taken as input.

Then the forward pass solves the CVXPY problem

and outputs a solution.

The backward pass computes the gradient

of the solution with respect to the parameters.

And here's the upshot.

Our software lets you learn the parameter's theta

in a convex optimization problem using gradient descent.

And this automates what has traditionally

been a very manual process.

So let's walk through a short code snippet

that shows how to use a CVXPY layer in TensorFlow.

In the first three lines highlighted here,

we import three packages.

We import CVXPY to construct the problem, TensorFlow.

And then we import CVXPY layers.

In this next highlighted block, we

construct a simple convex optimization problem

using CVXPY.

In line four, we constructed variable x.

In line five, we declared the parameters

in the optimization problem, which are A and b.

So A and b here correspond to the vector of theta

that we saw in the previous slides.

In line six, we constrain x to be non-negative.

In line seven, we construct the objective,

which, here, is to minimize the one norm of Ax minus b.

And in line eight, we construct the problem.

In line nine, we construct a CVXPY layer.

And all we have to do in order to do

this is to pass in the problem, the parameters,

and the variable to the CVXPY layer constructor.

And we can now use this layer in TensorFlow.

In this last highlighted block in lines 10 and 11,

we construct TensorFlow variables

that represent the parameters A and b in the problem.

And line 13 is where the magic starts to happen.

So in line 13, we solve the convex optimization problem.

And notice that the CVXPY layer is a callable object.

So you pass it the parameters, and it returns a solution

to the corresponding convex optimization problems.

And the details of the solution algorithm

are completely abstracted away from you, the user.

A lot of stuff happens behind the scenes,

but we abstract all that away.

In line 15, we compute the gradient

of the solution with respect to A and b, the parameters.

And again, the details of the derivative

are completely abstracted away from you.

And that's it.

So in just 15 lines of code, we constructed

a convex optimization layer, solved the convex optimization

problem, and then we differentiated through it.

I'll end by showing just one example of the things

that you can do with CVXPY layers.

And in this example, we're going to learn how

to control a self-driving car.

So the setting is we've got a car,

and we want to learn a control policy that allows it to track

a given path at a given speed.

We can control the car by choosing

its acceleration and its steering angle

at each time step.

And these two things are the control inputs.

So today, in the real world, you might choose the controls using

something like PID, or you might use a painstakingly

hand-tuned controller.

So in this example, we use a CVXPY layer

as the policy that chooses the control inputs.

And this allows us to naturally incorporate

constraints on the acceleration and the steering angle

in the policy itself.

In order to learn the policy, we randomly

sample a bunch of different paths,

and we tune the parameters in the layer

using gradient descent.

The objective is to minimize some cost function that

measures how well we're tracking the path and the speed.

So this plot shows the performance of a control policy

on a randomly sampled path, but before any training happened.

The solid black curve is the path

that we're trying to track, and the dotted curve is the path

that the car actually took.

So clearly, the untrained policy doesn't do too well.

So in order to try to improve the performance,

we train the policy for 100 iterations in simulation.

In each iteration, we solve a bunch of convex optimization

problems, and then we differentiate

through the sequence of optimization problems

using gradient descent to update the parameters.

The trained policy does a lot better.

That's the plot on the right here.

So the same desired path is actually

plotted on both the left and the right, the car on the left

just traveled more slowly and only made it

to the top of the loop.

So after 100 iterations of training,

the trained policy tracks the path much more accurately.

So if any of that sounded interesting to you,

I would encourage you to check out the links on this slide.

Our software package, CVXPY Layers, is online at GitHub.

And you can also download it using Pit.

If you want to learn more about the inner workings

of convex optimization layers, you

can check out our NeurIPS paper.

And if you like that car example we just saw,

check out our recent paper in L4DC,

which develops that example and a lot of other examples

in a lot more detail.

And the code for all these examples is online.

And that's it.

Thank you, guys, for listening.

Next up is Rohan to tell you about TF.data.

[MUSIC PLAYING]