My name is Dan Moldovan.

And today I will talk about some of the internals

and functionality of AutoGraph.

Now, this is definitely not an introductory talk.

And if you would like to learn more about the background

or the motivation behind AutoGraph,

here are a few resources that I believe can help.

The talk will be otherwise fairly fast paced, quite dense.

I'm hoping we'll be able to get through all of it in time.

But if not, I'm hoping the slides

will be able to serve as a good reference,

should you decide to come back and look at everything more

closely.

I should caution, though, that I am

oversimplifying a lot of things for the sake of brevity

and time.

But the essential things are in there.

The talk will be structured in roughly in three parts.

First I'll talk some about some of the more relevant

implementation details, which are useful to understanding

some of AutoGraph's behavior.

Then I'll describe the various ways in which

you can interact with it.

And lastly, I'll go through various use cases that

highlight what works, what doesn't work, common pitfalls,

how to stay away from them, and what are our plans

to eventually address them.

So let's begin with the implementation.

From a systems perspective, this is roughly

what AutoGraph looks like.

In broad strokes, we have the following.

Going from the bottom to the top,

we have an infrastructure for performing source code

transformations with various helpers.

And on top of that, we have individual transformations.

For instance, there is a separate transformation

that handles function calls.

Another one handles break statements.

And yet another transformation handles if statements.

And these transformations are independent and composable.

Many of these transformations then

replace your code with calls to special AutoGraph functions.

We call them overloads or operators.

The is reason that they are similar to Python's

overloaded operators.

Now, of those overloads, there are the most interesting ones,

the ones that specialize on creating TensorFlow ops.

And lastly, there's a high-level API

that glues them all together.

And this is typically what you usually

interact with as a user.

One last note that I should make is

that of all these pieces, only the TensorFlow specialized

overloads and perhaps the high-level APIs,

only these are specific to TensorFlow.

Everything else is fairly generic and reusable,

and we hope to eventually have them

in a separate library that can be used for other purposes

as well.

So one of the fundamental pieces of AutoGraph

is, of course, the source code transformation bit.

So let's look at that a bit more closely.

Source code transformation is essentially

what makes AutoGraph a transpiler.

It's unit of work is functions.

That is at runtime, a function is being

converted into a new function.

So let's look more closely at that process.

It is roughly, loosely speaking, a five-step process.

The first step is to obtain the source code of the function.

Now, the standard Python library makes that easy for us.

It provides that inspect module, which is built in,

and it lets us do that.

This also highlights one of the fundamental requirements

of AutoGraph.

In order to convert a function, that function

must expose its source code.

And that's typically true for almost all functions in Python,

although there are certain exceptions.

Normally, you can test this on your function

by calling the inspect get source.

If inspect get source returns data,

then AutoGraph should be fine with it.

The second step in this process is

to parse the code into an AST.

And once more, there is a standard Python API

for this, which is good.

We, in fact, use a thin layer on top of that.

It's a third-party library called Gast.

It's practically identical to AST,

but it handles all the version differences

between Python 2 and Python 3.

It's worth mentioning at this point

that AutoGraph operates entirely at AST level.

There is no lower-level intermediate representation.

And we never interact with the bytecode.

And that has some unique advantages.

Now, the third step does the bulk of the work.

And that's quite both literally and figuratively.

The standard Python library offers a mechanism that

helps us with that as well.

The AST module provides a mechanism

for visiting and transforming ASTs.

That mechanism uses the visitor pattern,

and it's sketched here.

Basically you get some callbacks whenever the visitor encounters

different types of nodes.

And on top of that, we have built an entire library

of such transformations, as we've

seen in the previous diagram.

These transformations are called in sequence.

Now, once transformed, the AST is unparsed back

into source code in the form of a string.

There isn't a standard library for doing that.

But thankfully there's a third-party library

called Astor, which does a decent job of that.

Essentially, it's lots of string concatenations.

There's nothing special about that.

Finally, the source code is being outputted into a file

and then loaded using a mechanism that's

identical to writing an import statement.

Once more, Python helps us with that,

with the standard module called Imp.

The special thing about Imp is that it only works

with files on disk, hence the need

to generate a temporary file.

I should also make a slight note that another mechanism that we

could have used would be Exec.

And we've been going back and forth

between using that and Imp.

There are pros and cons to using each.

So we might revisit this in the future.

A few other mechanisms that are worth mentioning, one of them

is the templating system that we developed to help us

with generating code.

It essentially lets us write templates

in the form of strings, code blocks of strings.

And they support placeholders, and they

let us generate more complex or new ASTs.

If you ever poke inside the transformations library,

you will see plenty of such templates.

Another important piece is the static analysis,

which is critical in supporting certain transformations,

and we'll see more about that in a bit.

The analysis itself can be viewed as just a simple walk

over the AST.

And it annotates nodes with relevant information.

Another important mechanism is caching.

Caching itself is completely transparent to the user.

But it does help us make sure that every function is

converted no more than once, loosely speaking.

This cache relies on the key assumption

that the conversion process is entirely static.

That is the generated code, what ends up

in the generated code does not depend

on any arguments or variables or any other state of Python.

Basically, if you look at some plain code on paper,

you would know exactly what the output code should be.

Next let's talk about some of the actual transformations that

are being made.

And before I proceed, I should clarify that I'll

use the word variable a lot.

These are meant to mean Python variables, not

to be confused with TensorFlow variables, which

are not involved here.

Once such transformation is simplifying the code

by replacing some statements with other statements, simpler

ones.

Such as, for instance, we replace break statements

with variables and additional if statements.

This essentially helps us avoid the need

to add to build special handlers in TensorFlow

for these statements, like break.

They're just easier to lower into something simpler.

And the process-- yes?

AUDIENCE: Does that imply that if I

do while n less than a million break,

that that's going to be very efficient when inefficiently

converted, because it will still loop over the maximal range?

DAN MOLDOVAN: It will not loop over the maximum range

because the while statement, as seen in this example,

will have its condition augmented.

So yes, the overhead is one maybe two extra conditionals,

not more than that.

Yes?

AUDIENCE: Are you guaranteed a [INAUDIBLE] variable names?

DAN MOLDOVAN: We have small modules.

It's not mentioned here [INAUDIBLE]..

We look at the function's globals, its closure.

And those indeed depend on the context variables.

So if you take a function, you convert it,

and you get a certain name.

And then suppose you create some other function.

And then you run the converted function.

It might clash.

That's very unlikely.

Because if you change the code that we transformed,

then the function will be re-converted, right?

So I'm not sure it's even possible to get a--

well, you could get a clash, in theory,

but you would have to work very hard to do that.

But, yeah, that's a very good observation.

That is one of the deviations from converted entirely static.

There are some minor exceptions.

So going back to the loading, I'm

not going to describe the entire process because it's

fairly straightforward.

I think an example would suffice.

For instance, here, notice that the break statement

was replaced with a did break variable.

And then we have a few extra conditionals,

like, for instance, this one at the bottom, if did not break,

i star equals to 2 to protect the code.

And the conversions for continue and return statements

are similar.

Another important type of conversion

is for function calls.

We do this for all function calls.

We replace them with a call to a special wrapper.

This wrapper, as its name suggests,

might decide to convert the function at the runtime,

to convert the target function at runtime.

But it may decide not to do that.

Many functions don't need to be converted.

But the important part is that we

replace all function calls because statically we

do not know what their type is.

And we do not know whether we should convert them or not.

So we defer that decision to runtime.

Another mention that's probably worth making here

is that from the graph's perspective,

functions are in line.

We don't in line.

We don't create any TF function.

We don't create any graph functions.

So from this perspective, AutoGraph is consistent with

[? V1-style ?] graph code.

AUDIENCE: What do you mean by runtime?

Do you mean when TensorFlow runs it

or when the Python user runs it?

DAN MOLDOVAN: That's a very good question.

There's more than one runtime.

In this case, I'm referring to the Python runtime.

Next, the quintessential transformation, we might say,

is converting control flow.

So let's look at if statements first.

The transformation itself is actually quite mechanical.

For example, the body of the if statement

becomes a new function, and we add some return values to it.

And we'll see more about that in a moment.

And the if itself becomes a call to a special AutoGraph