Placeholder Image

Subtitles section Play video

  • DAN MOLDOVAN: Thank you all for coming.

  • My name is Dan Moldovan.

  • And today I will talk about some of the internals

  • and functionality of AutoGraph.

  • Now, this is definitely not an introductory talk.

  • And if you would like to learn more about the background

  • or the motivation behind AutoGraph,

  • here are a few resources that I believe can help.

  • The talk will be otherwise fairly fast paced, quite dense.

  • I'm hoping we'll be able to get through all of it in time.

  • But if not, I'm hoping the slides

  • will be able to serve as a good reference,

  • should you decide to come back and look at everything more

  • closely.

  • I should caution, though, that I am

  • oversimplifying a lot of things for the sake of brevity

  • and time.

  • But the essential things are in there.

  • The talk will be structured in roughly in three parts.

  • First I'll talk some about some of the more relevant

  • implementation details, which are useful to understanding

  • some of AutoGraph's behavior.

  • Then I'll describe the various ways in which

  • you can interact with it.

  • And lastly, I'll go through various use cases that

  • highlight what works, what doesn't work, common pitfalls,

  • how to stay away from them, and what are our plans

  • to eventually address them.

  • So let's begin with the implementation.

  • From a systems perspective, this is roughly

  • what AutoGraph looks like.

  • In broad strokes, we have the following.

  • Going from the bottom to the top,

  • we have an infrastructure for performing source code

  • transformations with various helpers.

  • And on top of that, we have individual transformations.

  • For instance, there is a separate transformation

  • that handles function calls.

  • Another one handles break statements.

  • And yet another transformation handles if statements.

  • And these transformations are independent and composable.

  • Many of these transformations then

  • replace your code with calls to special AutoGraph functions.

  • We call them overloads or operators.

  • The is reason that they are similar to Python's

  • overloaded operators.

  • Now, of those overloads, there are the most interesting ones,

  • the ones that specialize on creating TensorFlow ops.

  • And lastly, there's a high-level API

  • that glues them all together.

  • And this is typically what you usually

  • interact with as a user.

  • One last note that I should make is

  • that of all these pieces, only the TensorFlow specialized

  • overloads and perhaps the high-level APIs,

  • only these are specific to TensorFlow.

  • Everything else is fairly generic and reusable,

  • and we hope to eventually have them

  • in a separate library that can be used for other purposes

  • as well.

  • So one of the fundamental pieces of AutoGraph

  • is, of course, the source code transformation bit.

  • So let's look at that a bit more closely.

  • Source code transformation is essentially

  • what makes AutoGraph a transpiler.

  • It's unit of work is functions.

  • That is at runtime, a function is being

  • converted into a new function.

  • So let's look more closely at that process.

  • It is roughly, loosely speaking, a five-step process.

  • The first step is to obtain the source code of the function.

  • Now, the standard Python library makes that easy for us.

  • It provides that inspect module, which is built in,

  • and it lets us do that.

  • This also highlights one of the fundamental requirements

  • of AutoGraph.

  • In order to convert a function, that function

  • must expose its source code.

  • And that's typically true for almost all functions in Python,

  • although there are certain exceptions.

  • Normally, you can test this on your function

  • by calling the inspect get source.

  • If inspect get source returns data,

  • then AutoGraph should be fine with it.

  • The second step in this process is

  • to parse the code into an AST.

  • And once more, there is a standard Python API

  • for this, which is good.

  • We, in fact, use a thin layer on top of that.

  • It's a third-party library called Gast.

  • It's practically identical to AST,

  • but it handles all the version differences

  • between Python 2 and Python 3.

  • It's worth mentioning at this point

  • that AutoGraph operates entirely at AST level.

  • There is no lower-level intermediate representation.

  • And we never interact with the bytecode.

  • And that has some unique advantages.

  • Now, the third step does the bulk of the work.

  • And that's quite both literally and figuratively.

  • The standard Python library offers a mechanism that

  • helps us with that as well.

  • The AST module provides a mechanism

  • for visiting and transforming ASTs.

  • That mechanism uses the visitor pattern,

  • and it's sketched here.

  • Basically you get some callbacks whenever the visitor encounters

  • different types of nodes.

  • And on top of that, we have built an entire library

  • of such transformations, as we've

  • seen in the previous diagram.

  • These transformations are called in sequence.

  • Now, once transformed, the AST is unparsed back

  • into source code in the form of a string.

  • There isn't a standard library for doing that.

  • But thankfully there's a third-party library

  • called Astor, which does a decent job of that.

  • Essentially, it's lots of string concatenations.

  • There's nothing special about that.

  • Finally, the source code is being outputted into a file

  • and then loaded using a mechanism that's

  • identical to writing an import statement.

  • Once more, Python helps us with that,

  • with the standard module called Imp.

  • The special thing about Imp is that it only works

  • with files on disk, hence the need

  • to generate a temporary file.

  • I should also make a slight note that another mechanism that we

  • could have used would be Exec.

  • And we've been going back and forth

  • between using that and Imp.

  • There are pros and cons to using each.

  • So we might revisit this in the future.

  • A few other mechanisms that are worth mentioning, one of them

  • is the templating system that we developed to help us

  • with generating code.

  • It essentially lets us write templates

  • in the form of strings, code blocks of strings.

  • And they support placeholders, and they

  • let us generate more complex or new ASTs.

  • If you ever poke inside the transformations library,

  • you will see plenty of such templates.

  • Another important piece is the static analysis,

  • which is critical in supporting certain transformations,

  • and we'll see more about that in a bit.

  • The analysis itself can be viewed as just a simple walk

  • over the AST.

  • And it annotates nodes with relevant information.

  • Another important mechanism is caching.

  • Caching itself is completely transparent to the user.

  • But it does help us make sure that every function is

  • converted no more than once, loosely speaking.

  • This cache relies on the key assumption

  • that the conversion process is entirely static.

  • That is the generated code, what ends up

  • in the generated code does not depend

  • on any arguments or variables or any other state of Python.

  • Basically, if you look at some plain code on paper,

  • you would know exactly what the output code should be.

  • Next let's talk about some of the actual transformations that

  • are being made.

  • And before I proceed, I should clarify that I'll

  • use the word variable a lot.

  • These are meant to mean Python variables, not

  • to be confused with TensorFlow variables, which

  • are not involved here.

  • Once such transformation is simplifying the code

  • by replacing some statements with other statements, simpler

  • ones.

  • Such as, for instance, we replace break statements

  • with variables and additional if statements.

  • This essentially helps us avoid the need

  • to add to build special handlers in TensorFlow

  • for these statements, like break.

  • They're just easier to lower into something simpler.

  • And the process-- yes?

  • AUDIENCE: Does that imply that if I

  • do while n less than a million break,

  • that that's going to be very efficient when inefficiently

  • converted, because it will still loop over the maximal range?

  • DAN MOLDOVAN: It will not loop over the maximum range

  • because the while statement, as seen in this example,

  • will have its condition augmented.

  • So yes, the overhead is one maybe two extra conditionals,

  • not more than that.

  • Yes?

  • AUDIENCE: Are you guaranteed a [INAUDIBLE] variable names?

  • DAN MOLDOVAN: We have small modules.

  • It's not mentioned here [INAUDIBLE]..

  • We look at the function's globals, its closure.

  • And those indeed depend on the context variables.

  • So if you take a function, you convert it,

  • and you get a certain name.

  • And then suppose you create some other function.

  • And then you run the converted function.

  • It might clash.

  • That's very unlikely.

  • Because if you change the code that we transformed,

  • then the function will be re-converted, right?

  • So I'm not sure it's even possible to get a--

  • well, you could get a clash, in theory,

  • but you would have to work very hard to do that.

  • But, yeah, that's a very good observation.

  • That is one of the deviations from converted entirely static.

  • There are some minor exceptions.

  • So going back to the loading, I'm

  • not going to describe the entire process because it's

  • fairly straightforward.

  • I think an example would suffice.

  • For instance, here, notice that the break statement

  • was replaced with a did break variable.

  • And then we have a few extra conditionals,

  • like, for instance, this one at the bottom, if did not break,

  • i star equals to 2 to protect the code.

  • And the conversions for continue and return statements

  • are similar.

  • Another important type of conversion

  • is for function calls.

  • We do this for all function calls.

  • We replace them with a call to a special wrapper.

  • This wrapper, as its name suggests,

  • might decide to convert the function at the runtime,

  • to convert the target function at runtime.

  • But it may decide not to do that.

  • Many functions don't need to be converted.

  • But the important part is that we

  • replace all function calls because statically we

  • do not know what their type is.

  • And we do not know whether we should convert them or not.

  • So we defer that decision to runtime.

  • Another mention that's probably worth making here

  • is that from the graph's perspective,

  • functions are in line.

  • We don't in line.

  • We don't create any TF function.

  • We don't create any graph functions.

  • So from this perspective, AutoGraph is consistent with

  • [? V1-style ?] graph code.

  • AUDIENCE: What do you mean by runtime?

  • Do you mean when TensorFlow runs it

  • or when the Python user runs it?

  • DAN MOLDOVAN: That's a very good question.

  • There's more than one runtime.

  • In this case, I'm referring to the Python runtime.

  • Next, the quintessential transformation, we might say,

  • is converting control flow.

  • So let's look at if statements first.

  • The transformation itself is actually quite mechanical.

  • For example, the body of the if statement

  • becomes a new function, and we add some return values to it.

  • And we'll see more about that in a moment.

  • And the if itself becomes a call to a special AutoGraph

  • overload.

  • I'm, of course omitting the else block here

  • for the sake of brevity.

  • But it's equivalent with the main block,

  • main body of the if statement.

  • Once more, all the if statements are converted in this way.

  • And here we have an example for a statement.

  • Note that and there's nothing out of the ordinary here.

  • The body of the if becomes a body in a new function.

  • And the if statement is replaced with the function call