Name: High-Level APIs in TensorFlow 2.0 (TensorFlow Meets)
Uploaded: 2020-03-31T06:56:10.000Z
Duration: 4 min 29 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Hi, everybody, and welcome to TensorFlow Meets.

It's my honor to be chatting with Karmel Allison, Engineering Manager

Now, Karmel, you do all of this outreach stuff,

and I know you did a great talk at the TensorFlow Developer Summit.

Can you tell us a little bit about you, what you do,

Yes, so as you mentioned, I'm an Engineering Manager for TensorFlow.

My team works on high-level APIs, so that's Estimator and Keras.

And my talk at the Dev Summit this year was about what we're bringing in 2.0

for high-level APIs, and, specifically, about Keras,

and how that's the primary high-level API

that we're consolidating a lot of the things we have under,

and bringing the scale of Estimator into Keras,

and how we're going to be doing that in 2.0.

That's really interesting-- bringing the scale of Estimator into Keras.

I know there's going to be tons of questions about that.

One of the things I thought was really interesting,

where there was this "spot the difference."

You had like training a DNN, I think it was in Fashion-MNIST in 1.13,

and then, in 2.0, and there was no difference between them.

Yes, so François Chollet, the creator of Keras,

is really, really one of the champions of the user experience,

and he's done a great job of that with the Keras API thus far,

and we wanted to be able to keep that-- minimize, first of all, the overhead

in having to convert for people who're already using Keras,

but also just to retain that simple API as we move into 2.0.

we were able to bring everything we're bringing to the table in 2.0,

So, that same model works in Eager Mode and in Graph Mode,

so the same one in 1.13 would be Graph Mode,

but in 2.0, it's going to be in Eager Mode,

which allows you to more easily debug, prototype, and all of that.

It also works with Distribution Strategies,

There's all these different tools and pieces that we're bringing in 2.0,

we wanted to make sure that same Keras API worked,

Now, you've mentioned Distribution Strategy.

So one of the things, it's not just making it easier

but the training part of the cycle and being able to go big.

So could you tell us a little bit about what Distribution Strategy

Yes, so Distribution Strategies is a set of strategies,

There are a number of them, including MirroredStrategy,

which is distribution across multiple GPUs on the same device or on the same machine.

There is also MultiWorkerMirroredStrategy

where you're mirroring across multiple machines,

And we've also got coming in the future ParameterServerStrategy

which is going to be distributing asynchronously

which is the kind of training we also do at Google.

It's really exciting to build that in as a simple API, a flexible API

that works for DeepMind, for Google researchers,

but also for the people who are outside of the TensorFlow repository right now.

To make it easy to use, but also really performant.

There's a lot of work that's gone in under the hood

to make sure that the scaling efficiency is really high

Nice, nice. And now this becomes available

- to almost anybody who wants to use it. - Yes, we hope.

So now, I'm going to put you on the spot for a minute,

because this all great new stuff that we've been talking about

in TensorFlow 2.0-- do you have a favorite?

I think some of the things I'm most excited about are tf.functions.

So, this is some of the magic we're bringing in in Eager Execution,

where you can actually continue to use graph style code

and get the performance of graph style code,

Another is what I just mentioned which is ParameterServerStrategy.

That's the way that we're going to be able to distribute

some of the largest workloads we have at Google, using Keras models.

And I know that there are a lot of researchers, internal to Google

and externally, at some of the largest companies we work with.

We're excited to be able to take the same model they prototype

and take it all the way to production, training, and serving,

That's something I'm really excited about.

Nice. It's hard to pick just one favorite, right?

- Thank you so much, Karmel! - Thank you!

This has been, as always, very informative,

and if you've got any questions for me, or if you've any questions for Karmel,

please, just leave them in the Comments below.

And whatever you do, don't forget to hit that Subscribe button,

so you'll be able to see the rest of Karmel's videos

Subtitles ListPlay Video

High-Level APIs in TensorFlow 2.0 (TensorFlow Meets)

stuff

subscribe

strategy

multiple