Name: TFRT: A new TensorFlow runtime (TF Dev Summit '20)
Uploaded: 2020-03-25T08:22:21.000Z
Duration: 10 min 10 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

MINGSHENG HONG: I'm Mingsheng Hong, tech lead and manager

Today, I'm excited to share with you our new project,

As you probably guessed, it stands for none other

Some people might tell you that, in the world of TensorFlow,

But since we are here talking about the runtime,

Here's a diagram on the training workflow.

Runtime is a low-level component that orchestrates all model

execution by calling into the relevant kernels that

implement machine learning primitives like matrix

We're building TFRT to replace the existing runtime.

We talked to many of you, our TensorFlow users,

and heard from your pain points and requests.

First, many of you are pushing the envelope in the performance

Second, you are making continuous innovations

through the addition of ops, kernels, and devices

And we need to make such extension work more

And once you are done with the model research and tuning,

you'll want to deploy TensorFlow everywhere,

across a diverse set of hardware platforms.

For those great reasons, we're building a new runtime

to help you, a new runtime that provides the performance,

extensibility, and unification that you all are looking for.

So how does TFRT fit into the workflow of an ML model?

Here, we see the TF training stack again.

can either eagerly dispatch ops to the runtime,

as you can see from the blue arrows on the left side

Or as the red arrows on the right side show,

in the case of graph execution, your program first

gets lowered to the optimized target-specific program,

uses the MLIR compiler framework, which Jacques just

to complete a model execution, as the purple arrow shows.

to device-specific operations, like a GPU-based matrix

TFRT orchestrates the efficient kernel execution

Now let's dive a little more into the technical design

and look at how we realized the vision of building

a performant, extensible, and unified runtime.

We have also made the eager op dispatch stack very, very thin.

And the eager API course will call into the relevant kernels

drives host CPU and I/O work, and it also

drives locally attached devices through the device runtimes.

TFRT keeps device runtimes separate from the host runtime,

so that wheere you add new device runtimes,

you don't have to extend the rest of the runtime.

common abstractions, such as shape functions and kernels,

to be used in both graph and eager execution.

Now, if you feel a bit lost in the last slide,

Let's step back and let's look at how these key design

decisions will benefit the core TensorFlow use cases.

For those of you who care about training,

you will see improved performance as well as

And that should make it easier to debug your models.

If you deploy TensorFlow models in production,

you'll be glad to see some improved performance

And I will show you in a benchmarking study shortly.

Subtitles ListPlay Video

TFRT: A new TensorFlow runtime (TF Dev Summit '20)

specific

relevant

improve

average