Placeholder Image

Subtitles section Play video

  • [MUSIC PLAYING]

  • TRIS WARKENTIN: Hi, everyone.

  • I'm Tris Warkentin and I'm a product manager

  • on TensorFlow Extended, or TFX.

  • ZHITAO LI: Hi, my name is Zhitao Li.

  • I'm a tech lead manager from TFX Open Source.

  • TRIS WARKENTIN: At Google, putting machine learning models

  • into production is one of the most important things

  • our engineers and researchers do.

  • But to achieve this global reach and production readiness,

  • a reliable production platform is

  • critical to Google's success.

  • And that's the goal of TensorFlow Extended--

  • to create a stable platform for production ML at Google,

  • and a stable platform for you to build production-ready ML

  • systems, too.

  • So how does that work?

  • Our philosophy is to take modern software engineering

  • and combine it with what we've learned about machine learning

  • development at Google.

  • So what's the difference between writing code and doing machine

  • learning engineering?

  • In coding, you might build something

  • that one person can create end to end.

  • You might have untested code, undocumented code,

  • and code that's hard to reuse.

  • In modern software engineering, we

  • have solutions for all of those problems-- test-driven

  • development, modular designs, scalable performance

  • optimization, and much more.

  • So how is that different in machine learning development?

  • Well, a lot of the problems from coding still apply to ML,

  • but we also have a variety of new problems.

  • We have no problem statements.

  • We might need some continuous optimization.

  • We might need to understand when changes in data

  • will result in different shapes of our models.

  • We've been doing this at Google for a long time.

  • In 2007, we launched Sibyl, which

  • was our production scalable platform for production ML

  • here at Google.

  • And since 2016, we've been working on TFX,

  • and last year we open sourced it to make

  • it even easier for you to build production

  • ML in your platforms.

  • What does it look like in practice?

  • Well, the entirety of TFX as an end to end platform

  • runs from best practices all the way through to end to end.

  • From best practices, you don't even

  • have to use a single line of Google developed code

  • in order to get some of the best of TFX, all the way

  • through to end to end pipelines that allow you to deploy

  • scalable production scale ML.

  • This is what a pipeline might look like.

  • On the left side of the screen, you'll see data intake,

  • then it runs through the pipeline,

  • doing things like data validation, schema generation,

  • and much more in order to make sure

  • that you're doing things in a repeatable, testable,

  • consistent way and producing ML production results.

  • So it's hard to believe that we've only

  • been one year in open source for our end

  • to end pipeline offering, but we have

  • a lot of interesting things that we've done in 2019,

  • including building the foundations of metadata,

  • building basic 2.0 support for things like estimators,

  • as well as launches of Fairness Indicators and TFMA.

  • But we're definitely not done.

  • In 2020, we have a wide variety of interesting developments

  • coming, including NativeKeras on TFX,

  • which you'll hear more about from Zhitao later,

  • as well as TensorFlow Lite trainer rewrite

  • and some warm starting, which can make your machine learning

  • training a hundred times faster by using caching.

  • But we have something really exciting

  • that we're announcing today, which

  • you may have heard from Megan about in the keynote, which

  • is end to end ML pipelines.

  • These are our Cloud AI Platform Pipelines.

  • We're really excited about these, because they combine

  • a lot of the best of Google AI Platform with TFX

  • to create Cloud AI Platform Pipelines, available today.

  • Please check out our blog for more information.

  • You should be able to find it if you just Google

  • "Cloud API Platform Pipelines."

  • And now, can we please cut to the demo?

  • ZHITAO LI: So I'll be giving an explanation about this demo.

  • This is the Cloud AI Platform Pipelines page.

  • As you see, you can see all your existing Cloud AI Pipeline

  • clusters in this page.

  • We've already created one, and this page

  • can be found at AI Platforms Pipelines tab

  • from the left of the Google Cloud Console.

  • If you don't have any pipelines cluster created yet,

  • you can use the New Instance button to create a new one.

  • This gives you a click button experience

  • while creating clusters, which is usually

  • one of the difficult jobs in the past.

  • You can use the Config button to create a Cloud AI

  • Pipelines on Google Cloud.

  • This gives you Cloud AI Pipelines space on Kubenetes.

  • Runs on Google's GKE.

  • You can choose a class it will run it from, choose

  • the namespace where you want to create a class for inside,

  • and choose a name of a cluster.

  • Once you are done, you can simply click Deploy and Done.

  • Since I already have a cluster, I

  • will open up the Pipeline dashboard here.

  • In this page, you can see a list of demo pipelines

  • that you can play with.

  • You can see tutorials about creating pipelines and doing

  • various techniques, and you can use the Pipelines tab

  • from the left to view all your existing pipelines here.

  • Since this class is newly created,

  • there is no TFX pipelines in it yet.

  • We are going to use the newly launched TFX

  • templates to create the Cloud AI Pipelines in this cluster.

  • This is the Cloud AI Notebook.

  • I'm pretty much using this as a Python shell

  • to write some simple Python commands.

  • First, you set up your environments

  • and then making sure TFX is properly installed, together

  • with some other dependencies.

  • Making sure you have environment variables

  • like Path Properties set up, and the TFX version is up to date.

  • Now you're making sure you have a Google Cloud project--

  • perfect config.

  • In the Config, the Cloud AI Pipelines cluster endpoint.

  • Simply copy that from the URL into the Notebook shell.

  • Now, we're also making sure we create the Google Container

  • Image repo so that we can upload our containers, too.

  • Once that is done, we config the pipeline name and the project

  • directory.

  • Now we can use the Template Creation

  • to create a new template.

  • Since they are created, I'm going

  • to show the content created by the templates.

  • As you see, there are pipeline code in the pipeline.py file.

  • This includes our classical taxi pipeline

  • from TFX with all the components necessary to do

  • production machine learning.

  • There is configs.py, with some configurations

  • related to Google Cloud as well as some configuration about TFX

  • itself.

  • Once that is done, we enter the Templates Directory,

  • making sure all the templates valid are there.

  • You can even run some pre-generated unit test

  • on the features to making sure the configuration looks right.

  • Once that's done, you can then use TFX CLI command

  • to create a TFX pipeline on Google Cloud Pipelines page.

  • This will create a temporary image

  • with all your code and the dependencies.

  • Upload them to DCR, then create a pipeline using this container

  • image on the Pipelines page.

  • As we see, the pipeline compiles and the creation is successful,

  • and we go back to the Pipeline page.

  • Click on Refresh.

  • Boom-- we have our new pipeline.

  • Now, if we click through the pipeline,

  • you are going to see all the TFX components here

  • readily available.

  • We can create a test to run on this one.

  • And click on the Run.

  • We are going to see each of the components.

  • When they run, they will gradually

  • show up on the web page.

  • The first component should be ExampleGen.

  • So it has to be ExampleGen. Yes, it is there.

  • This component has started running.

  • You can click on it.

  • On the tab, you can look at the artifacts, inputs and outputs,

  • what Kubernetes' volumes used for the component, manifest,

  • and you can even inspect the logs of a component run.

  • We call this ExampleGen, StatsGen, SchemaGen.

  • And now the pipeline enters feature transform

  • and the example validation at the same time.

  • So now all the data preparation is finished.

  • The pipeline enters into a training stage,

  • which is producing a TensorFlow model.

  • If we click on the trainer component,

  • we can even inspect these logs.

  • Now, once trainer is complete, we

  • do some model validation and evaluation

  • using TFX components.

  • And once all the model evaluation is done,

  • we use Pusher to push the generated model

  • onto external serving system.

  • So everything-- so you have a model

  • ready to use in production.

  • You can also use the tabs on the left

  • to navigate on existing experiments, artifacts,

  • and executions.

  • We are going to take a look at the artifacts generated

  • from this pipeline using the Artifacts tab.

  • So here, you can see you have a pipeline.

  • If you click on the Model Output Artifacts from Trainer,

  • that represents a TensorFlow model.

  • This is the artifact ML metadata.

  • We can see it's a model artifact produced by trainer.

  • And this is a lineage view of the model--

  • this explains what components produced this model

  • from what input artifacts and how this artifact is further

  • used by other components which takes this one's input

  • and what outputs are generated from the downstream components.

  • OK this is all of the demo.

  • Now I'm going to talk about another important development

  • in TFX, which is supporting NativeKeras.

  • For those of you who are not very familiar with TensorFlow

  • 2, let me capture a little of the history.

  • TensorFlow 2 was released in Q3 2019

  • with a focus on providing a more Pythonic experience.

  • That includes supporting the Keras

  • API, eager execution by default, and the Pythonic execution.

  • So this is a timeline of how TFX Open Source has been working

  • on supporting all of this.

  • We released the first version with TFX Open Source

  • in the last Dev Summit, which only

  • supports estimator-based TensorFlow training code.

  • Back to last TensorFlow World, TensorFlow 2.0 was launched

  • and we started working on supporting the Keras API.

  • Previous slide, please?

  • Previous slide, please?

  • Back to Q4 2019.

  • We released the basic TensorFlow 2.0 support.

  • In TFX 0.20-- in that version, we supported TensorFlow 2.0

  • package end to end with a limited Keras support with

  • Keras Estimator.

  • And now, in the latest release TFX, I'm happy to release--

  • we are releasing experimental support of NativeKeras training

  • end to end.

  • So what does that mean?

  • Let's take a deeper look.

  • For data ingestion analysis, everything pretty much remains

  • the same, because TFTP, our data analysis library,

  • is model agnostic.

  • For future transform, we added a new Keras compatible layer

  • in TFT library so that we can transform features

  • in Keras model.

  • This layer will also take care of asset management

  • and the model exporting.

  • For training, we created a new generic trainer executor

  • which can be used to run any TensorFlow training code which

  • explores a saved model.

  • This also covers the training using NativeKeras API.

  • For model analysis and validation,

  • we create a new evaluator component,

  • which combines both evaluation and the model validation

  • capabilities.

  • This new component supports NativeKeras auto blocks.

  • And finally, when it gets to model serving validation,

  • we will release a new component called infra validator.

  • This component can be used to verify inference request, where

  • TensorFlow is serving binaries, to making

  • sure any exported TensorFlow model can be used correctly

  • in production, including anything

  • traded with NativeKeras.

  • Now, let's take a look at a case from one

  • of our partners, Concur Labs.

  • Concur Labs is a team in SAP Concur that explores new ideas

  • and are building prototypes.

  • They help all the developers in SAP Concur using ML effectively

  • in their solutions.

  • To do this, they need a modern machine learning pipeline

  • to secure scales that are facing their data platform.

  • They find that TFX with NativeKeras trainer

  • allows them to do more things.

  • One of the successful story here is the efficient BERT

  • deployment.

  • With the TF and the Keras, they can create simple models

  • by using TF Transform for the data pre-processing,

  • using state of the art models from TensorFlow hub

  • and create simple model deployments

  • with TensorFlow Serving.

  • The applications here covers some

  • of the sentimental analysis and some of the question and answer

  • type problems.

  • They also create a TFX pipelines to build

  • the pre-processing steps in export data BERT models.

  • We have just published a blog post on this,

  • so feel free to check out the TFX blog post.

  • Another successful story is TFX Pipelines for TFLite Models.

  • This means people-- they can create a TFX pipelines that

  • produced two models--

  • one in saved model format and the one

  • in TensorFlow Lite version.

  • This simplifies our pipeline building process

  • and it reduces manual conversion steps.

  • One of the most important things for the future of our ecosystem

  • is great partners, like all of you on the livestream.

  • We hope you join us to help making TFX work for your use

  • case.

  • Some of the areas we would love your help,

  • including portability, on-prem and the multi-cloud,

  • Spark/Flink/HDFS integration, as well as

  • data and the model governance.

  • One of the great things about TFX

  • is the wide diversity of the ways It can be used.

  • We have a exciting guest from--

  • Marcel from Airbus.

  • He could not be here physically today,

  • but he recorded a video to talk about one

  • of the interesting ways to use TFX which is TFX in space.

  • For more, here is Marcel.

  • [MUSIC PLAYING]

  • MARCEL RUMMENS: This is one of the most important moments

  • for everyone involved in manned spaceflight.

  • This is why we at Airbus are working hard

  • and keep on innovating to ensure everyone on that space

  • station is safe and can return back to Earth,

  • to friends and family.

  • Hello everyone, my name is Marcel Rummens,

  • and I have the honor to tell you how Airbus uses anomaly

  • detection with TFX to ensure everyone's safety onboard

  • the International Space Station.

  • You might ask yourself, Airbus, space--

  • how does that fit?

  • Well, Airbus actually has many different products,

  • like commercial aircrafts, helicopters, satellites,

  • and we are also involved in manned spaceflight.

  • For example, the Columbus Module,

  • which is part of the ISS.

  • It was built and designed by Airbus

  • and finally launched in 2008.

  • It is used for experiments in space--

  • for example, in the field of biology,

  • chemistry, material science, or medicine.

  • As you kind of mentioned, such a module produces a lot of data.

  • To give you an idea, we have recorded

  • between 15,000 and 20,000 parameters per second

  • for the last 10 years.

  • And every second, we receive another 17,000 parameters.

  • So we are talking about trillions and trillions

  • of data points.

  • But what does this data actually represent?

  • Well, it could be any kind of sensor data,

  • and we want to detect anomalies in it to prevent accidents.

  • Let me give you an example.

  • If you have a power surge in your electrical system,

  • this could cause a fire.

  • If you have a drop in temperature or pressure,

  • this could mean that you have a hole in your module

  • that you need to fix.

  • So it is very important that we detect these anomalies

  • and fix them before something can happen.

  • Because just imagine for a second

  • it is you up there, 400 kilometers above earth,

  • and you only have these few inches of metal and plastic

  • to protect you from space.

  • It's a cold and dark place without any oxygen.

  • If something would have happened to this little protection

  • layer of yours, you would be in a life-threatening situation.

  • This is why we already have countless autonomous systems

  • on board the spacecraft to prevent

  • these kinds of accidents.

  • But with more and more automation,

  • we can handle more complex data streams,

  • increasing our precision offset predictions.

  • But it is not about safety alone.

  • It is also about plannability and predictive maintenance,

  • because the sooner we know that a certain part needs

  • replacement, the sooner we can schedule and plan

  • a supply mission, decreasing the cost of said launches.

  • How does this work?

  • Well, right now this is a manual process.

  • Our automation works in parallel with the engineers.

  • Let me run you through it.

  • We have our database on premise, storing all these trillions

  • and trillions of data points.

  • And then we use a Spark cluster to extract the data

  • and remove secret parts from it, because some things we are not

  • allowed to upload.

  • Then we use TFX on KubeFlow to train the model.

  • First, we use TF Transform to prepare the data,

  • and then we use TF Estimator to train

  • a model which tries to represent the normal state

  • of a subsystem--

  • so a state without any anomalies.

  • Once we've done enough hyperparameter tuning,

  • we are happy with the model we deployed using TF Serving.

  • Now, here comes the interesting part.

  • We have the ISS streaming data to ground stations

  • on earth, which stream data to our NIfTI cluster

  • running in our data center.

  • Here we remove the secret part of the data

  • again and then stream it into the cloud.

  • In the cloud, we have a custom Python application

  • running on Kubernetes which does the actual anomaly detection.

  • It will create a model.

  • The model will try to predict the current state

  • of a subsystem based in whatever it has seen in the past.

  • And then using its prediction and the reality

  • coming from the space station, we

  • can calculate a so-called representation arrow.

  • If this arrow is above a certain threshold,

  • we can use this as an indicator for an anomaly.

  • Now, if we have an anomaly, we would create a report

  • and then compare it against a database of previously happened

  • anomalies, because if something like this happened in the past,

  • we can reuse the information we have on it.

  • And then the final step is to hand this over

  • to an engineer, who will then fix the problem.

  • And this is very, very important to us,

  • because we are talking about human life on that space

  • station, so we want a human to make the final decision

  • in this process.

  • But this is TF Dev Summit, so I want

  • to have at least one slide about our model architecture.

  • We are using LSTM-based autoencoder with dropout,

  • and then we replaced the inner layers--

  • so the layers between encoder and decoder--

  • with LSTMs instead of dense layers,

  • because our tests have shown that sequences just better

  • represent the kind of information

  • we have, producing less false positives.

  • What kind of impact did this project have?

  • Well, we have been able to reduce our cost by about 44%.

  • Some of this is projected because, as I said earlier,

  • we are running in parallel right now.

  • But the cost/benefit mainly comes from the fact

  • that our engineers can dedicate more and more time on more

  • important, less repetitive tasks, tasks for which you

  • really need that human creativity and intuition

  • to fix these problems.

  • Also, our response time has decreased.

  • In the past, it could take hours, days, or sometimes even

  • weeks to find and fix a problem.

  • Now, we are talking about minutes, maybe hours.

  • Another benefit is that now we have a central storage

  • of all anomalies that ever happened,

  • plus how they have been fixed.

  • This is not just good for our customer, because we

  • have better documentation, but also great for us

  • because, for example, it simplifies the onboarding

  • process for new colleagues.

  • Next steps, looking into the future.

  • We want to extend the solution to more subsystems and more

  • products, like Bartolomeo, which is the latest

  • addition to the Columbus Module and scheduled to be launched

  • later this month.

  • But overall, these kinds of technologies

  • are very important for future space missions,

  • because as plans to go to moon and Mars become more and more

  • concrete, we need new ways and new technologies

  • to tackle problems like latency and the limited amount

  • of computational hardware onboard the spacecraft.

  • Coming back to TFX, we want to integrate more components--

  • for example, the model validator--

  • because now we have more labeled data

  • that allows us to actually do this automatic model

  • validation.

  • And finally, the migration to TF 2,

  • which is ongoing but very important to us, because

  • of course, we want to keep up and use the latest

  • version of TensorFlow.

  • If you have any questions or want

  • to learn more about the challenges

  • we faced during that project, have a look at the Google blog.

  • We will publish a blog post in the coming weeks

  • that goes into more detail than I could do in 10 minutes.

  • Before I close, I want to especially thank

  • [? Philip ?] [INAUDIBLE],, Jonas Hansen,

  • as well as everyone else of the ISS analytics

  • team for their incredible work and support,

  • as well as anyone else who helped to prepare this talk.

  • If you have any questions, feel free to find me

  • on the internet.

  • Write me those questions--

  • I'm happy to answer them.

  • And also, I will be available in the Q&A

  • section of this livestream.

  • Thank you very much and goodbye.

  • ZHITAO LI: Thank you, Marcel, for the great video.

  • And if you want to learn more about TFX,

  • please check out our website, which

  • has all the blog post, user guide tutorials, and API docs.

  • Please also feel free to check out our GitHub repo

  • with all the source code and engage about us

  • on the Google Group.

  • [MUSIC PLAYING]

[MUSIC PLAYING]

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it