Subtitles section Play video Print subtitles FOR THINGS LIKE CALCULATING THE FOR THINGS LIKE CALCULATING THE FOR THINGS LIKE CALCULATING THE MEDIAN FOR STANDARD DEVIATION MEDIAN FOR STANDARD DEVIATION MEDIAN FOR STANDARD DEVIATION OF A FEATURE, NUMBERS THAT ARE OF A FEATURE, NUMBERS THAT ARE OF A FEATURE, NUMBERS THAT ARE THE SAME FOR ALL EXAMPLES, THE SAME FOR ALL EXAMPLES, THE SAME FOR ALL EXAMPLES, TRANSFORM WILL OUTPUT A TRANSFORM WILL OUTPUT A TRANSFORM WILL OUTPUT A CONSTANT. CONSTANT. CONSTANT. FOR THINGS LIKE NORMALIZING A FOR THINGS LIKE NORMALIZING A FOR THINGS LIKE NORMALIZING A VALUE, VALUES WHICH WILL BE VALUE, VALUES WHICH WILL BE VALUE, VALUES WHICH WILL BE DIFFERENT FOR DIFFERENT DIFFERENT FOR DIFFERENT DIFFERENT FOR DIFFERENT EXAMPLES, TRANSFORM WILL OUTPUT EXAMPLES, TRANSFORM WILL OUTPUT EXAMPLES, TRANSFORM WILL OUTPUT TENSORFLOW OPS, IT WILL THEN TENSORFLOW OPS, IT WILL THEN TENSORFLOW OPS, IT WILL THEN PUT AN OUTFLOW GRAPH WITH THE PUT AN OUTFLOW GRAPH WITH THE PUT AN OUTFLOW GRAPH WITH THE CONSTANTS AND OPS. CONSTANTS AND OPS. CONSTANTS AND OPS. THAT IS HER MEDIC. THAT IS HER MEDIC. THAT IS HER MEDIC. IT CONTAINS ALL THE INFORMATION IT CONTAINS ALL THE INFORMATION IT CONTAINS ALL THE INFORMATION YOU NEED TO APPLY THOSE YOU NEED TO APPLY THOSE YOU NEED TO APPLY THOSE TRANSFORMATIONS AND FORM THE TRANSFORMATIONS AND FORM THE TRANSFORMATIONS AND FORM THE INPUT STAGE FOR YOUR MODEL. INPUT STAGE FOR YOUR MODEL. INPUT STAGE FOR YOUR MODEL. THAT MEANS THAT THE SAME THAT MEANS THAT THE SAME THAT MEANS THAT THE SAME TRANSFORMATIONS ARE APPLIED TRANSFORMATIONS ARE APPLIED TRANSFORMATIONS ARE APPLIED CONSISTENTLY BETWEEN TRAINING CONSISTENTLY BETWEEN TRAINING CONSISTENTLY BETWEEN TRAINING AND SERVING, WHICH ELIMINATES AND SERVING, WHICH ELIMINATES AND SERVING, WHICH ELIMINATES TRAINING/SERVING SKEW. TRAINING/SERVING SKEW. TRAINING/SERVING SKEW. IF INSTEAD YOU ARE MOVING YOUR IF INSTEAD YOU ARE MOVING YOUR IF INSTEAD YOU ARE MOVING YOUR MODEL FROM A TRAINING MODEL FROM A TRAINING MODEL FROM A TRAINING ENVIRONMENT INTO A SERVING ENVIRONMENT INTO A SERVING ENVIRONMENT INTO A SERVING ENVIRONMENT OR APPLICATION AND ENVIRONMENT OR APPLICATION AND ENVIRONMENT OR APPLICATION AND TRYING TO APPLY THE SAME TRYING TO APPLY THE SAME TRYING TO APPLY THE SAME FEATURE ENGINEERING IN BOTH FEATURE ENGINEERING IN BOTH FEATURE ENGINEERING IN BOTH PLACES, YOU HOPE THAT THE PLACES, YOU HOPE THAT THE PLACES, YOU HOPE THAT THE TRANSFORMATIONS ARE THE SAME TRANSFORMATIONS ARE THE SAME TRANSFORMATIONS ARE THE SAME BUT SOMETIMES YOU FIND THAT BUT SOMETIMES YOU FIND THAT BUT SOMETIMES YOU FIND THAT THEY'RE NOT. THEY'RE NOT. THEY'RE NOT. WE CALL THAT TRAINING SERVING WE CALL THAT TRAINING SERVING WE CALL THAT TRAINING SERVING SKEW AND TRANSFORM ELIMINATES SKEW AND TRANSFORM ELIMINATES SKEW AND TRANSFORM ELIMINATES IT BY USING EXACTLY THE SAME IT BY USING EXACTLY THE SAME IT BY USING EXACTLY THE SAME CODE ANYWHERE YOU RUN YOUR CODE ANYWHERE YOU RUN YOUR CODE ANYWHERE YOU RUN YOUR MODEL. MODEL. MODEL. NOW WE'RE FINALLY READY TO NOW WE'RE FINALLY READY TO NOW WE'RE FINALLY READY TO TRAIN OUR MODEL. TRAIN OUR MODEL. TRAIN OUR MODEL. THE PART OF THE PROCESS THAT THE PART OF THE PROCESS THAT THE PART OF THE PROCESS THAT YOU OFTEN THINK ABOUT WHEN YOU YOU OFTEN THINK ABOUT WHEN YOU YOU OFTEN THINK ABOUT WHEN YOU THINK ABOUT MACHINE LEARNING. THINK ABOUT MACHINE LEARNING. THINK ABOUT MACHINE LEARNING. TRAINER TAKES IN THE TRANSFORM TRAINER TAKES IN THE TRANSFORM TRAINER TAKES IN THE TRANSFORM GRAPH AND DATA FROM TRANSFORM GRAPH AND DATA FROM TRANSFORM GRAPH AND DATA FROM TRANSFORM AND ESCHEMA FROM SCHEMA GEN AND AND ESCHEMA FROM SCHEMA GEN AND AND ESCHEMA FROM SCHEMA GEN AND TRAINS THE MODEL USING YOUR TRAINS THE MODEL USING YOUR TRAINS THE MODEL USING YOUR MODELLING CODE. MODELLING CODE. MODELLING CODE. NORMAL MODEL TRAINING. NORMAL MODEL TRAINING. NORMAL MODEL TRAINING. BUT WHEN TRAINING IS COMPLETE, BUT WHEN TRAINING IS COMPLETE, BUT WHEN TRAINING IS COMPLETE, TRAINER WILL SAVE TWO DIFFERENT TRAINER WILL SAVE TWO DIFFERENT TRAINER WILL SAVE TWO DIFFERENT SAVED MODELS. SAVED MODELS. SAVED MODELS. ONE IS A NORMAL SAVE MODEL THAT ONE IS A NORMAL SAVE MODEL THAT ONE IS A NORMAL SAVE MODEL THAT WILL BE DEPLOYED TO PRODUCTION. WILL BE DEPLOYED TO PRODUCTION. WILL BE DEPLOYED TO PRODUCTION. AND THE OTHER IS AN EVAL SAVE AND THE OTHER IS AN EVAL SAVE AND THE OTHER IS AN EVAL SAVE MODEL THAT WILL BE USED FOR MODEL THAT WILL BE USED FOR MODEL THAT WILL BE USED FOR ANALYZING THE PERFORMANCE OF ANALYZING THE PERFORMANCE OF ANALYZING THE PERFORMANCE OF YOUR MODEL. YOUR MODEL. YOUR MODEL. THE CONFIGURATION FOR TRAINER THE CONFIGURATION FOR TRAINER THE CONFIGURATION FOR TRAINER IS WHAT YOU WOULD EXPECT. IS WHAT YOU WOULD EXPECT. IS WHAT YOU WOULD EXPECT. THINGS LIKE THE NUMBER OF STEPS THINGS LIKE THE NUMBER OF STEPS THINGS LIKE THE NUMBER OF STEPS AND WHETHER OR NOT TO USE WARM AND WHETHER OR NOT TO USE WARM AND WHETHER OR NOT TO USE WARM STARTING. STARTING. STARTING. THE CODE THAT YOU CREATE FOR THE CODE THAT YOU CREATE FOR THE CODE THAT YOU CREATE FOR TRAINER IS YOUR MODELING CODE. TRAINER IS YOUR MODELING CODE. TRAINER IS YOUR MODELING CODE. SO IT CAN BE AS SIMPLE OR SO IT CAN BE AS SIMPLE OR SO IT CAN BE AS SIMPLE OR COMPLEX AS YOU NEED IT TO BE. COMPLEX AS YOU NEED IT TO BE. COMPLEX AS YOU NEED IT TO BE. TO MONITOR AND ANALYZE THE TO MONITOR AND ANALYZE THE TO MONITOR AND ANALYZE THE TRAINING PROCESS YOU CAN USE TRAINING PROCESS YOU CAN USE TRAINING PROCESS YOU CAN USE TENSOR BOARD JUST LIKE YOU TENSOR BOARD JUST LIKE YOU TENSOR BOARD JUST LIKE YOU WOULD NORMALLY. WOULD NORMALLY. WOULD NORMALLY. IN THIS CASE, YOU CAN LOOK AT IN THIS CASE, YOU CAN LOOK AT IN THIS CASE, YOU CAN LOOK AT THE CURRENT MODEL TRAINING RUN THE CURRENT MODEL TRAINING RUN THE CURRENT MODEL TRAINING RUN OR COMPARE THE RESULTS FROM OR COMPARE THE RESULTS FROM OR COMPARE THE RESULTS FROM MULTIPLE MODEL TRAINING RUNS. MULTIPLE MODEL TRAINING RUNS. MULTIPLE MODEL TRAINING RUNS. THIS IS ONLY POSSIBLE BECAUSE THIS IS ONLY POSSIBLE BECAUSE THIS IS ONLY POSSIBLE BECAUSE OF THE ML METADATA STORE THAT OF THE ML METADATA STORE THAT OF THE ML METADATA STORE THAT WE TALKED ABOUT IN OUR LAST WE TALKED ABOUT IN OUR LAST WE TALKED ABOUT IN OUR LAST EPISODE. EPISODE. EPISODE. TFX MAKES IT FAIRLY EASY TO DO TFX MAKES IT FAIRLY EASY TO DO TFX MAKES IT FAIRLY EASY TO DO THIS KIND OF COMPARISON WHICH THIS KIND OF COMPARISON WHICH THIS KIND OF COMPARISON WHICH IS OFTEN REVEALING. IS OFTEN REVEALING. IS OFTEN REVEALING. NOW THAT WE'VE TRAINED OUR NOW THAT WE'VE TRAINED OUR NOW THAT WE'VE TRAINED OUR MODEL, HOW DO THE RESULTS LOOK? MODEL, HOW DO THE RESULTS LOOK? MODEL, HOW DO THE RESULTS LOOK? THE EVALUATOR COMPONENT WILL THE EVALUATOR COMPONENT WILL THE EVALUATOR COMPONENT WILL TAKE THE MODEL TRAINER CREATED TAKE THE MODEL TRAINER CREATED TAKE THE MODEL TRAINER CREATED AND USE DEEP ANALYSIS USING AND USE DEEP ANALYSIS USING AND USE DEEP ANALYSIS USING BEAM AND THE MODEL ANALYSIS BEAM AND THE MODEL ANALYSIS BEAM AND THE MODEL ANALYSIS LIBRARY. LIBRARY. LIBRARY. NOT JUST LOOKING AT THE TOP NOT JUST LOOKING AT THE TOP NOT JUST LOOKING AT THE TOP LEVEL RESULTS ACROSS THE WHOLE LEVEL RESULTS ACROSS THE WHOLE LEVEL RESULTS ACROSS THE WHOLE DATASET, IT IS LOOKING DEEPER DATASET, IT IS LOOKING DEEPER DATASET, IT IS LOOKING DEEPER THAN THAT. THAN THAT. THAN THAT. AT INDIVIDUAL SLICES OF OUR AT INDIVIDUAL SLICES OF OUR AT INDIVIDUAL SLICES OF OUR DATASET. DATASET. DATASET. THAT'S IMPORTANT BECAUSE THE THAT'S IMPORTANT BECAUSE THE THAT'S IMPORTANT BECAUSE THE EXPERIENCE THAT EACH USER OF EXPERIENCE THAT EACH USER OF EXPERIENCE THAT EACH USER OF OUR MODEL HAS WILL DEPEND ON OUR MODEL HAS WILL DEPEND ON OUR MODEL HAS WILL DEPEND ON THEIR INDIVIDUAL DATA POINT. THEIR INDIVIDUAL DATA POINT. THEIR INDIVIDUAL DATA POINT. OUR MODEL MAY DO WELL OVER OUR OUR MODEL MAY DO WELL OVER OUR OUR MODEL MAY DO WELL OVER OUR ENTIRE DATASET, BUT IF IT DOES ENTIRE DATASET, BUT IF IT DOES ENTIRE DATASET, BUT IF IT DOES POORLY ON A DATA POINT THAT A POORLY ON A DATA POINT THAT A POORLY ON A DATA POINT THAT A USER GIVES IT. USER GIVES IT. USER GIVES IT. THAT USER'S EXPERIENCE IS POOR. THAT USER'S EXPERIENCE IS POOR. THAT USER'S EXPERIENCE IS POOR. WE'LL TALK ABOUT THIS MORE IN WE'LL TALK ABOUT THIS MORE IN WE'LL TALK ABOUT THIS MORE IN OUR NEXT EPISODE. OUR NEXT EPISODE. OUR NEXT EPISODE. SO NOW THAT WE'VE LOOKED AT OUR SO NOW THAT WE'VE LOOKED AT OUR SO NOW THAT WE'VE LOOKED AT OUR MODEL'S PERFORMANCE, SHOULD WE MODEL'S PERFORMANCE, SHOULD WE MODEL'S PERFORMANCE, SHOULD WE PUSH IT TO PRODUCTION? PUSH IT TO PRODUCTION? PUSH IT TO PRODUCTION? IS IT BETTER OR WORSE THAN WHAT IS IT BETTER OR WORSE THAN WHAT IS IT BETTER OR WORSE THAN WHAT WE ALREADY HAVE IN PRODUCTION? WE ALREADY HAVE IN PRODUCTION? WE ALREADY HAVE IN PRODUCTION? WE DON'T WANT TO PUSH A WORSE WE DON'T WANT TO PUSH A WORSE WE DON'T WANT TO PUSH A WORSE MODEL JUST BECAUSE IT'S NEW. MODEL JUST BECAUSE IT'S NEW. MODEL JUST BECAUSE IT'S NEW. SO THE MODEL VALIDATOR SO THE MODEL VALIDATOR SO THE MODEL VALIDATOR COMPONENT USES BEAM TO DO THAT COMPONENT USES BEAM TO DO THAT COMPONENT USES BEAM TO DO THAT COMPARISON USING CRITERIA THAT COMPARISON USING CRITERIA THAT COMPARISON USING CRITERIA THAT WE DEFINE TO DECIDE WHETHER OR WE DEFINE TO DECIDE WHETHER OR WE DEFINE TO DECIDE WHETHER OR NOT TO PUSH THE NEW MODEL TO NOT TO PUSH THE NEW MODEL TO NOT TO PUSH THE NEW MODEL TO PRODUCTION. PRODUCTION. PRODUCTION. IF MODEL VALIDATOR DECIDES THAT IF MODEL VALIDATOR DECIDES THAT IF MODEL VALIDATOR DECIDES THAT OUR NEW MODEL IS READY FOR OUR NEW MODEL IS READY FOR OUR NEW MODEL IS READY FOR PRODUCTION, THEN PUSHER DOES PRODUCTION, THEN PUSHER DOES PRODUCTION, THEN PUSHER DOES THE WORK OF ACTUALLY PUSHING IT THE WORK OF ACTUALLY PUSHING IT THE WORK OF ACTUALLY PUSHING IT TO OUR DEPLOYMENT TARGETS. TO OUR DEPLOYMENT TARGETS. TO OUR DEPLOYMENT TARGETS. THOSE TARGETS COULD BE THOSE TARGETS COULD BE THOSE TARGETS COULD BE TENSORFLOW LIGHT IN WE'RE DOING TENSORFLOW LIGHT IN WE'RE DOING TENSORFLOW LIGHT IN WE'RE DOING A MOBILE APPLICATION OR A MOBILE APPLICATION OR A MOBILE APPLICATION OR TENSORFLOW JS IF WE'RE TENSORFLOW JS IF WE'RE TENSORFLOW JS IF WE'RE DEPLOYING TO A JAVASCRIPT DEPLOYING TO A JAVASCRIPT DEPLOYING TO A JAVASCRIPT ENVIRONMENT OR TENSORFLOW ENVIRONMENT OR TENSORFLOW ENVIRONMENT OR TENSORFLOW SERVING IF WE ARE DEPLOYING TO SERVING IF WE ARE DEPLOYING TO SERVING IF WE ARE DEPLOYING TO A SERVER FARM OR ALL OF THE A SERVER FARM OR ALL OF THE A SERVER FARM OR ALL OF THE ABOVE. ABOVE. ABOVE. SO NOW HOPEFULLY WE'VE GIVEN SO NOW HOPEFULLY WE'VE GIVEN SO NOW HOPEFULLY WE'VE GIVEN YOU A BASIC UNDERSTANDING OF YOU A BASIC UNDERSTANDING OF YOU A BASIC UNDERSTANDING OF TFX AND ML PIPELINES IN GENERAL TFX AND ML PIPELINES IN GENERAL TFX AND ML PIPELINES IN GENERAL AND SOME OF THE ISSUES AROUND AND SOME OF THE ISSUES AROUND AND SOME OF THE ISSUES AROUND PUTTING AN ML APPLICATION INTO PUTTING AN ML APPLICATION INTO PUTTING AN ML APPLICATION INTO PRODUCTION. PRODUCTION. PRODUCTION. AND REMEMBER, TFX IS OPEN AND REMEMBER, TFX IS OPEN AND REMEMBER, TFX IS OPEN SOURCE. SOURCE. SOURCE. WE WANT YOU TO HELP CONTRIBUTE WE WANT YOU TO HELP CONTRIBUTE WE WANT YOU TO HELP CONTRIBUTE TO MAKING TFX BETTER. TO MAKING TFX BETTER. TO MAKING TFX BETTER. IN OUR NEXT EPISODE, WE'LL LOOK IN OUR NEXT EPISODE, WE'LL LOOK IN OUR NEXT EPISODE, WE'LL LOOK AT A REAL WORLD EXAMPLE OF WHY AT A REAL WORLD EXAMPLE OF WHY AT A REAL WORLD EXAMPLE OF WHY ANALYZING MODEL PERFORMANCE IS ANALYZING MODEL PERFORMANCE IS ANALYZING MODEL PERFORMANCE IS IMPORTANT TO YOUR BUSINESS. IMPORTANT TO YOUR BUSINESS. IMPORTANT TO YOUR BUSINESS. FOR MORE INFORMATION ON TFX, FOR MORE INFORMATION ON TFX, FOR MORE INFORMATION ON TFX, VISIT US AS TENSORFLOW.ORG VISIT US AS TENSORFLOW.ORG VISIT US AS TENSORFLOW.ORG SLASH TFX AND DON'T FORGET TO SLASH TFX AND DON'T FORGET TO SLASH TFX AND DON'T FORGET TO COMMENT AND LIKE US BELOW AND COMMENT AND LIKE US BELOW AND COMMENT AND LIKE US BELOW AND THANKS FOR WATCHING. >> GOOD AFTERNOON. >> GOOD AFTERNOON. >> GOOD AFTERNOON. I WOULD LIKE TO WELCOME YOU TO I WOULD LIKE TO WELCOME YOU TO I WOULD LIKE TO WELCOME YOU TO TENSORFLOW MODEL OPTIMIZATION, TENSORFLOW MODEL OPTIMIZATION, TENSORFLOW MODEL OPTIMIZATION, QUAUNTIZATION AND PRUNING. QUAUNTIZATION AND PRUNING. QUAUNTIZATION AND PRUNING. A TALK GIVEN MY MY COLLEAGUE AT A TALK GIVEN MY MY COLLEAGUE AT A TALK GIVEN MY MY COLLEAGUE AT OOG WILL, RAZIEL ALVEREZ. OOG WILL, RAZIEL ALVEREZ. OOG WILL, RAZIEL ALVEREZ. >> HI. >> HI. >> HI. MY NAME IS RAZIEL. MY NAME IS RAZIEL. MY NAME IS RAZIEL. I AM TALKING ABOUT OPTIMIZATION I AM TALKING ABOUT OPTIMIZATION I AM TALKING ABOUT OPTIMIZATION AND TALK IN PARTICULAR AROUND AND TALK IN PARTICULAR AROUND AND TALK IN PARTICULAR AROUND TECHNIQUES AROUND QUANTIZATION TECHNIQUES AROUND QUANTIZATION TECHNIQUES AROUND QUANTIZATION AND PRUNING. AND PRUNING. AND PRUNING. SO FIRST I WILL INTRODUCE WHY SO FIRST I WILL INTRODUCE WHY SO FIRST I WILL INTRODUCE WHY WHAT IS MORE OPTIMIZATION AND WHAT IS MORE OPTIMIZATION AND WHAT IS MORE OPTIMIZATION AND PRUNING AND WHY WE THINK IT'S PRUNING AND WHY WE THINK IT'S PRUNING AND WHY WE THINK IT'S IMPORTANT WHY WE'RE INVESTING IMPORTANT WHY WE'RE INVESTING IMPORTANT WHY WE'RE INVESTING IN THIS AREA. IN THIS AREA. IN THIS AREA. I WILL COVER THE TOOLS THAT WE I WILL COVER THE TOOLS THAT WE I WILL COVER THE TOOLS THAT WE HAVE AVAILABLE AND AT THE END I HAVE AVAILABLE AND AT THE END I HAVE AVAILABLE AND AT THE END I WILL GIVE AN OVERVIEW ON OUR WILL GIVE AN OVERVIEW ON OUR WILL GIVE AN OVERVIEW ON OUR ROADMAP FOR THE SHORT TERM AND ROADMAP FOR THE SHORT TERM AND ROADMAP FOR THE SHORT TERM AND THE LONGER TERM. THE LONGER TERM. THE LONGER TERM. AND AT THE END OF THE AND AT THE END OF THE AND AT THE END OF THE PRESENTATION WE HAVE SOME PRESENTATION WE HAVE SOME PRESENTATION WE HAVE SOME MINUTES TO GO OVER Q & A. MINUTES TO GO OVER Q & A. MINUTES TO GO OVER Q & A. SO IT SHOULD ALLOW YOU TO SO IT SHOULD ALLOW YOU TO SO IT SHOULD ALLOW YOU TO OPTIMIZE MACHINE LEARNING MODEL OPTIMIZE MACHINE LEARNING MODEL OPTIMIZE MACHINE LEARNING MODEL FOR DEPLOYMENT AND EXECUTION. FOR DEPLOYMENT AND EXECUTION. FOR DEPLOYMENT AND EXECUTION. WE THINK THIS IS IMPORTANT WE THINK THIS IS IMPORTANT WE THINK THIS IS IMPORTANT BECAUSE MACHINE LEARNING IS A BECAUSE MACHINE LEARNING IS A BECAUSE MACHINE LEARNING IS A VERY IMPORTANT FIELD AND WE VERY IMPORTANT FIELD AND WE VERY IMPORTANT FIELD AND WE THINK THAT THERE IS A LOT OF THINK THAT THERE IS A LOT OF THINK THAT THERE IS A LOT OF ROOM TO MAKE IT MORE EFFICIENT ROOM TO MAKE IT MORE EFFICIENT ROOM TO MAKE IT MORE EFFICIENT AND THIS HAS SOME IMPLICATIONS AND THIS HAS SOME IMPLICATIONS AND THIS HAS SOME IMPLICATIONS BOTH ECONOMY. BOTH ECONOMY. BOTH ECONOMY. YOU CAN MAKE ALL THESE YOU CAN MAKE ALL THESE YOU CAN MAKE ALL THESE APPLICATIONS MUCH BETTER, RIGHT? APPLICATIONS MUCH BETTER, RIGHT? APPLICATIONS MUCH BETTER, RIGHT? LIKE THE QUALITY OR CHEAPER TO LIKE THE QUALITY OR CHEAPER TO LIKE THE QUALITY OR CHEAPER TO EXECUTE. EXECUTE. EXECUTE. WE CAN ENABLE SOME NEW MODELS WE CAN ENABLE SOME NEW MODELS WE CAN ENABLE SOME NEW MODELS AND NEW DEPLOYMENT AND NEW AND NEW DEPLOYMENT AND NEW AND NEW DEPLOYMENT AND NEW PROTOCOL EVEN IF YOU JUST TRIED PROTOCOL EVEN IF YOU JUST TRIED PROTOCOL EVEN IF YOU JUST TRIED TO DO -- EXECUTE THIS MUCH TO DO -- EXECUTE THIS MUCH TO DO -- EXECUTE THIS MUCH LEARNING MODE ON THE SERVER. LEARNING MODE ON THE SERVER. LEARNING MODE ON THE SERVER. SO CURRENTLY MACHINE LEARNING SO CURRENTLY MACHINE LEARNING SO CURRENTLY MACHINE LEARNING RUNS EITHER ON THE SERVER OR ON RUNS EITHER ON THE SERVER OR ON RUNS EITHER ON THE SERVER OR ON THE EDGE. THE EDGE. THE EDGE. ON THE SERVER YOU MAY THINK ON THE SERVER YOU MAY THINK ON THE SERVER YOU MAY THINK THERE IS A LOT OF CAPACITY AND THERE IS A LOT OF CAPACITY AND THERE IS A LOT OF CAPACITY AND A LOT OF COMPUTER MEMORY. A LOT OF COMPUTER MEMORY. A LOT OF COMPUTER MEMORY. WHAT IS THE BENEFIT OF THESE WHAT IS THE BENEFIT OF THESE WHAT IS THE BENEFIT OF THESE MODELS? MODELS? MODELS? APPLICATIONS ARE STILL BOUND BY APPLICATIONS ARE STILL BOUND BY APPLICATIONS ARE STILL BOUND BY LATENCY. LATENCY. LATENCY. STILL A VERY IMPORTANT METRIC STILL A VERY IMPORTANT METRIC STILL A VERY IMPORTANT METRIC FOR A LOT OF APPLICATIONS OR FOR A LOT OF APPLICATIONS OR FOR A LOT OF APPLICATIONS OR YOU WANT TO IMPROVE HOW MANY YOU WANT TO IMPROVE HOW MANY YOU WANT TO IMPROVE HOW MANY TASKS CAN RUN ON YOUR SERVER. TASKS CAN RUN ON YOUR SERVER. TASKS CAN RUN ON YOUR SERVER. AND THESE TWO ARE ALSO RELATED AND THESE TWO ARE ALSO RELATED AND THESE TWO ARE ALSO RELATED TO MONEY. TO MONEY. TO MONEY. EVERYBODY WILL WANT TO SAVE EVERYBODY WILL WANT TO SAVE EVERYBODY WILL WANT TO SAVE MONEY AND POTENTIALLY WE'RE MONEY AND POTENTIALLY WE'RE MONEY AND POTENTIALLY WE'RE TALKING A LOT OF MONEY. TALKING A LOT OF MONEY. TALKING A LOT OF MONEY. NOW ON THE EDGE IS A LITTLE NOW ON THE EDGE IS A LITTLE NOW ON THE EDGE IS A LITTLE MORE OBVIOUS WHEN YOU HAVE MORE OBVIOUS WHEN YOU HAVE MORE OBVIOUS WHEN YOU HAVE OPTIMIZATION. OPTIMIZATION. OPTIMIZATION. CONSTRAINED ENVIRONMENT AS YOU CONSTRAINED ENVIRONMENT AS YOU CONSTRAINED ENVIRONMENT AS YOU TALK ABOUT APPLICATIONS IN TALK ABOUT APPLICATIONS IN TALK ABOUT APPLICATIONS IN GENERAL. GENERAL. GENERAL. WE NEED TO DEAL WITH REDUCED WE NEED TO DEAL WITH REDUCED WE NEED TO DEAL WITH REDUCED MEMORY, POWER CON SUMS, MEMORY, POWER CON SUMS, MEMORY, POWER CON SUMS, BANDWIDTH BOTH DOWNLOADING BANDWIDTH BOTH DOWNLOADING BANDWIDTH BOTH DOWNLOADING MODELS FROM THE CLOUD OR EVEN MODELS FROM THE CLOUD OR EVEN MODELS FROM THE CLOUD OR EVEN WE'VE SEEN TO BE ABLE TO WE'VE SEEN TO BE ABLE TO WE'VE SEEN TO BE ABLE TO TRANSFER FROM MEMORY TO THE TRANSFER FROM MEMORY TO THE TRANSFER FROM MEMORY TO THE PROCESSOR. PROCESSOR. PROCESSOR. THIS COULD BE A PROBLEM IF THE THIS COULD BE A PROBLEM IF THE THIS COULD BE A PROBLEM IF THE MODEL IS TOO LARGE. MODEL IS TOO LARGE. MODEL IS TOO LARGE. PLUS WE HAVE A WIDE VARIETY OF PLUS WE HAVE A WIDE VARIETY OF PLUS WE HAVE A WIDE VARIETY OF HARDWARE. HARDWARE. HARDWARE. MORE THAN IN THE SERVER AND WE MORE THAN IN THE SERVER AND WE MORE THAN IN THE SERVER AND WE NEED TO MAKE SURE THAT THESE NEED TO MAKE SURE THAT THESE NEED TO MAKE SURE THAT THESE MODELS RUN EFFICIENTLY ON ALL MODELS RUN EFFICIENTLY ON ALL MODELS RUN EFFICIENTLY ON ALL THESE DIFFERENT TYPES OF THESE DIFFERENT TYPES OF THESE DIFFERENT TYPES OF HARDWARE. HARDWARE. HARDWARE. SO IT FOLLOWS IF WE ARE SO IT FOLLOWS IF WE ARE SO IT FOLLOWS IF WE ARE OPTIMIZING THIS MODEL WE CAN OPTIMIZING THIS MODEL WE CAN OPTIMIZING THIS MODEL WE CAN HAVE BETTER MODELS AND THEN HAVE BETTER MODELS AND THEN HAVE BETTER MODELS AND THEN EVENTUALLY STARTS TRANSLATING EVENTUALLY STARTS TRANSLATING EVENTUALLY STARTS TRANSLATING IN NEW PRODUCTS THAT OTHERWISE IN NEW PRODUCTS THAT OTHERWISE IN NEW PRODUCTS THAT OTHERWISE COULDN'T EXIST IF WE JUST WERE COULDN'T EXIST IF WE JUST WERE COULDN'T EXIST IF WE JUST WERE RUNNING THESE MODELS ON A RUNNING THESE MODELS ON A RUNNING THESE MODELS ON A SERVER. SERVER. SERVER. AND THESE OPPORTUNITIES ARE AND THESE OPPORTUNITIES ARE AND THESE OPPORTUNITIES ARE LARGER THAN JUST -- MACHINE LARGER THAN JUST -- MACHINE LARGER THAN JUST -- MACHINE LEARNING IS TRICKLING DOWN INTO LEARNING IS TRICKLING DOWN INTO LEARNING IS TRICKLING DOWN INTO MORE ENVIRONMENTS. MORE ENVIRONMENTS. MORE ENVIRONMENTS. WE HAVE MACHINE LEARNING MODELS WE HAVE MACHINE LEARNING MODELS WE HAVE MACHINE LEARNING MODELS THAT ARE USED TO DETECT THAT ARE USED TO DETECT THAT ARE USED TO DETECT FAILURES IN MACHINERY IN FAILURES IN MACHINERY IN FAILURES IN MACHINERY IN FACTORIES OR WE USE IT IN FACTORIES OR WE USE IT IN FACTORIES OR WE USE IT IN SELF-DRIVING CARS. SELF-DRIVING CARS. SELF-DRIVING CARS. WE USE IT IN THE OFFICE TO SCAN WE USE IT IN THE OFFICE TO SCAN WE USE IT IN THE OFFICE TO SCAN DOCUMENTS AND TRY TO UNDERSTAND DOCUMENTS AND TRY TO UNDERSTAND DOCUMENTS AND TRY TO UNDERSTAND THEM. THEM. THEM. AND JUST TO GIVE YOU SOME AND JUST TO GIVE YOU SOME AND JUST TO GIVE YOU SOME NUMBERS, THE SIZE OF THE CELL NUMBERS, THE SIZE OF THE CELL NUMBERS, THE SIZE OF THE CELL PHONE MARKET IS REALLY A PHONE MARKET IS REALLY A PHONE MARKET IS REALLY A FRACTION OF THE POTENTIAL FOR FRACTION OF THE POTENTIAL FOR FRACTION OF THE POTENTIAL FOR THE EDGE DEVICES IN GENERAL. THE EDGE DEVICES IN GENERAL. THE EDGE DEVICES IN GENERAL. SO BASICALLY THE TWO REASONS IS SO BASICALLY THE TWO REASONS IS SO BASICALLY THE TWO REASONS IS WE WANT TO MAKE MACHINE WE WANT TO MAKE MACHINE WE WANT TO MAKE MACHINE LEARNING MORE EFFICIENT. LEARNING MORE EFFICIENT. LEARNING MORE EFFICIENT. IT IS ALREADY VERY IMPORTANT IT IS ALREADY VERY IMPORTANT IT IS ALREADY VERY IMPORTANT FOR THE SERVERS. FOR THE SERVERS. FOR THE SERVERS. PRETTY CRUCIAL FOR EMBEDDED PRETTY CRUCIAL FOR EMBEDDED PRETTY CRUCIAL FOR EMBEDDED DEVICES. DEVICES. DEVICES. SO WE STARTED THIS TOOLKIT SO WE STARTED THIS TOOLKIT SO WE STARTED THIS TOOLKIT ABOUT A YEAR AGO. ABOUT A YEAR AGO. ABOUT A YEAR AGO. WE INITIALLY LAUNCHED TRAINING WE INITIALLY LAUNCHED TRAINING WE INITIALLY LAUNCHED TRAINING WITH THESE HYBRID AND NOW GOING WITH THESE HYBRID AND NOW GOING WITH THESE HYBRID AND NOW GOING INTO MORE DETAIL IN THE INTO MORE DETAIL IN THE INTO MORE DETAIL IN THE PRESENTATION. PRESENTATION. PRESENTATION. EARLIER THIS YEAR WE LAUNCHED EARLIER THIS YEAR WE LAUNCHED EARLIER THIS YEAR WE LAUNCHED API FOR NEURAL CONNECTION API FOR NEURAL CONNECTION API FOR NEURAL CONNECTION PRUNING AND WE CREATED THIS PRUNING AND WE CREATED THIS PRUNING AND WE CREATED THIS OPTIMIZATION FOR TENSORFLOW OPTIMIZATION FOR TENSORFLOW OPTIMIZATION FOR TENSORFLOW LIGHT AND WE LAUNCHED THOSE LIGHT AND WE LAUNCHED THOSE LIGHT AND WE LAUNCHED THOSE QUANTIZATION FOR TARGETING QUANTIZATION FOR TARGETING QUANTIZATION FOR TARGETING SPECICATION. SPECICATION. SPECICATION. MORE RECENTLY REDUCED FLOW MORE RECENTLY REDUCED FLOW MORE RECENTLY REDUCED FLOW PRECISION AND HOPEFULLY SOON PRECISION AND HOPEFULLY SOON PRECISION AND HOPEFULLY SOON WE'RE GOING TO BE LAUNCHING WE'RE GOING TO BE LAUNCHING WE'RE GOING TO BE LAUNCHING QUANTIZATION TRAINING API AND QUANTIZATION TRAINING API AND QUANTIZATION TRAINING API AND TO APPLY FOR SPARSE COMPUTATION. TO APPLY FOR SPARSE COMPUTATION. TO APPLY FOR SPARSE COMPUTATION. NOW WE GO INTO THESE TECHNIQUES NOW WE GO INTO THESE TECHNIQUES NOW WE GO INTO THESE TECHNIQUES AND TOOLS IN A LITTLE BIT MORE AND TOOLS IN A LITTLE BIT MORE AND TOOLS IN A LITTLE BIT MORE DETAIL. DETAIL. DETAIL. LET'S START WITH QUANTIZATION. LET'S START WITH QUANTIZATION. LET'S START WITH QUANTIZATION. FIRST I THINK IT'S IMPORTANT IF FIRST I THINK IT'S IMPORTANT IF FIRST I THINK IT'S IMPORTANT IF WE HAVE SOME AT LEAST BASIC WE HAVE SOME AT LEAST BASIC WE HAVE SOME AT LEAST BASIC UNDERSTANDING OF WHAT IS UNDERSTANDING OF WHAT IS UNDERSTANDING OF WHAT IS QUANTIZATION AND WHY WE ARE QUANTIZATION AND WHY WE ARE QUANTIZATION AND WHY WE ARE APPROACHING OUR TOOLS THE WAY APPROACHING OUR TOOLS THE WAY APPROACHING OUR TOOLS THE WAY WE'RE DOING IT. WE'RE DOING IT. WE'RE DOING IT. LET'S START WITH THIS. LET'S START WITH THIS. LET'S START WITH THIS. MULTIPLY ITS BASIC OPERATION MULTIPLY ITS BASIC OPERATION MULTIPLY ITS BASIC OPERATION FOR MACHINE LEARNING MODELS. FOR MACHINE LEARNING MODELS. FOR MACHINE LEARNING MODELS. YOU HAVE TWO MAJOR TENSORFLOW A YOU HAVE TWO MAJOR TENSORFLOW A YOU HAVE TWO MAJOR TENSORFLOW A AND B AND YOU DO SOME MULTIPLY AND B AND YOU DO SOME MULTIPLY AND B AND YOU DO SOME MULTIPLY INDICATION. INDICATION. INDICATION. IT'S A BUNCH OF VALUES THAT IT'S A BUNCH OF VALUES THAT IT'S A BUNCH OF VALUES THAT PRODUCE THE THIRD TENSOR. PRODUCE THE THIRD TENSOR. PRODUCE THE THIRD TENSOR. THEN JUST A LITTLE REMINDER THEN JUST A LITTLE REMINDER THEN JUST A LITTLE REMINDER ABOUT HOW THIS MULTIPLY WORKS. ABOUT HOW THIS MULTIPLY WORKS. ABOUT HOW THIS MULTIPLY WORKS. EACH ONE OF THESE RESULTS OF EACH ONE OF THESE RESULTS OF EACH ONE OF THESE RESULTS OF THE TENSOR C ARE COMPUTED AS THE TENSOR C ARE COMPUTED AS THE TENSOR C ARE COMPUTED AS MULTIPLICATION AND ACCUMULATION. MULTIPLICATION AND ACCUMULATION. MULTIPLICATION AND ACCUMULATION. SO IF WE LOOK AT ONE OF THEM SO IF WE LOOK AT ONE OF THEM SO IF WE LOOK AT ONE OF THEM THEN WE THINK OUR TRAINING THEN WE THINK OUR TRAINING THEN WE THINK OUR TRAINING THESE MODELS TYPICALLY IN THESE MODELS TYPICALLY IN THESE MODELS TYPICALLY IN HIGHER PRECISION, FLOW 32, YOU HIGHER PRECISION, FLOW 32, YOU HIGHER PRECISION, FLOW 32, YOU FOLLOW THE -- -- THE FOLLOW THE -- -- THE FOLLOW THE -- -- THE ACCUMULATION WILL ALSO BE FLOW ACCUMULATION WILL ALSO BE FLOW ACCUMULATION WILL ALSO BE FLOW 32. 32. 32. SO THIS IS -- (?) MACHINE SO THIS IS -- (?) MACHINE SO THIS IS -- (?) MACHINE LEARNING IS PRETTY GOOD DEALING LEARNING IS PRETTY GOOD DEALING LEARNING IS PRETTY GOOD DEALING WITH IT AT THESE LEVELS OF WITH IT AT THESE LEVELS OF WITH IT AT THESE LEVELS OF PRECISION SO NO PROBLEM. PRECISION SO NO PROBLEM. PRECISION SO NO PROBLEM. WHAT DOES IT HAVE TO DO WITH WHAT DOES IT HAVE TO DO WITH WHAT DOES IT HAVE TO DO WITH QUANTIZATION? QUANTIZATION? QUANTIZATION? WHAT OUR GOALS FOR OPTIMIZATION. WHAT OUR GOALS FOR OPTIMIZATION. WHAT OUR GOALS FOR OPTIMIZATION. WE WANT TO BE ABLE TO ADDRESS WE WANT TO BE ABLE TO ADDRESS WE WANT TO BE ABLE TO ADDRESS ALL THESE RESTRICTIONS. ALL THESE RESTRICTIONS. ALL THESE RESTRICTIONS. ALSO WE WANT TO BE ABLE TO ALSO WE WANT TO BE ABLE TO ALSO WE WANT TO BE ABLE TO DEPLOY AS MUCH HARDWARE AS DEPLOY AS MUCH HARDWARE AS DEPLOY AS MUCH HARDWARE AS POSSIBLE. POSSIBLE. POSSIBLE. SO COMMON THING WE DO IS LET'S SO COMMON THING WE DO IS LET'S SO COMMON THING WE DO IS LET'S REVIEW THE OPTIMIZATION WE'RE REVIEW THE OPTIMIZATION WE'RE REVIEW THE OPTIMIZATION WE'RE OPRATING. OPRATING. OPRATING. LET'S SAY GO FROM THE 32-BIT LET'S SAY GO FROM THE 32-BIT LET'S SAY GO FROM THE 32-BIT FLOWS TO -- (?) AND -- AND FLOWS TO -- (?) AND -- AND FLOWS TO -- (?) AND -- AND THESE WILL BE GOOD BECAUSE THEN THESE WILL BE GOOD BECAUSE THEN THESE WILL BE GOOD BECAUSE THEN WE ARE GOING FROM 32 BITS. WE ARE GOING FROM 32 BITS. WE ARE GOING FROM 32 BITS. SO WE'RE USING MEMORY, THE SO WE'RE USING MEMORY, THE SO WE'RE USING MEMORY, THE MODELS ARE FOUR TIMES SMALLER. MODELS ARE FOUR TIMES SMALLER. MODELS ARE FOUR TIMES SMALLER. AND THESE OPERATIONS ARE FASTER AND THESE OPERATIONS ARE FASTER AND THESE OPERATIONS ARE FASTER TO EXECUTE AND CONSUME LESS TO EXECUTE AND CONSUME LESS TO EXECUTE AND CONSUME LESS POWER. POWER. POWER. AND THEN BECAUSE THE PARAMETERS AND THEN BECAUSE THE PARAMETERS AND THEN BECAUSE THE PARAMETERS AND ALSO THE DYNAMIC VALUES ARE AND ALSO THE DYNAMIC VALUES ARE AND ALSO THE DYNAMIC VALUES ARE SMALLER THAN WE REDUCED -- IT SMALLER THAN WE REDUCED -- IT SMALLER THAN WE REDUCED -- IT MEANS THERE IS MORE ROOM FOR MEANS THERE IS MORE ROOM FOR MEANS THERE IS MORE ROOM FOR THINGS TO FLOW AROUND WHICH CAN THINGS TO FLOW AROUND WHICH CAN THINGS TO FLOW AROUND WHICH CAN ALSO TRANSLATE TO FASTER ALSO TRANSLATE TO FASTER ALSO TRANSLATE TO FASTER COMPUTERS AND REDUCED POWER. COMPUTERS AND REDUCED POWER. COMPUTERS AND REDUCED POWER. OPERATIONS THE END TO BE FAIRLY OPERATIONS THE END TO BE FAIRLY OPERATIONS THE END TO BE FAIRLY COMMON DENOMINATOR ACROSS COMMON DENOMINATOR ACROSS COMMON DENOMINATOR ACROSS HARDWARE. HARDWARE. HARDWARE. CPU, DIFFERENT APU SUPPORT THE CPU, DIFFERENT APU SUPPORT THE CPU, DIFFERENT APU SUPPORT THE OPERATIONS. OPERATIONS. OPERATIONS. SO WE HAVE -- WE WILL REVIEW SO WE HAVE -- WE WILL REVIEW SO WE HAVE -- WE WILL REVIEW THE PRECISION. THE PRECISION. THE PRECISION. HOW DO WE CONVERT 32 BITS FLOW. HOW DO WE CONVERT 32 BITS FLOW. HOW DO WE CONVERT 32 BITS FLOW. WE DO SOMETHING RIGHT NOW VERY WE DO SOMETHING RIGHT NOW VERY WE DO SOMETHING RIGHT NOW VERY SIMPLE, RIGHT? SIMPLE, RIGHT? SIMPLE, RIGHT? WE HAVE THIS LINEAR MAPPING WE HAVE THIS LINEAR MAPPING WE HAVE THIS LINEAR MAPPING WHERE WE SAY WE TAKE THE VALUES WHERE WE SAY WE TAKE THE VALUES WHERE WE SAY WE TAKE THE VALUES FROM A TENSOR AND COMPUTE THE FROM A TENSOR AND COMPUTE THE FROM A TENSOR AND COMPUTE THE MINIMUM AND MAXIMUM VALUE AND MINIMUM AND MAXIMUM VALUE AND MINIMUM AND MAXIMUM VALUE AND BASED ON THAT WE SPREAD THEM BASED ON THAT WE SPREAD THEM BASED ON THAT WE SPREAD THEM EVENLY. EVENLY. EVENLY. VERY SIMPLE, RIGHT? VERY SIMPLE, RIGHT? VERY SIMPLE, RIGHT? THESE ARE ALL WE NEED TO DO. THESE ARE ALL WE NEED TO DO. THESE ARE ALL WE NEED TO DO. WELL, I WISH. WELL, I WISH. WELL, I WISH. IT IS NOT THAT SIMPLE. IT IS NOT THAT SIMPLE. IT IS NOT THAT SIMPLE. HERE IS AN EXAMPLE. HERE IS AN EXAMPLE. HERE IS AN EXAMPLE. WE HAVE THE MULTIPLIER AND NOW WE HAVE THE MULTIPLIER AND NOW WE HAVE THE MULTIPLIER AND NOW LET'S SAY WE -- (?) THE LET'S SAY WE -- (?) THE LET'S SAY WE -- (?) THE MULTIPLIERS ARE -- AND THEN THE MULTIPLIERS ARE -- AND THEN THE MULTIPLIERS ARE -- AND THEN THE MULTIPLICATION THE PRODUCTS NOW MULTIPLICATION THE PRODUCTS NOW MULTIPLICATION THE PRODUCTS NOW YOU NEED 16 BITS REPRESENTED YOU NEED 16 BITS REPRESENTED YOU NEED 16 BITS REPRESENTED AND THEN YOU NEED TO ACCUMULATE AND THEN YOU NEED TO ACCUMULATE AND THEN YOU NEED TO ACCUMULATE AND YOU PROBABLY WANT 32 BITS AND YOU PROBABLY WANT 32 BITS AND YOU PROBABLY WANT 32 BITS TO ACCUMULATE, RIGHT? TO ACCUMULATE, RIGHT? TO ACCUMULATE, RIGHT? SO THE PROBLEM IS NOW YOU HAVE SO THE PROBLEM IS NOW YOU HAVE SO THE PROBLEM IS NOW YOU HAVE TENSOR RECEIVE FULL OF 32-BIT TENSOR RECEIVE FULL OF 32-BIT TENSOR RECEIVE FULL OF 32-BIT VALUES AND THAT IS NOT GREAT VALUES AND THAT IS NOT GREAT VALUES AND THAT IS NOT GREAT WHEN YOU WANT TO FEED THAT INTO WHEN YOU WANT TO FEED THAT INTO WHEN YOU WANT TO FEED THAT INTO ANOTHER -- YOU WANT TO EXECUTE ANOTHER -- YOU WANT TO EXECUTE ANOTHER -- YOU WANT TO EXECUTE -- ALREADY -- (?) WHAT YOU DO, -- ALREADY -- (?) WHAT YOU DO, -- ALREADY -- (?) WHAT YOU DO, YOU SCALE IT BACK DOWN. YOU SCALE IT BACK DOWN. YOU SCALE IT BACK DOWN. YOU QUANTIZE THEM ON THE FLY. YOU QUANTIZE THEM ON THE FLY. YOU QUANTIZE THEM ON THE FLY. YOU CAN FEED THEM AND MULTIPLY YOU CAN FEED THEM AND MULTIPLY YOU CAN FEED THEM AND MULTIPLY AND ALL GOOD. AND ALL GOOD. AND ALL GOOD. BUT WHILE THE IMPLICATIONS OF BUT WHILE THE IMPLICATIONS OF BUT WHILE THE IMPLICATIONS OF ALL THESE PROCESS? ALL THESE PROCESS? ALL THESE PROCESS? WE NEED TO STUDY THE WE NEED TO STUDY THE WE NEED TO STUDY THE PARAMETERS, THE WEIGHTS. PARAMETERS, THE WEIGHTS. PARAMETERS, THE WEIGHTS. WE ARE CHANGING ALSO THE WE ARE CHANGING ALSO THE WE ARE CHANGING ALSO THE DYNAMIC VALUES, ACTIVATIONS DYNAMIC VALUES, ACTIVATIONS DYNAMIC VALUES, ACTIVATIONS BECAUSE WE ARE SCALING THEM AND BECAUSE WE ARE SCALING THEM AND BECAUSE WE ARE SCALING THEM AND QUANTIZING THEM ON THE FLY. QUANTIZING THEM ON THE FLY. QUANTIZING THEM ON THE FLY. WE'RE CHANGING THE COMPUTATION, WE'RE CHANGING THE COMPUTATION, WE'RE CHANGING THE COMPUTATION, SIMPLE, WE HAVE A SCALING SIMPLE, WE HAVE A SCALING SIMPLE, WE HAVE A SCALING OPERATION. OPERATION. OPERATION. IT CAN BE MORE INVOLVED. IT CAN BE MORE INVOLVED. IT CAN BE MORE INVOLVED. SO YOU CAN SAY OKAY THAT SO YOU CAN SAY OKAY THAT SO YOU CAN SAY OKAY THAT DOESN'T SEEM THAT HARD. DOESN'T SEEM THAT HARD. DOESN'T SEEM THAT HARD. WE JUST HAD A SCALING OPERATION WE JUST HAD A SCALING OPERATION WE JUST HAD A SCALING OPERATION AND IT IS JUST EASY, RIGHT? AND IT IS JUST EASY, RIGHT? AND IT IS JUST EASY, RIGHT? WELL, SOME MATH IS A LITTLE BIT WELL, SOME MATH IS A LITTLE BIT WELL, SOME MATH IS A LITTLE BIT MORE COMPLICATED THAN THAT. MORE COMPLICATED THAN THAT. MORE COMPLICATED THAN THAT. THESE EXAMPLE FROM THESE EXAMPLE FROM THESE EXAMPLE FROM NORMALIZATION OF AN LSTM AND NORMALIZATION OF AN LSTM AND NORMALIZATION OF AN LSTM AND THIS ONE ASIDE FROM LOOKING A THIS ONE ASIDE FROM LOOKING A THIS ONE ASIDE FROM LOOKING A BIT MORE COMPLEX IS AN EXAMPLE BIT MORE COMPLEX IS AN EXAMPLE BIT MORE COMPLEX IS AN EXAMPLE WHERE YOU APPLY THE RULES OF -- WHERE YOU APPLY THE RULES OF -- WHERE YOU APPLY THE RULES OF -- (?). (?). (?). BLACK HOLE WHERE THE SCALES BLACK HOLE WHERE THE SCALES BLACK HOLE WHERE THE SCALES CANCEL EACH OTHER AND BASICALLY CANCEL EACH OTHER AND BASICALLY CANCEL EACH OTHER AND BASICALLY THINGS DON'T WORK IF YOU JUST THINGS DON'T WORK IF YOU JUST THINGS DON'T WORK IF YOU JUST GO NAIVELY ABOUT IT. GO NAIVELY ABOUT IT. GO NAIVELY ABOUT IT. AND THEN IT'S MORE COMPLICATED AND THEN IT'S MORE COMPLICATED AND THEN IT'S MORE COMPLICATED BECAUSE IN YOUR DECISION ABOUT BECAUSE IN YOUR DECISION ABOUT BECAUSE IN YOUR DECISION ABOUT HOW YOU ARE GOING TO REPRESET HOW YOU ARE GOING TO REPRESET HOW YOU ARE GOING TO REPRESET THIS COMPUTATION AND INTO YOUR THIS COMPUTATION AND INTO YOUR THIS COMPUTATION AND INTO YOUR FORM WE WANT TO BE EFFICIENT. FORM WE WANT TO BE EFFICIENT. FORM WE WANT TO BE EFFICIENT. LOWER PERCENTAGE IS GOOD. LOWER PERCENTAGE IS GOOD. LOWER PERCENTAGE IS GOOD. WE ALSO WANT TO BE ACCURATE WE ALSO WANT TO BE ACCURATE WE ALSO WANT TO BE ACCURATE WHICH MEANS LOWER PRECISION. WHICH MEANS LOWER PRECISION. WHICH MEANS LOWER PRECISION. SO IT'S A LOT OF TRADEOFFS THAT SO IT'S A LOT OF TRADEOFFS THAT SO IT'S A LOT OF TRADEOFFS THAT YOU HAVE TO DO. YOU HAVE TO DO. YOU HAVE TO DO. THEN FURTHER COMPLICATING THEN FURTHER COMPLICATING THEN FURTHER COMPLICATING THINGS IS WE HAVE THE HARDWARE. THINGS IS WE HAVE THE HARDWARE. THINGS IS WE HAVE THE HARDWARE. THERE IS ALL DIFFERENT TYPES OF THERE IS ALL DIFFERENT TYPES OF THERE IS ALL DIFFERENT TYPES OF HARDWARE WITH DIFFERENT CAP HARDWARE WITH DIFFERENT CAP HARDWARE WITH DIFFERENT CAP B*ILTS READY FOR OPERATIONS AND B*ILTS READY FOR OPERATIONS AND B*ILTS READY FOR OPERATIONS AND EACH HARDWARE HAS DIFFERENCE EACH HARDWARE HAS DIFFERENCE EACH HARDWARE HAS DIFFERENCE PREFERENCES. PREFERENCES. PREFERENCES. SOME HARDWARE IS BETTER AT SOME HARDWARE IS BETTER AT SOME HARDWARE IS BETTER AT SECURING OPERATIONS AND PRODUCE SECURING OPERATIONS AND PRODUCE SECURING OPERATIONS AND PRODUCE LOADS OR WITH DIFFERENT WIDTHS LOADS OR WITH DIFFERENT WIDTHS LOADS OR WITH DIFFERENT WIDTHS AND RESTRICTIONS. AND RESTRICTIONS. AND RESTRICTIONS. WE WANT TO ACCOUNT FOR ALL WE WANT TO ACCOUNT FOR ALL WE WANT TO ACCOUNT FOR ALL THOSE THINGS WHEN WE'RE THOSE THINGS WHEN WE'RE THOSE THINGS WHEN WE'RE CREATING OUR QUANTIZED PROGRAM CREATING OUR QUANTIZED PROGRAM CREATING OUR QUANTIZED PROGRAM OR RECIPE. OR RECIPE. OR RECIPE. THEN THERE IS A MACHINE THEN THERE IS A MACHINE THEN THERE IS A MACHINE LEARNING. LEARNING. LEARNING. IT IS HARD TO INTERPRET. IT IS HARD TO INTERPRET. IT IS HARD TO INTERPRET. WE DON'T UNDERSTAND HOW IT WE DON'T UNDERSTAND HOW IT WE DON'T UNDERSTAND HOW IT WORKS. WORKS. WORKS. NOT TO A LEVEL THAT WE CAN HAVE NOT TO A LEVEL THAT WE CAN HAVE NOT TO A LEVEL THAT WE CAN HAVE GOOD PROOFS TO KNOW THAT A GOOD PROOFS TO KNOW THAT A GOOD PROOFS TO KNOW THAT A TRANSFORMATION WE'RE DOING TO TRANSFORMATION WE'RE DOING TO TRANSFORMATION WE'RE DOING TO THESE MODELS OR PROGRAMS WILL THESE MODELS OR PROGRAMS WILL THESE MODELS OR PROGRAMS WILL ACTUALLY WORK OR NOT RESULT IN ACTUALLY WORK OR NOT RESULT IN ACTUALLY WORK OR NOT RESULT IN A CATASTROPHIC ERROR. A CATASTROPHIC ERROR. A CATASTROPHIC ERROR. YOU DON'T WANT TO QUANTIZE A YOU DON'T WANT TO QUANTIZE A YOU DON'T WANT TO QUANTIZE A MODEL AND THEY START GIVING YOU MODEL AND THEY START GIVING YOU MODEL AND THEY START GIVING YOU WEIRD RESULTS. WEIRD RESULTS. WEIRD RESULTS. SO IT MAKES IT A LITTLE -- MUCH SO IT MAKES IT A LITTLE -- MUCH SO IT MAKES IT A LITTLE -- MUCH MORE COMPLICATED TO DEFINE THIS MORE COMPLICATED TO DEFINE THIS MORE COMPLICATED TO DEFINE THIS TRANSFORMATION. TRANSFORMATION. TRANSFORMATION. AND THEN FINALLY I WILL SAY AND THEN FINALLY I WILL SAY AND THEN FINALLY I WILL SAY THIS IS ALL MORE COMPLICATED THIS IS ALL MORE COMPLICATED THIS IS ALL MORE COMPLICATED BECAUSE THE MODEL IS NOT ENOUGH. BECAUSE THE MODEL IS NOT ENOUGH. BECAUSE THE MODEL IS NOT ENOUGH. THE PROGRAM IS NOT ENOUGH. THE PROGRAM IS NOT ENOUGH. THE PROGRAM IS NOT ENOUGH. DEPENDING ON HOW THE DEPENDING ON HOW THE DEPENDING ON HOW THE QUANTIZATION IS DEFINED YOU QUANTIZATION IS DEFINED YOU QUANTIZATION IS DEFINED YOU MIGHT ALSO NEED SOME EXTRA DATA. MIGHT ALSO NEED SOME EXTRA DATA. MIGHT ALSO NEED SOME EXTRA DATA. AN EXAMPLE OF THE MULTIPLIER WE AN EXAMPLE OF THE MULTIPLIER WE AN EXAMPLE OF THE MULTIPLIER WE NEED IT TO COMPUTE THE MINIMUM NEED IT TO COMPUTE THE MINIMUM NEED IT TO COMPUTE THE MINIMUM AND MAXIMUM VALUES OF THE AND MAXIMUM VALUES OF THE AND MAXIMUM VALUES OF THE DYNAMIC ACTIVATIONS, RIGHT? DYNAMIC ACTIVATIONS, RIGHT? DYNAMIC ACTIVATIONS, RIGHT? AND THAT ONLY CAN BE DONE IF WE AND THAT ONLY CAN BE DONE IF WE AND THAT ONLY CAN BE DONE IF WE RUN INFERENCES THROUGH THE RUN INFERENCES THROUGH THE RUN INFERENCES THROUGH THE MODEL, RIGHT? MODEL, RIGHT? MODEL, RIGHT? WHICH MEANS YOU NEED TO PROVIDE WHICH MEANS YOU NEED TO PROVIDE WHICH MEANS YOU NEED TO PROVIDE SOME REPRESENTATIVE THERE. SOME REPRESENTATIVE THERE. SOME REPRESENTATIVE THERE. SO BASICALLY JUST ANOTHER THING SO BASICALLY JUST ANOTHER THING SO BASICALLY JUST ANOTHER THING OF WHAT YOU HAVE TO ACCOUNT OF WHAT YOU HAVE TO ACCOUNT OF WHAT YOU HAVE TO ACCOUNT WITH QUANTIZING THIS PROGRAM. WITH QUANTIZING THIS PROGRAM. WITH QUANTIZING THIS PROGRAM. SO BASICALLY WHEN WE TALK ABOUT SO BASICALLY WHEN WE TALK ABOUT SO BASICALLY WHEN WE TALK ABOUT QUANTIZATION WE'RE TALKING QUANTIZATION WE'RE TALKING QUANTIZATION WE'RE TALKING ABOUT REWRITING AND ABOUT REWRITING AND ABOUT REWRITING AND TRANSFORMING THESE MACHINE TRANSFORMING THESE MACHINE TRANSFORMING THESE MACHINE LEARNING PROGRAM TO A LEARNING PROGRAM TO A LEARNING PROGRAM TO A REPRESENTATION BASED ON THE REPRESENTATION BASED ON THE REPRESENTATION BASED ON THE OPERATIONS YOU HAVE AVAILABE. OPERATIONS YOU HAVE AVAILABE. OPERATIONS YOU HAVE AVAILABE. SO NOW HOW ARE WE ADDRESSING SO NOW HOW ARE WE ADDRESSING SO NOW HOW ARE WE ADDRESSING THESE IN OUR TOOLKIT? THESE IN OUR TOOLKIT? THESE IN OUR TOOLKIT? WELL, THE FIRST THING THAT WE WELL, THE FIRST THING THAT WE WELL, THE FIRST THING THAT WE DECIDED TO DO WAS TRY TO SCOPE DECIDED TO DO WAS TRY TO SCOPE DECIDED TO DO WAS TRY TO SCOPE DOWN THE PROBLEM AND SAY WE'RE DOWN THE PROBLEM AND SAY WE'RE DOWN THE PROBLEM AND SAY WE'RE GOING TO DEFINE THE GOING TO DEFINE THE GOING TO DEFINE THE SPECIFICATIONS FOR COMMON SPECIFICATIONS FOR COMMON SPECIFICATIONS FOR COMMON OPERATIONS LIKE IN THIS CASE -- OPERATIONS LIKE IN THIS CASE -- OPERATIONS LIKE IN THIS CASE -- (?) HAVE A WELL DEFINED (?) HAVE A WELL DEFINED (?) HAVE A WELL DEFINED QUANTIZATION BEHAVIOR, RIGHT? QUANTIZATION BEHAVIOR, RIGHT? QUANTIZATION BEHAVIOR, RIGHT? WE KNOW THAT NOW WITH THIS LOW WE KNOW THAT NOW WITH THIS LOW WE KNOW THAT NOW WITH THIS LOW LEVEL INFORMATION THERE IS LEVEL INFORMATION THERE IS LEVEL INFORMATION THERE IS RELEVANT QUANTIZATION THEN HOW RELEVANT QUANTIZATION THEN HOW RELEVANT QUANTIZATION THEN HOW DOES THE TARGET AND DOES THE TARGET AND DOES THE TARGET AND SPECIFICATIONS OR TOOLS CAN SPECIFICATIONS OR TOOLS CAN SPECIFICATIONS OR TOOLS CAN TARGET THE SPECIFICATION AND TARGET THE SPECIFICATION AND TARGET THE SPECIFICATION AND THEN WE CAN ALL WORK AT THIS THEN WE CAN ALL WORK AT THIS THEN WE CAN ALL WORK AT THIS LEVEL AND THEN WE ALSO GET THE LEVEL AND THEN WE ALSO GET THE LEVEL AND THEN WE ALSO GET THE BENEFIT FROM THE USER POINT OF BENEFIT FROM THE USER POINT OF BENEFIT FROM THE USER POINT OF VIEW YOU CAN QUANTIZE THE MODEL VIEW YOU CAN QUANTIZE THE MODEL VIEW YOU CAN QUANTIZE THE MODEL AND THIS MODEL CAN RUN IN AND THIS MODEL CAN RUN IN AND THIS MODEL CAN RUN IN DIFFERENT HARDWARE WITHOUT ANY DIFFERENT HARDWARE WITHOUT ANY DIFFERENT HARDWARE WITHOUT ANY CHANGE. CHANGE. CHANGE. SO RIGHT NOW IN OUR TOOLKIT WE SO RIGHT NOW IN OUR TOOLKIT WE SO RIGHT NOW IN OUR TOOLKIT WE SUPPORT THREE DIFFERENT SUPPORT THREE DIFFERENT SUPPORT THREE DIFFERENT QUANTIZATION TYPES. QUANTIZATION TYPES. QUANTIZATION TYPES. INCLUDING HERE REDUCED LOAD INCLUDING HERE REDUCED LOAD INCLUDING HERE REDUCED LOAD QUANTIZATION TYPE JUST A MUCH QUANTIZATION TYPE JUST A MUCH QUANTIZATION TYPE JUST A MUCH SIMPLER THING WHERE WE GO FROM SIMPLER THING WHERE WE GO FROM SIMPLER THING WHERE WE GO FROM FLOW 16 PARAMETERS AND FLOW 16 PARAMETERS AND FLOW 16 PARAMETERS AND COMPUTATIONS. COMPUTATIONS. COMPUTATIONS. PRETTY STRAIGHT FORWARD. PRETTY STRAIGHT FORWARD. PRETTY STRAIGHT FORWARD. THE NEXT ONE IS OUR HYBRID THE NEXT ONE IS OUR HYBRID THE NEXT ONE IS OUR HYBRID QUANTIZATION WHICH MAKES USE OF QUANTIZATION WHICH MAKES USE OF QUANTIZATION WHICH MAKES USE OF AV FOR THE BAROMETERS, BIASES AV FOR THE BAROMETERS, BIASES AV FOR THE BAROMETERS, BIASES AND ACTIVATIONS LEAVE US 32-BIT AND ACTIVATIONS LEAVE US 32-BIT AND ACTIVATIONS LEAVE US 32-BIT FLOWS AND THEN WE TRY TO BE AS FLOWS AND THEN WE TRY TO BE AS FLOWS AND THEN WE TRY TO BE AS SMART AS POSSIBLE FOR HOW WE SMART AS POSSIBLE FOR HOW WE SMART AS POSSIBLE FOR HOW WE EXECUTE THESE PROGRAMS. EXECUTE THESE PROGRAMS. EXECUTE THESE PROGRAMS. SO THE GOAL BEING THAT FOR SO THE GOAL BEING THAT FOR SO THE GOAL BEING THAT FOR EXAMPLE, HEAVY OPERATIONS LIKE EXAMPLE, HEAVY OPERATIONS LIKE EXAMPLE, HEAVY OPERATIONS LIKE MULTIPLIERS ARE LEFT IN (?) AND MULTIPLIERS ARE LEFT IN (?) AND MULTIPLIERS ARE LEFT IN (?) AND WE USE FLOATING POINTS FOR WE USE FLOATING POINTS FOR WE USE FLOATING POINTS FOR THINGS LIKE ACTIVATION THINGS LIKE ACTIVATION THINGS LIKE ACTIVATION FUNCTIONS. FUNCTIONS. FUNCTIONS. ACCURACY, PERFORMANCE. ACCURACY, PERFORMANCE. ACCURACY, PERFORMANCE. THEN THE THIRD ONE IS -- THEN THE THIRD ONE IS -- THEN THE THIRD ONE IS -- QUANTIZATION. QUANTIZATION. QUANTIZATION. EVERYTHING IS IN -- ALL THE EVERYTHING IS IN -- ALL THE EVERYTHING IS IN -- ALL THE PARAMETERS AND OPERATIONS ARE PARAMETERS AND OPERATIONS ARE PARAMETERS AND OPERATIONS ARE (?) THE MORE COMPLICTED ONE. (?) THE MORE COMPLICTED ONE. (?) THE MORE COMPLICTED ONE. SO THE BENEFITS OF THE REUSE SO THE BENEFITS OF THE REUSE SO THE BENEFITS OF THE REUSE FLOATS YOUR MODELS ARE NOW HALF FLOATS YOUR MODELS ARE NOW HALF FLOATS YOUR MODELS ARE NOW HALF SIZE. SIZE. SIZE. AND THEN DEPENDING ON THE AND THEN DEPENDING ON THE AND THEN DEPENDING ON THE SUPPORT YOU MIGHT GET SOME -- SUPPORT YOU MIGHT GET SOME -- SUPPORT YOU MIGHT GET SOME -- (?) AND LOSS IS MINIMAL. (?) AND LOSS IS MINIMAL. (?) AND LOSS IS MINIMAL. PRETTY MUCH ALWAYS WORKS. PRETTY MUCH ALWAYS WORKS. PRETTY MUCH ALWAYS WORKS. I HAVEN'T SEEN MYSELF A MODEL I HAVEN'T SEEN MYSELF A MODEL I HAVEN'T SEEN MYSELF A MODEL THAT DOESN'T WORK IN FLOW 16. THAT DOESN'T WORK IN FLOW 16. THAT DOESN'T WORK IN FLOW 16. IT WAS FLOW 32. IT WAS FLOW 32. IT WAS FLOW 32. QUANT -- FOR USER SIZE AND QUANT -- FOR USER SIZE AND QUANT -- FOR USER SIZE AND DEPENDING THE USER SIZE AND DEPENDING THE USER SIZE AND DEPENDING THE USER SIZE AND OPERATIONS YOU'RE USING YOU MAY OPERATIONS YOU'RE USING YOU MAY OPERATIONS YOU'RE USING YOU MAY GET DIFFERENT PERFORMANCE GET DIFFERENT PERFORMANCE GET DIFFERENT PERFORMANCE IMPROVEMENTS. IMPROVEMENTS. IMPROVEMENTS. TENDS TO BE LARGER FOR FULLY TENDS TO BE LARGER FOR FULLY TENDS TO BE LARGER FOR FULLY CONNECTED MODELS OR RNNS AND CONNECTED MODELS OR RNNS AND CONNECTED MODELS OR RNNS AND THEN THE THIRD ONE IS THE THEN THE THIRD ONE IS THE THEN THE THIRD ONE IS THE INTEGER -- IN TERMS OF MEMORY INTEGER -- IN TERMS OF MEMORY INTEGER -- IN TERMS OF MEMORY SIZE BUT IS FASTER TO EXECUTE SIZE BUT IS FASTER TO EXECUTE SIZE BUT IS FASTER TO EXECUTE AND HAS A GREATER ADVANTAGE IT AND HAS A GREATER ADVANTAGE IT AND HAS A GREATER ADVANTAGE IT HAS MORE HARDWARE COVERAGE. HAS MORE HARDWARE COVERAGE. HAS MORE HARDWARE COVERAGE. FOR EXAMPLE. FOR EXAMPLE. FOR EXAMPLE. TYPICAL MPUs SOME OF THEM ARE TYPICAL MPUs SOME OF THEM ARE TYPICAL MPUs SOME OF THEM ARE INTEGER BASED. INTEGER BASED. INTEGER BASED. NOW, LET'S TALK ABOUT THE TOOLS NOW, LET'S TALK ABOUT THE TOOLS NOW, LET'S TALK ABOUT THE TOOLS TO QUANTIZE THOSE TYPES. TO QUANTIZE THOSE TYPES. TO QUANTIZE THOSE TYPES. WE HAVE THOSE TYPES OF TOOLS. WE HAVE THOSE TYPES OF TOOLS. WE HAVE THOSE TYPES OF TOOLS. ONE THAT WORKS POST TRAINING, ONE THAT WORKS POST TRAINING, ONE THAT WORKS POST TRAINING, REGULAR TRAIN MODEL AND THE REGULAR TRAIN MODEL AND THE REGULAR TRAIN MODEL AND THE OTHER ONE THAT IS WORK IN OTHER ONE THAT IS WORK IN OTHER ONE THAT IS WORK IN PROGRESS DURING TRAINING. PROGRESS DURING TRAINING. PROGRESS DURING TRAINING. SO LET'S TALK ABOUT THE POST SO LET'S TALK ABOUT THE POST SO LET'S TALK ABOUT THE POST TRAINING. TRAINING. TRAINING. THE PROCESS IS VERY SIMPLE. THE PROCESS IS VERY SIMPLE. THE PROCESS IS VERY SIMPLE. YOU BASICALLY ASSUME THE USERS YOU BASICALLY ASSUME THE USERS YOU BASICALLY ASSUME THE USERS TRAIN MODEL. TRAIN MODEL. TRAIN MODEL. DOESN'T REALLY CARE HOW YOU DOESN'T REALLY CARE HOW YOU DOESN'T REALLY CARE HOW YOU TRAIN IT. TRAIN IT. TRAIN IT. JUST -- (?) THEN CURRENTLY WE JUST -- (?) THEN CURRENTLY WE JUST -- (?) THEN CURRENTLY WE HAVE TENSORFLOW CONVERTER, YOU HAVE TENSORFLOW CONVERTER, YOU HAVE TENSORFLOW CONVERTER, YOU CONVERT IT TO TENSORFLOW LIGHT CONVERT IT TO TENSORFLOW LIGHT CONVERT IT TO TENSORFLOW LIGHT AND QUANTIZES IT ON THE FLIGHT AND QUANTIZES IT ON THE FLIGHT AND QUANTIZES IT ON THE FLIGHT AND YOU HAVE A MODEL THAT YOU AND YOU HAVE A MODEL THAT YOU AND YOU HAVE A MODEL THAT YOU CAN EXECUTE ON WHATEVER CAN EXECUTE ON WHATEVER CAN EXECUTE ON WHATEVER HARDWARE IS SUPPORTED IN THE HARDWARE IS SUPPORTED IN THE HARDWARE IS SUPPORTED IN THE QUANTIZATION TYPE. QUANTIZATION TYPE. QUANTIZATION TYPE. SO NOW LET'S LOOK AT THE SO NOW LET'S LOOK AT THE SO NOW LET'S LOOK AT THE SPECIFIC QUANTIZATION TYPES. SPECIFIC QUANTIZATION TYPES. SPECIFIC QUANTIZATION TYPES. SO THE FIRST ONE WE HAVE A SO THE FIRST ONE WE HAVE A SO THE FIRST ONE WE HAVE A COUPLE OF FLIGHTS. COUPLE OF FLIGHTS. COUPLE OF FLIGHTS. USER DEFAULT OPTIMIZATIONS AND USER DEFAULT OPTIMIZATIONS AND USER DEFAULT OPTIMIZATIONS AND THESE ARE TAKE CARE OF CHANGING THESE ARE TAKE CARE OF CHANGING THESE ARE TAKE CARE OF CHANGING ALL THE PARAMETERS AND THE ALL THE PARAMETERS AND THE ALL THE PARAMETERS AND THE COMPUTATION AND AGAIN DEPENDING COMPUTATION AND AGAIN DEPENDING COMPUTATION AND AGAIN DEPENDING ON THE HARDWARE YOU'RE RUNNING ON THE HARDWARE YOU'RE RUNNING ON THE HARDWARE YOU'RE RUNNING THESE MODELS YOU MIGHT GET A THESE MODELS YOU MIGHT GET A THESE MODELS YOU MIGHT GET A SPEED-UP RIGHT NOW. SPEED-UP RIGHT NOW. SPEED-UP RIGHT NOW. IF YOU USE SUPPORT FLOW 16. IF YOU USE SUPPORT FLOW 16. IF YOU USE SUPPORT FLOW 16. YOU MIGHT GET SOME SPEED-UP YOU MIGHT GET SOME SPEED-UP YOU MIGHT GET SOME SPEED-UP THERE. THERE. THERE. EITHER BECAUSE OF THE EITHER BECAUSE OF THE EITHER BECAUSE OF THE COMPUTATION OR BECAUSE OF THE COMPUTATION OR BECAUSE OF THE COMPUTATION OR BECAUSE OF THE BANDWIDTH IN YOUR TUBE WILL BE BANDWIDTH IN YOUR TUBE WILL BE BANDWIDTH IN YOUR TUBE WILL BE REDUCED. REDUCED. REDUCED. LIKE I SAID BENEFITS THE SIZE LIKE I SAID BENEFITS THE SIZE LIKE I SAID BENEFITS THE SIZE GOES TO HALF. GOES TO HALF. GOES TO HALF. AND YOU KNOW, THE DROP IS VERY AND YOU KNOW, THE DROP IS VERY AND YOU KNOW, THE DROP IS VERY MINIMAL. MINIMAL. MINIMAL. I WILL SAY WITHIN THE NOISE. I WILL SAY WITHIN THE NOISE. I WILL SAY WITHIN THE NOISE. THEN THE NEXT ONE IS -- THIS IS THEN THE NEXT ONE IS -- THIS IS THEN THE NEXT ONE IS -- THIS IS VERY EASY. VERY EASY. VERY EASY. THIS IS THE DEFAULT FOR THIS IS THE DEFAULT FOR THIS IS THE DEFAULT FOR TENSORFLOW LIGHT DEFAULT AND TENSORFLOW LIGHT DEFAULT AND TENSORFLOW LIGHT DEFAULT AND THEN IT WILL MAKE SURE THE THEN IT WILL MAKE SURE THE THEN IT WILL MAKE SURE THE QUANTIZE PARAMETERS AND THEN QUANTIZE PARAMETERS AND THEN QUANTIZE PARAMETERS AND THEN OPERATIONS THAT DOESN'T HAVE OPERATIONS THAT DOESN'T HAVE OPERATIONS THAT DOESN'T HAVE DEFINED SPECIFICATIONS FOR DEFINED SPECIFICATIONS FOR DEFINED SPECIFICATIONS FOR QUANTIZE FORM WILL BE KEPT IN QUANTIZE FORM WILL BE KEPT IN QUANTIZE FORM WILL BE KEPT IN THE PRECISION AND YOU WILL GET THE PRECISION AND YOU WILL GET THE PRECISION AND YOU WILL GET SOME SPEED-OUTS AND BE ABLE TO SOME SPEED-OUTS AND BE ABLE TO SOME SPEED-OUTS AND BE ABLE TO EXECUTE IN WHATEVER HARDWARE EXECUTE IN WHATEVER HARDWARE EXECUTE IN WHATEVER HARDWARE COMPLIES WITH THE SPECIFICATION. COMPLIES WITH THE SPECIFICATION. COMPLIES WITH THE SPECIFICATION. SO TYPICALLY THIS ONE WORKS SO TYPICALLY THIS ONE WORKS SO TYPICALLY THIS ONE WORKS WELL FOR CPUs. WELL FOR CPUs. WELL FOR CPUs. AND AGAIN BENEFITS FOR ITS AND AGAIN BENEFITS FOR ITS AND AGAIN BENEFITS FOR ITS COMPRESSION FOR THE MODELS, AND COMPRESSION FOR THE MODELS, AND COMPRESSION FOR THE MODELS, AND THEN YOU GET SOME SPEED-OUTS, THEN YOU GET SOME SPEED-OUTS, THEN YOU GET SOME SPEED-OUTS, CONVOLUTION BASED MODELS. CONVOLUTION BASED MODELS. CONVOLUTION BASED MODELS. I WILL SAY THESE ARE ONE YEAR I WILL SAY THESE ARE ONE YEAR I WILL SAY THESE ARE ONE YEAR OLD NUMBERS SO PROBABLY RIGHT OLD NUMBERS SO PROBABLY RIGHT OLD NUMBERS SO PROBABLY RIGHT NOW IT IS FASTER. NOW IT IS FASTER. NOW IT IS FASTER. AND THE SAME FOR ACCURACY. AND THE SAME FOR ACCURACY. AND THE SAME FOR ACCURACY. THAT'S PRETTY GOOD. THAT'S PRETTY GOOD. THAT'S PRETTY GOOD. THAT'S WE'RE WORKING ON SOME THAT'S WE'RE WORKING ON SOME THAT'S WE'RE WORKING ON SOME CHANGES FOR CONVOLUTION MODELS CHANGES FOR CONVOLUTION MODELS CHANGES FOR CONVOLUTION MODELS IT WILL BE MORE ACCURATE. IT WILL BE MORE ACCURATE. IT WILL BE MORE ACCURATE. THEN THE THIRD ONE IS THE THEN THE THIRD ONE IS THE THEN THE THIRD ONE IS THE INTEGER QUANTIZATION. INTEGER QUANTIZATION. INTEGER QUANTIZATION. THIS ONE IS MORE COMPLEX. THIS ONE IS MORE COMPLEX. THIS ONE IS MORE COMPLEX. YOU NEED TO PROVIDE DATA. YOU NEED TO PROVIDE DATA. YOU NEED TO PROVIDE DATA. I WANT TO HAVE A MODEL BUT I I WANT TO HAVE A MODEL BUT I I WANT TO HAVE A MODEL BUT I WANT TO USE QUANTIZATION OR NOW WANT TO USE QUANTIZATION OR NOW WANT TO USE QUANTIZATION OR NOW YOU NEED TO PROVIDE SOME DATA YOU NEED TO PROVIDE SOME DATA YOU NEED TO PROVIDE SOME DATA AND BY DATA I MEAN LABEL AND BY DATA I MEAN LABEL AND BY DATA I MEAN LABEL SAMPLES OF WHAT YOUR NEURAL SAMPLES OF WHAT YOUR NEURAL SAMPLES OF WHAT YOUR NEURAL NETWORK WILL TYPICALLY SEE IN NETWORK WILL TYPICALLY SEE IN NETWORK WILL TYPICALLY SEE IN REALITY. REALITY. REALITY. IN THE PROCESSING MODEL YOU IN THE PROCESSING MODEL YOU IN THE PROCESSING MODEL YOU NEED TO SEE THE SOME NEED TO SEE THE SOME NEED TO SEE THE SOME PRE-PROCESSING MEASURES. PRE-PROCESSING MEASURES. PRE-PROCESSING MEASURES. WE AREN'T TALKING ABOUT A LOT WE AREN'T TALKING ABOUT A LOT WE AREN'T TALKING ABOUT A LOT OF DATA FOR THE RESULT THAT OF DATA FOR THE RESULT THAT OF DATA FOR THE RESULT THAT I'LL SHOW NEXT. I'LL SHOW NEXT. I'LL SHOW NEXT. WE ARE TALKING ABOUT 100 WE ARE TALKING ABOUT 100 WE ARE TALKING ABOUT 100 SAMPLES. SAMPLES. SAMPLES. THAT WORKS REALLY WELL. THAT WORKS REALLY WELL. THAT WORKS REALLY WELL. IT IS A LITTLE MORE COMPLICATED IT IS A LITTLE MORE COMPLICATED IT IS A LITTLE MORE COMPLICATED BUT NOT VERY COMPLICATED. BUT NOT VERY COMPLICATED. BUT NOT VERY COMPLICATED. SO THESE ARE SOME RESULTS FROM SO THESE ARE SOME RESULTS FROM SO THESE ARE SOME RESULTS FROM DECISION ACROSS DIFFERENT MODES. DECISION ACROSS DIFFERENT MODES. DECISION ACROSS DIFFERENT MODES. FOR THE MAJORITY OF MODELS THE FOR THE MAJORITY OF MODELS THE FOR THE MAJORITY OF MODELS THE LOSS IS NOT THAT BIG WITH LOSS IS NOT THAT BIG WITH LOSS IS NOT THAT BIG WITH RESPECT TO THE FULL PRECISION RESPECT TO THE FULL PRECISION RESPECT TO THE FULL PRECISION TRAINING BASELINE. TRAINING BASELINE. TRAINING BASELINE. THE ONLY ONE I WILL SAY IS THIS THE ONLY ONE I WILL SAY IS THIS THE ONLY ONE I WILL SAY IS THIS MODEL. MODEL. MODEL. THAT IS A BIT MORE MEANINGFUL THAT IS A BIT MORE MEANINGFUL THAT IS A BIT MORE MEANINGFUL DROP. DROP. DROP. BUT AGAIN OF OUR MODELS, WITH BUT AGAIN OF OUR MODELS, WITH BUT AGAIN OF OUR MODELS, WITH POST TRAINING QUANTIZATION. POST TRAINING QUANTIZATION. POST TRAINING QUANTIZATION. NOW TALK ABOUT -- (?) I AM NOW TALK ABOUT -- (?) I AM NOW TALK ABOUT -- (?) I AM SHOWING THE PREVIOUS RESULTS. SHOWING THE PREVIOUS RESULTS. SHOWING THE PREVIOUS RESULTS. THE MODELS -- FROM DOING THIS THE MODELS -- FROM DOING THIS THE MODELS -- FROM DOING THIS TRAINING QUANTIZATION TRAINING. TRAINING QUANTIZATION TRAINING. TRAINING QUANTIZATION TRAINING. BY THAT WE MEAN WE TRY TO BY THAT WE MEAN WE TRY TO BY THAT WE MEAN WE TRY TO EMULATE THE QUANTIZATION EMULATE THE QUANTIZATION EMULATE THE QUANTIZATION OPERATIONS, THE QUANTIZATION OPERATIONS, THE QUANTIZATION OPERATIONS, THE QUANTIZATION LOSSES DURING THE FORWARD PASS LOSSES DURING THE FORWARD PASS LOSSES DURING THE FORWARD PASS OF THE NEURAL NETWORK AND WE OF THE NEURAL NETWORK AND WE OF THE NEURAL NETWORK AND WE HOPE THE PARAMETERS WILL BE HOPE THE PARAMETERS WILL BE HOPE THE PARAMETERS WILL BE ABLE TO ACCOUNT FOR THAT. ABLE TO ACCOUNT FOR THAT. ABLE TO ACCOUNT FOR THAT. THE PROCESS FOR DOING THE THE PROCESS FOR DOING THE THE PROCESS FOR DOING THE QUANTIZATION WORK TRAINING FOR QUANTIZATION WORK TRAINING FOR QUANTIZATION WORK TRAINING FOR USING OUR API IS A LITTLE MORE USING OUR API IS A LITTLE MORE USING OUR API IS A LITTLE MORE INVOLVED BUT WE ARE TRYING TO INVOLVED BUT WE ARE TRYING TO INVOLVED BUT WE ARE TRYING TO MAKE IT VERY SIMPLE. MAKE IT VERY SIMPLE. MAKE IT VERY SIMPLE. SO WE BUILD THIS API IN KAIROS SO WE BUILD THIS API IN KAIROS SO WE BUILD THIS API IN KAIROS TO MAKE IT VERY EASY TO USE. TO MAKE IT VERY EASY TO USE. TO MAKE IT VERY EASY TO USE. WE ASSUME YOU ALREADY HAVE A WE ASSUME YOU ALREADY HAVE A WE ASSUME YOU ALREADY HAVE A KAIROS MODEL AND THEN YOU NEED KAIROS MODEL AND THEN YOU NEED KAIROS MODEL AND THEN YOU NEED TO CALL API TO APPLY THE TO CALL API TO APPLY THE TO CALL API TO APPLY THE QUANTIZATION. QUANTIZATION. QUANTIZATION. AND THIS MIGHT CHANGE A LITTLE AND THIS MIGHT CHANGE A LITTLE AND THIS MIGHT CHANGE A LITTLE BIT BUT IT WILL LOOK SOMETHING BIT BUT IT WILL LOOK SOMETHING BIT BUT IT WILL LOOK SOMETHING LIKE THIS. LIKE THIS. LIKE THIS. SO YOU JUST HAVE A MODEL THAT SO YOU JUST HAVE A MODEL THAT SO YOU JUST HAVE A MODEL THAT YOU ALREADY BUILD IN KAIROS AND YOU ALREADY BUILD IN KAIROS AND YOU ALREADY BUILD IN KAIROS AND THE ONLY THING YOU NEED TO DO THE ONLY THING YOU NEED TO DO THE ONLY THING YOU NEED TO DO IS GO API ON YOUR MODEL AND YOU IS GO API ON YOUR MODEL AND YOU IS GO API ON YOUR MODEL AND YOU GET NOW A MODEL THAT IS WRITTEN GET NOW A MODEL THAT IS WRITTEN GET NOW A MODEL THAT IS WRITTEN TO HAVE ALL THE QUANTIZATION TO HAVE ALL THE QUANTIZATION TO HAVE ALL THE QUANTIZATION AND THEN YOU JUST CALL YOUR FIT AND THEN YOU JUST CALL YOUR FIT AND THEN YOU JUST CALL YOUR FIT FUNCTION AND THAT'S IT. FUNCTION AND THAT'S IT. FUNCTION AND THAT'S IT. SO THEN YOU JUST TRAIN YOUR SO THEN YOU JUST TRAIN YOUR SO THEN YOU JUST TRAIN YOUR MODEL AS USUAL. MODEL AS USUAL. MODEL AS USUAL. AND THEN YOU CAN GO THROUGH THE AND THEN YOU CAN GO THROUGH THE AND THEN YOU CAN GO THROUGH THE CONVERTER AND IT WILL TAKE THIS CONVERTER AND IT WILL TAKE THIS CONVERTER AND IT WILL TAKE THIS MODEL AND QUANTIZATION AND HAVE MODEL AND QUANTIZATION AND HAVE MODEL AND QUANTIZATION AND HAVE THE MATERIAL TO QUANTIZE IT AND THE MATERIAL TO QUANTIZE IT AND THE MATERIAL TO QUANTIZE IT AND PRODUCE A MODEL JUST LIKE THE PRODUCE A MODEL JUST LIKE THE PRODUCE A MODEL JUST LIKE THE POST TRAIN MODEL YOU WILL BE POST TRAIN MODEL YOU WILL BE POST TRAIN MODEL YOU WILL BE ABLE TO EXECUTE IN DIFFERENT ABLE TO EXECUTE IN DIFFERENT ABLE TO EXECUTE IN DIFFERENT HARDWARE. HARDWARE. HARDWARE. THESE ARE SOME NUMBERS FROM THESE ARE SOME NUMBERS FROM THESE ARE SOME NUMBERS FROM QUANTIZATION TRAINING, QUANTIZATION TRAINING, QUANTIZATION TRAINING, PRELIMINARY NUMBERS. PRELIMINARY NUMBERS. PRELIMINARY NUMBERS. IF YOU SEE THE DELTA IS A IF YOU SEE THE DELTA IS A IF YOU SEE THE DELTA IS A LITTLE BIT BETTER THAN POST LITTLE BIT BETTER THAN POST LITTLE BIT BETTER THAN POST TRAINING QUANTIZATION THERE IS TRAINING QUANTIZATION THERE IS TRAINING QUANTIZATION THERE IS NOT A BIG DIFFERENCE. NOT A BIG DIFFERENCE. NOT A BIG DIFFERENCE. IT WAS 4% BEFORE, IN THIS CASE IT WAS 4% BEFORE, IN THIS CASE IT WAS 4% BEFORE, IN THIS CASE 2.9%. 2.9%. 2.9%. QUANTIZATION TRAINING IS THE QUANTIZATION TRAINING IS THE QUANTIZATION TRAINING IS THE USUAL TOOL AND WHY WE'RE USUAL TOOL AND WHY WE'RE USUAL TOOL AND WHY WE'RE BUILDING IT. BUILDING IT. BUILDING IT. NOW YOU MAY WONDER THOSE WERE A NOW YOU MAY WONDER THOSE WERE A NOW YOU MAY WONDER THOSE WERE A LOT OF QUANTIZATION TYPES AND LOT OF QUANTIZATION TYPES AND LOT OF QUANTIZATION TYPES AND PROOFS. PROOFS. PROOFS. WHICH ONE SHOULD I USE? WHICH ONE SHOULD I USE? WHICH ONE SHOULD I USE? MY RECOMMENDATION IS IF YOU ARE MY RECOMMENDATION IS IF YOU ARE MY RECOMMENDATION IS IF YOU ARE JUST STARTING START REDUCE JUST STARTING START REDUCE JUST STARTING START REDUCE FLOAT THE FIRST ONE TO TRY. FLOAT THE FIRST ONE TO TRY. FLOAT THE FIRST ONE TO TRY. VERY EASY TO USE. VERY EASY TO USE. VERY EASY TO USE. DOESN'T REQUIRE ANY DATA. DOESN'T REQUIRE ANY DATA. DOESN'T REQUIRE ANY DATA. IT WILL PROBABLY BE THE SAME IT WILL PROBABLY BE THE SAME IT WILL PROBABLY BE THE SAME AND LATENCY DEPENDING ON THE AND LATENCY DEPENDING ON THE AND LATENCY DEPENDING ON THE HARDWARE YOU MIGHT GET SOME HARDWARE YOU MIGHT GET SOME HARDWARE YOU MIGHT GET SOME BENEFITS. BENEFITS. BENEFITS. REDUCED LATENCY AND THEN REDUCED LATENCY AND THEN REDUCED LATENCY AND THEN COMPATIBLE EVERYWHERE YOU CAN COMPATIBLE EVERYWHERE YOU CAN COMPATIBLE EVERYWHERE YOU CAN EXECUTE FLOATING POINT EXECUTE FLOATING POINT EXECUTE FLOATING POINT OPERATIONS YOU WILL BE ABLE TO OPERATIONS YOU WILL BE ABLE TO OPERATIONS YOU WILL BE ABLE TO USE IT. USE IT. USE IT. THE NEXT THING TO TRY WOULD BE THE NEXT THING TO TRY WOULD BE THE NEXT THING TO TRY WOULD BE THE HYBRID QUANTIZATION. THE HYBRID QUANTIZATION. THE HYBRID QUANTIZATION. AGAIN THERE IS NO DATA AGAIN THERE IS NO DATA AGAIN THERE IS NO DATA REQUIREMENTS. REQUIREMENTS. REQUIREMENTS. THE -- FLOW 16 IN SOME CASES THE -- FLOW 16 IN SOME CASES THE -- FLOW 16 IN SOME CASES BUT IT IS STILL GOOD. BUT IT IS STILL GOOD. BUT IT IS STILL GOOD. IT WILL BE FASTER THAN THE IT WILL BE FASTER THAN THE IT WILL BE FASTER THAN THE REDUCED FLOAT AND BASICALLY REDUCED FLOAT AND BASICALLY REDUCED FLOAT AND BASICALLY COMPATIBLE EVERYWHEE THAT YOU COMPATIBLE EVERYWHEE THAT YOU COMPATIBLE EVERYWHEE THAT YOU HAVE SUPPORT FOR FLOWS AND HAVE SUPPORT FOR FLOWS AND HAVE SUPPORT FOR FLOWS AND INTEGER OPERATIONS. INTEGER OPERATIONS. INTEGER OPERATIONS. AND THE THIRD ONE TO TRY IS THE AND THE THIRD ONE TO TRY IS THE AND THE THIRD ONE TO TRY IS THE INTEGER QUANTIZATION WITH POST INTEGER QUANTIZATION WITH POST INTEGER QUANTIZATION WITH POST TRAINING TOOL. TRAINING TOOL. TRAINING TOOL. SO THIS ONE IS A BIT MORE SO THIS ONE IS A BIT MORE SO THIS ONE IS A BIT MORE COMPLICATED BECAUSE YOU NEED TO COMPLICATED BECAUSE YOU NEED TO COMPLICATED BECAUSE YOU NEED TO PROVIDE A LITTLE BIT OF DATA. PROVIDE A LITTLE BIT OF DATA. PROVIDE A LITTLE BIT OF DATA. THE -- WILL BE WORKS THROUGH THE -- WILL BE WORKS THROUGH THE -- WILL BE WORKS THROUGH THE SAME AS HYBRID BUT THE THE SAME AS HYBRID BUT THE THE SAME AS HYBRID BUT THE LATENCY WILL BE THE FASTEST. LATENCY WILL BE THE FASTEST. LATENCY WILL BE THE FASTEST. IT WILL ALSO GIVE YOU MORE IT WILL ALSO GIVE YOU MORE IT WILL ALSO GIVE YOU MORE HARDWARE COVERAGE. HARDWARE COVERAGE. HARDWARE COVERAGE. AND THEN THE LAST THING TO TRY AND THEN THE LAST THING TO TRY AND THEN THE LAST THING TO TRY WILL BE THE INTEGER WILL BE THE INTEGER WILL BE THE INTEGER QUANTIZATION WITH QUANTIZATION QUANTIZATION WITH QUANTIZATION QUANTIZATION WITH QUANTIZATION TRAINING AND BASICALLY THIS IS TRAINING AND BASICALLY THIS IS TRAINING AND BASICALLY THIS IS -- -- -- THIS WILL BE THE -- A LITTLE THIS WILL BE THE -- A LITTLE THIS WILL BE THE -- A LITTLE BIT MORE INVOLVED BECAUSE NOW BIT MORE INVOLVED BECAUSE NOW BIT MORE INVOLVED BECAUSE NOW YOU ARE DOING TRAINING AND YOU ARE DOING TRAINING AND YOU ARE DOING TRAINING AND SUPPOSED TO HAVE A TRAINING SUPPOSED TO HAVE A TRAINING SUPPOSED TO HAVE A TRAINING SETUP. SETUP. SETUP. IT WILL BE BETTER THAN DOING IT WILL BE BETTER THAN DOING IT WILL BE BETTER THAN DOING THE POST TRAINING VERSION AND THE POST TRAINING VERSION AND THE POST TRAINING VERSION AND AGAIN YOU GET THE BENEFITS OF AGAIN YOU GET THE BENEFITS OF AGAIN YOU GET THE BENEFITS OF BEING THE FASTEST ONE AND THE BEING THE FASTEST ONE AND THE BEING THE FASTEST ONE AND THE ONE WITH MORE HARDWARE COVERAGE. ONE WITH MORE HARDWARE COVERAGE. ONE WITH MORE HARDWARE COVERAGE. SO THAT WAS QUANTIZATION. SO THAT WAS QUANTIZATION. SO THAT WAS QUANTIZATION. AGAIN, ALL THESE TOOLS WE'RE AGAIN, ALL THESE TOOLS WE'RE AGAIN, ALL THESE TOOLS WE'RE TRYING TO MAKE VERY EASY TO USE. TRYING TO MAKE VERY EASY TO USE. TRYING TO MAKE VERY EASY TO USE. IT WILL BE GREAT IF YOU TRY IT WILL BE GREAT IF YOU TRY IT WILL BE GREAT IF YOU TRY THEM OUT AND GIVE US SOME THEM OUT AND GIVE US SOME THEM OUT AND GIVE US SOME FEEDBACK. FEEDBACK. FEEDBACK. CONNECTION PRUNING. CONNECTION PRUNING. CONNECTION PRUNING. SO WHAT IS NEURAL CONNECTION SO WHAT IS NEURAL CONNECTION SO WHAT IS NEURAL CONNECTION PRUNING? PRUNING? PRUNING? THE WAY WE HAVE IMPLEMENTED IT THE WAY WE HAVE IMPLEMENTED IT THE WAY WE HAVE IMPLEMENTED IT SO FAR IT MEANS THE TRAINING SO FAR IT MEANS THE TRAINING SO FAR IT MEANS THE TRAINING TEHNIQUE THAT DURING THE TEHNIQUE THAT DURING THE TEHNIQUE THAT DURING THE TRAINING PROCESS YOU WILL START TRAINING PROCESS YOU WILL START TRAINING PROCESS YOU WILL START REMOVING DROPPING CONNECTIONS REMOVING DROPPING CONNECTIONS REMOVING DROPPING CONNECTIONS FROM THE NEURAL NETWORK. FROM THE NEURAL NETWORK. FROM THE NEURAL NETWORK. AND THEN THESE CONNECTIONS WILL AND THEN THESE CONNECTIONS WILL AND THEN THESE CONNECTIONS WILL -- THE DROP CONNECTIONS IN THE -- THE DROP CONNECTIONS IN THE -- THE DROP CONNECTIONS IN THE TENSORS OF YOUR TRAINING AND TENSORS OF YOUR TRAINING AND TENSORS OF YOUR TRAINING AND YOU END UP WITH SPARSE TENSORS. YOU END UP WITH SPARSE TENSORS. YOU END UP WITH SPARSE TENSORS. THAT'S GREAT BECAUSE YOU CAN THAT'S GREAT BECAUSE YOU CAN THAT'S GREAT BECAUSE YOU CAN COMPRESS THEM AND POTENTIALLY COMPRESS THEM AND POTENTIALLY COMPRESS THEM AND POTENTIALLY EXECUTE THEM FASTER. EXECUTE THEM FASTER. EXECUTE THEM FASTER. SO THESE ARE AN EXAMPLE. SO THESE ARE AN EXAMPLE. SO THESE ARE AN EXAMPLE. THIS IS SAY A TENSOR HOW WE THIS IS SAY A TENSOR HOW WE THIS IS SAY A TENSOR HOW WE START, RANDOM INITIALIZED. START, RANDOM INITIALIZED. START, RANDOM INITIALIZED. THE DARK VALUES MEAN VALUES ARE THE DARK VALUES MEAN VALUES ARE THE DARK VALUES MEAN VALUES ARE NONE TO 0 AND WHITE MEANS THAT NONE TO 0 AND WHITE MEANS THAT NONE TO 0 AND WHITE MEANS THAT VALUES ARE HERE. VALUES ARE HERE. VALUES ARE HERE. AS THE TRAINING PROGRESSES YOU AS THE TRAINING PROGRESSES YOU AS THE TRAINING PROGRESSES YOU START BECOMING SPARSEER AND START BECOMING SPARSEER AND START BECOMING SPARSEER AND SPARSEER. SPARSEER. SPARSEER. IF YOU SEE THE TENSORS WE'RE IF YOU SEE THE TENSORS WE'RE IF YOU SEE THE TENSORS WE'RE REMOVING MOST OF THE PARAMETERS REMOVING MOST OF THE PARAMETERS REMOVING MOST OF THE PARAMETERS THERE. THERE. THERE. THE PRESENT API IS SIMILAR TO THE PRESENT API IS SIMILAR TO THE PRESENT API IS SIMILAR TO THE WORD TRAINING API TRYING TO THE WORD TRAINING API TRYING TO THE WORD TRAINING API TRYING TO BRING CONSISTENCY TO OUR API. BRING CONSISTENCY TO OUR API. BRING CONSISTENCY TO OUR API. YOU HAVE A MODEL THAT IS YOU HAVE A MODEL THAT IS YOU HAVE A MODEL THAT IS TRAINABLE IN KAIROS AND YOU ARE TRAINABLE IN KAIROS AND YOU ARE TRAINABLE IN KAIROS AND YOU ARE GOING TO CALL API TO APPLY THE GOING TO CALL API TO APPLY THE GOING TO CALL API TO APPLY THE PRUNING. PRUNING. PRUNING. THESE AGAIN WE'RE TRYING TO THESE AGAIN WE'RE TRYING TO THESE AGAIN WE'RE TRYING TO MAKE THIS AS SIMPLE AS POSSIBLE. MAKE THIS AS SIMPLE AS POSSIBLE. MAKE THIS AS SIMPLE AS POSSIBLE. THE ONLY THING YOU NEED TO THE ONLY THING YOU NEED TO THE ONLY THING YOU NEED TO DEFINE IS THE PRUNING SCHEDULE. DEFINE IS THE PRUNING SCHEDULE. DEFINE IS THE PRUNING SCHEDULE. BASICALLY WHEN YOU WANT TO BASICALLY WHEN YOU WANT TO BASICALLY WHEN YOU WANT TO START DROPPING THESE START DROPPING THESE START DROPPING THESE CONNECTIONS UNTIL WHEN AND HOW CONNECTIONS UNTIL WHEN AND HOW CONNECTIONS UNTIL WHEN AND HOW FAST AND HOW AGGRESSIVE YOU FAST AND HOW AGGRESSIVE YOU FAST AND HOW AGGRESSIVE YOU WANT THE PRUNING TO BE. WANT THE PRUNING TO BE. WANT THE PRUNING TO BE. AND THEN YOU JUST GO OVER PRUNE AND THEN YOU JUST GO OVER PRUNE AND THEN YOU JUST GO OVER PRUNE FUNCTION WHICH AGAIN WILL FUNCTION WHICH AGAIN WILL FUNCTION WHICH AGAIN WILL MODIFY YOUR GRAPH TO ADD ALL MODIFY YOUR GRAPH TO ADD ALL MODIFY YOUR GRAPH TO ADD ALL THE PRUNING OPERATIONS THE PRUNING OPERATIONS THE PRUNING OPERATIONS INTERNALLY AND YOU JUST CALL INTERNALLY AND YOU JUST CALL INTERNALLY AND YOU JUST CALL YOUR FIT FUNCTION AND TRAINING YOUR FIT FUNCTION AND TRAINING YOUR FIT FUNCTION AND TRAINING AS USUAL. AS USUAL. AS USUAL. SO BASICALLY YOU TRAIN AS USUAL SO BASICALLY YOU TRAIN AS USUAL SO BASICALLY YOU TRAIN AS USUAL AND THEN ONCE YOU TRAIN YOU AND THEN ONCE YOU TRAIN YOU AND THEN ONCE YOU TRAIN YOU HAVE TWO OPTIONS NOW. HAVE TWO OPTIONS NOW. HAVE TWO OPTIONS NOW. SOON YOU WILL HAVE TWO OPTIONS. SOON YOU WILL HAVE TWO OPTIONS. SOON YOU WILL HAVE TWO OPTIONS. YOU CAN JUST TAKE THIS SAVE YOU CAN JUST TAKE THIS SAVE YOU CAN JUST TAKE THIS SAVE MODEL AND JUST COMPRESS IT AND MODEL AND JUST COMPRESS IT AND MODEL AND JUST COMPRESS IT AND THE MODEL WILL BE SMALLER. THE MODEL WILL BE SMALLER. THE MODEL WILL BE SMALLER. AND SOON YOU WILL BE ABLE TO AND SOON YOU WILL BE ABLE TO AND SOON YOU WILL BE ABLE TO CONVERT IT TO TENSORFLOW LIGHT CONVERT IT TO TENSORFLOW LIGHT CONVERT IT TO TENSORFLOW LIGHT AND YOU WILL GET ALSO REDUCTION AND YOU WILL GET ALSO REDUCTION AND YOU WILL GET ALSO REDUCTION IN SIZE AND POTENTIALLY -- IN SIZE AND POTENTIALLY -- IN SIZE AND POTENTIALLY -- DEPENDING ON WHAT CONFIGURATION DEPENDING ON WHAT CONFIGURATION DEPENDING ON WHAT CONFIGURATION YOU'RE USING AND THE HARDWARE YOU'RE USING AND THE HARDWARE YOU'RE USING AND THE HARDWARE YOU'RE TARGETING. YOU'RE TARGETING. YOU'RE TARGETING. SO THIS SHOULD BE DONE PRETTY SO THIS SHOULD BE DONE PRETTY SO THIS SHOULD BE DONE PRETTY SOON. SOON. SOON. NOW WHAT ARE THE BENEFITS OF NOW WHAT ARE THE BENEFITS OF NOW WHAT ARE THE BENEFITS OF PRUNING? PRUNING? PRUNING? WE'VE TRIED IT IN A LOT OF WE'VE TRIED IT IN A LOT OF WE'VE TRIED IT IN A LOT OF TASKS, REALLY A LOT OF TASKS TASKS, REALLY A LOT OF TASKS TASKS, REALLY A LOT OF TASKS AND IT WORKS PRETTY WELL. AND IT WORKS PRETTY WELL. AND IT WORKS PRETTY WELL. A LOT OF THINGS ARE HYPER A LOT OF THINGS ARE HYPER A LOT OF THINGS ARE HYPER PARAMETER TUNING AND CAREFUL PARAMETER TUNING AND CAREFUL PARAMETER TUNING AND CAREFUL FOR STARTING YOUR MODELS AND FOR STARTING YOUR MODELS AND FOR STARTING YOUR MODELS AND THINGS LIKE THAT. THINGS LIKE THAT. THINGS LIKE THAT. IT HAS WORKED WELL WITHOUT A IT HAS WORKED WELL WITHOUT A IT HAS WORKED WELL WITHOUT A LOT OF BABYSITTING. LOT OF BABYSITTING. LOT OF BABYSITTING. IT HAS POTENTIAL FOR -- IT HAS POTENTIAL FOR -- IT HAS POTENTIAL FOR -- DEPENDING ON HARDWARE SUPPORT DEPENDING ON HARDWARE SUPPORT DEPENDING ON HARDWARE SUPPORT AND WE ALSO HAVE PRETTY GOOD AND WE ALSO HAVE PRETTY GOOD AND WE ALSO HAVE PRETTY GOOD RESULTS. RESULTS. RESULTS. WE CAN MAKE A LOT OF THE WE CAN MAKE A LOT OF THE WE CAN MAKE A LOT OF THE PARAMETERS BASICALLY GO AWAY. PARAMETERS BASICALLY GO AWAY. PARAMETERS BASICALLY GO AWAY. WE SEE 50% WITH NEGLIGIBLE WE SEE 50% WITH NEGLIGIBLE WE SEE 50% WITH NEGLIGIBLE ACCURACY LOSS AND THE OTHER ACCURACY LOSS AND THE OTHER ACCURACY LOSS AND THE OTHER GREAT THING IT WORKS WELL WITH GREAT THING IT WORKS WELL WITH GREAT THING IT WORKS WELL WITH QUANTIZATION. QUANTIZATION. QUANTIZATION. A TYPICAL SETUP WE TRY. A TYPICAL SETUP WE TRY. A TYPICAL SETUP WE TRY. TRAINING PRUNING AND POST TRAINING PRUNING AND POST TRAINING PRUNING AND POST TRAINING QUANTIZATION AND YOU TRAINING QUANTIZATION AND YOU TRAINING QUANTIZATION AND YOU GET THE COMPOUND BENEFIT OF GET THE COMPOUND BENEFIT OF GET THE COMPOUND BENEFIT OF BOTH TECHNIQUES. BOTH TECHNIQUES. BOTH TECHNIQUES. THESE ARE SOME -- WE SHOW WE THESE ARE SOME -- WE SHOW WE THESE ARE SOME -- WE SHOW WE HAD WHEN WE LAUNCHED THIS. HAD WHEN WE LAUNCHED THIS. HAD WHEN WE LAUNCHED THIS. YOU SEE WE CAN GET ALL THE WAY YOU SEE WE CAN GET ALL THE WAY YOU SEE WE CAN GET ALL THE WAY ALMOST TO 90% SPARSETY WITH ALMOST TO 90% SPARSETY WITH ALMOST TO 90% SPARSETY WITH RELATIVELY SMALL ACCURACY RELATIVELY SMALL ACCURACY RELATIVELY SMALL ACCURACY LOSSES. LOSSES. LOSSES. WE CAN TAKE IT TO ALMOST 90% WE CAN TAKE IT TO ALMOST 90% WE CAN TAKE IT TO ALMOST 90% PRUNING AND ALSO SMALLER PRUNING AND ALSO SMALLER PRUNING AND ALSO SMALLER ACCURACY LOSSES. ACCURACY LOSSES. ACCURACY LOSSES. WE HAVE DONE THESE FOR -- (?) WE HAVE DONE THESE FOR -- (?) WE HAVE DONE THESE FOR -- (?) WE HAD RECENTLY THE GOOGLE WE HAD RECENTLY THE GOOGLE WE HAD RECENTLY THE GOOGLE PIXEL EVENT WHERE THE MODELS PIXEL EVENT WHERE THE MODELS PIXEL EVENT WHERE THE MODELS ARE USED PRUNING AND ARE USED PRUNING AND ARE USED PRUNING AND QUANTIZATION AND WE WERE ABLE QUANTIZATION AND WE WERE ABLE QUANTIZATION AND WE WERE ABLE TO HAVE A MODEL WITH SERVERS TO HAVE A MODEL WITH SERVERS TO HAVE A MODEL WITH SERVERS HIGH QUALITY RUNNING ON OUR HIGH QUALITY RUNNING ON OUR HIGH QUALITY RUNNING ON OUR PHONE. PHONE. PHONE. WHICH IS REALLY GOOD. WHICH IS REALLY GOOD. WHICH IS REALLY GOOD. SO NOW FINALLY REAL QUICK. SO NOW FINALLY REAL QUICK. SO NOW FINALLY REAL QUICK. LIKE I MENTIONED WE'RE WORKING LIKE I MENTIONED WE'RE WORKING LIKE I MENTIONED WE'RE WORKING ON QUANTIZATION TRAINING API ON QUANTIZATION TRAINING API ON QUANTIZATION TRAINING API AND IT SHOULD BE READY SOON AND AND IT SHOULD BE READY SOON AND AND IT SHOULD BE READY SOON AND QUANTIZING OUR -- (?). QUANTIZING OUR -- (?). QUANTIZING OUR -- (?). THEN I INCLUDE WE'RE MAKING THEN I INCLUDE WE'RE MAKING THEN I INCLUDE WE'RE MAKING IMPROVE MGTS TO THE HYBRID IMPROVE MGTS TO THE HYBRID IMPROVE MGTS TO THE HYBRID QUANTIZATION TO BE MORE QUANTIZATION TO BE MORE QUANTIZATION TO BE MORE ACCURATE FOR -- (?) ACCURATE FOR -- (?) ACCURATE FOR -- (?) LONGER TERM I DON'T KNOW IF YOU LONGER TERM I DON'T KNOW IF YOU LONGER TERM I DON'T KNOW IF YOU HAVE HEARD ABOUT -- HAVE HEARD ABOUT -- HAVE HEARD ABOUT -- STATE-OF-THE-ART INFRASTRUCTURE. STATE-OF-THE-ART INFRASTRUCTURE. STATE-OF-THE-ART INFRASTRUCTURE. THIS IS PARTICULARLY THIS IS PARTICULARLY THIS IS PARTICULARLY INTERESTING TO US BECAUSE IT IS INTERESTING TO US BECAUSE IT IS INTERESTING TO US BECAUSE IT IS A BETTER WAY FOR US TO WRITE A BETTER WAY FOR US TO WRITE A BETTER WAY FOR US TO WRITE THESE TRANSFORMATIONS AND AT THESE TRANSFORMATIONS AND AT THESE TRANSFORMATIONS AND AT THE BEGINNING OF THE TALK WE'RE THE BEGINNING OF THE TALK WE'RE THE BEGINNING OF THE TALK WE'RE TAKING THE MODERN WAY OF TAKING THE MODERN WAY OF TAKING THE MODERN WAY OF TRANSFORMING ONE PROGRAM INTO TRANSFORMING ONE PROGRAM INTO TRANSFORMING ONE PROGRAM INTO ANOTHER. ANOTHER. ANOTHER. SOME OF THE THINGS WE WANT TO SOME OF THE THINGS WE WANT TO SOME OF THE THINGS WE WANT TO ENABLE IS BETTER TARGETED ENABLE IS BETTER TARGETED ENABLE IS BETTER TARGETED HARDWARE. HARDWARE. HARDWARE. WHERE SPECIFICATIONS ARE GREAT. WHERE SPECIFICATIONS ARE GREAT. WHERE SPECIFICATIONS ARE GREAT. USERS CAN TARGET AND EXECUTE USERS CAN TARGET AND EXECUTE USERS CAN TARGET AND EXECUTE DIFFERENT HARDWARE. DIFFERENT HARDWARE. DIFFERENT HARDWARE. SOME USERS JUST WANT TO STAY IN SOME USERS JUST WANT TO STAY IN SOME USERS JUST WANT TO STAY IN ONE HARDWARE AND GET THE BEST ONE HARDWARE AND GET THE BEST ONE HARDWARE AND GET THE BEST OUT OF IT. OUT OF IT. OUT OF IT. WE'RE HOPING WITH THE NEW WE'RE HOPING WITH THE NEW WE'RE HOPING WITH THE NEW INFRASTRUCTURE WE'RE BUILDING INFRASTRUCTURE WE'RE BUILDING INFRASTRUCTURE WE'RE BUILDING IT SHOULD BE POSSIBLE. IT SHOULD BE POSSIBLE. IT SHOULD BE POSSIBLE. FINALLY I REALLY JUST WANT TO FINALLY I REALLY JUST WANT TO FINALLY I REALLY JUST WANT TO ENCOURAGE YOU TO TRY IT OUT AND ENCOURAGE YOU TO TRY IT OUT AND ENCOURAGE YOU TO TRY IT OUT AND GIVE US FEEDBACK WHAT GIVE US FEEDBACK WHAT GIVE US FEEDBACK WHAT TECHNIQUES YOU WOULD LIKE TO TECHNIQUES YOU WOULD LIKE TO TECHNIQUES YOU WOULD LIKE TO SEE. SEE. SEE. RESEARCHES, THERE IS TECHNIQUES RESEARCHES, THERE IS TECHNIQUES RESEARCHES, THERE IS TECHNIQUES POPPING UP-TO-DATE ALL OVER THE POPPING UP-TO-DATE ALL OVER THE POPPING UP-TO-DATE ALL OVER THE PLACE AND A LOT OF THE WORK WE PLACE AND A LOT OF THE WORK WE PLACE AND A LOT OF THE WORK WE HAVE TO GO THROUGH IS WHAT IS HAVE TO GO THROUGH IS WHAT IS HAVE TO GO THROUGH IS WHAT IS USEFUL AND NOT AND WHAT IS USEFUL AND NOT AND WHAT IS USEFUL AND NOT AND WHAT IS GENERAL AND WHAT IS VERY GENERAL AND WHAT IS VERY GENERAL AND WHAT IS VERY SPECIFIC. SPECIFIC. SPECIFIC. WE WOULD LOVE TO HEAR YOUR WE WOULD LOVE TO HEAR YOUR WE WOULD LOVE TO HEAR YOUR FEEDBACK ABOUT THAT AND ALSO FEEDBACK ABOUT THAT AND ALSO FEEDBACK ABOUT THAT AND ALSO ABOUT THE TOOLS THAT WE ALREADY ABOUT THE TOOLS THAT WE ALREADY ABOUT THE TOOLS THAT WE ALREADY HAVE IN THE TOOLKIT. HAVE IN THE TOOLKIT. HAVE IN THE TOOLKIT. WE'RE TRYING TO MAKE THEM WE'RE TRYING TO MAKE THEM WE'RE TRYING TO MAKE THEM EASIER FOR PEOPLE TO USE. EASIER FOR PEOPLE TO USE. EASIER FOR PEOPLE TO USE. WE KNOW WE STILL HAVE A LONG WE KNOW WE STILL HAVE A LONG WE KNOW WE STILL HAVE A LONG WAY TO GO BUT ANYTHING FEEDBACK WAY TO GO BUT ANYTHING FEEDBACK WAY TO GO BUT ANYTHING FEEDBACK YOU CAN PROVIDE WOULD BE REALLY YOU CAN PROVIDE WOULD BE REALLY YOU CAN PROVIDE WOULD BE REALLY APPRECIATED AND I THINK THERE APPRECIATED AND I THINK THERE APPRECIATED AND I THINK THERE IS A LITTLE BIT OF TIME FOR IS A LITTLE BIT OF TIME FOR IS A LITTLE BIT OF TIME FOR QUESTIONS IF ANY OF YOU HAVE QUESTIONS IF ANY OF YOU HAVE QUESTIONS IF ANY OF YOU HAVE QUESTIONS. QUESTIONS. QUESTIONS. [APPLAUSE] [APPLAUSE] [APPLAUSE] THANKS. >> HI, THANK YOU FOR THE >> HI, THANK YOU FOR THE >> HI, THANK YOU FOR THE PRESENTATION. PRESENTATION. PRESENTATION. I HAVE A QUESTION REGARDING THE I HAVE A QUESTION REGARDING THE I HAVE A QUESTION REGARDING THE TRAINING WITH INTEGER TRAINING WITH INTEGER TRAINING WITH INTEGER QUANTIZATIONS. QUANTIZATIONS. QUANTIZATIONS. IN THE PIPELINE IS THAT GOING IN THE PIPELINE IS THAT GOING IN THE PIPELINE IS THAT GOING TO BE TRUE QUANTIZATION DURING TO BE TRUE QUANTIZATION DURING TO BE TRUE QUANTIZATION DURING TRAINING? TRAINING? TRAINING? >> NO, RIGHT NOW THE WAY -- BY >> NO, RIGHT NOW THE WAY -- BY >> NO, RIGHT NOW THE WAY -- BY TRUE YOU EXPECT ALL APPEAR TRUE YOU EXPECT ALL APPEAR TRUE YOU EXPECT ALL APPEAR RATIONS HAPPEN IN THE INTEGER. RATIONS HAPPEN IN THE INTEGER. RATIONS HAPPEN IN THE INTEGER. NOT RIGHT NOW. NOT RIGHT NOW. NOT RIGHT NOW. SOMETHING I WANT ENABLED SOMETHING I WANT ENABLED SOMETHING I WANT ENABLED BECAUSE I WANT TO MAKE TRAINING BECAUSE I WANT TO MAKE TRAINING BECAUSE I WANT TO MAKE TRAINING AVAILABLE BUT RIGHT NOW THE WAY AVAILABLE BUT RIGHT NOW THE WAY AVAILABLE BUT RIGHT NOW THE WAY THAT WE'RE TARGETING IS I DON'T THAT WE'RE TARGETING IS I DON'T THAT WE'RE TARGETING IS I DON'T KNOW IF YOU'RE FAMILIAR WITH KNOW IF YOU'RE FAMILIAR WITH KNOW IF YOU'RE FAMILIAR WITH TENSORFLOW APIs, WE HAVE THE TENSORFLOW APIs, WE HAVE THE TENSORFLOW APIs, WE HAVE THE LOW-LEVEL API QUANTIZATION THAT LOW-LEVEL API QUANTIZATION THAT LOW-LEVEL API QUANTIZATION THAT EMULATES THESE LOSSES. EMULATES THESE LOSSES. EMULATES THESE LOSSES. AND THAT ONE IS STILL BASICALLY AND THAT ONE IS STILL BASICALLY AND THAT ONE IS STILL BASICALLY WHAT WE DO THERE IS WE QUANTIZE WHAT WE DO THERE IS WE QUANTIZE WHAT WE DO THERE IS WE QUANTIZE PARAMETERS AND QUANTIZE THEM PARAMETERS AND QUANTIZE THEM PARAMETERS AND QUANTIZE THEM AND DO THE FLOW OPERATION. AND DO THE FLOW OPERATION. AND DO THE FLOW OPERATION. THAT'S WHAT WE'RE USING RIGHT THAT'S WHAT WE'RE USING RIGHT THAT'S WHAT WE'RE USING RIGHT NOW. NOW. NOW. LONGER TERM WE WANT TO USE IT LONGER TERM WE WANT TO USE IT LONGER TERM WE WANT TO USE IT THROUGH INTEGER FORWARD PACKETS. THROUGH INTEGER FORWARD PACKETS. THROUGH INTEGER FORWARD PACKETS. >> THANK YOU. >> THANK YOU. >> THANK YOU. >> ONE QUESTION. >> ONE QUESTION. >> ONE QUESTION. SO AFTER YOU DO THE SO AFTER YOU DO THE SO AFTER YOU DO THE QUANTIZATION, IS THERE A WAY QUANTIZATION, IS THERE A WAY QUANTIZATION, IS THERE A WAY YOU CAN ALSO VISUALIZE OF THE YOU CAN ALSO VISUALIZE OF THE YOU CAN ALSO VISUALIZE OF THE FINISHED QUANT ADVERTISED MODEL. FINISHED QUANT ADVERTISED MODEL. FINISHED QUANT ADVERTISED MODEL. IS THERE A WAY YOU CAN ALSO -- IS THERE A WAY YOU CAN ALSO -- IS THERE A WAY YOU CAN ALSO -- THE OTHER QUESTION WAS WHAT THE OTHER QUESTION WAS WHAT THE OTHER QUESTION WAS WHAT SORT OF TOOLS WILL YOU PROVIDE SORT OF TOOLS WILL YOU PROVIDE SORT OF TOOLS WILL YOU PROVIDE AS FAR AS SORT OF DO LIKE MODEL AS FAR AS SORT OF DO LIKE MODEL AS FAR AS SORT OF DO LIKE MODEL CORRECTNESS AT LEAST EVALUATE CORRECTNESS AT LEAST EVALUATE CORRECTNESS AT LEAST EVALUATE WHETHER THE QUANTIZED MODEL IS WHETHER THE QUANTIZED MODEL IS WHETHER THE QUANTIZED MODEL IS FUNCTIONALLY CORRECT IN A SENSE? FUNCTIONALLY CORRECT IN A SENSE? FUNCTIONALLY CORRECT IN A SENSE? >> VISUALIZATION, WE HAVE >> VISUALIZATION, WE HAVE >> VISUALIZATION, WE HAVE ADDITIONAL -- YOU CAN SEE THE ADDITIONAL -- YOU CAN SEE THE ADDITIONAL -- YOU CAN SEE THE QUANTIZED MODEL. QUANTIZED MODEL. QUANTIZED MODEL. I DON'T KNOW IF IT WILL GIVE I DON'T KNOW IF IT WILL GIVE I DON'T KNOW IF IT WILL GIVE YOU A LOT OF INFORMATION YOU A LOT OF INFORMATION YOU A LOT OF INFORMATION DEPENDING WHAT YOU ARE LOOKING DEPENDING WHAT YOU ARE LOOKING DEPENDING WHAT YOU ARE LOOKING FOR. FOR. FOR. WE ALSO WANT TO MAKE OUR WE ALSO WANT TO MAKE OUR WE ALSO WANT TO MAKE OUR TOOLING A BIT BETTER PERHAPS TOOLING A BIT BETTER PERHAPS TOOLING A BIT BETTER PERHAPS FOR WHATEVER REASON YOU WANT TO FOR WHATEVER REASON YOU WANT TO FOR WHATEVER REASON YOU WANT TO GET RESEARCH AND START LOOKING GET RESEARCH AND START LOOKING GET RESEARCH AND START LOOKING AT THE ACTIVATIONS AND HOW THEY AT THE ACTIVATIONS AND HOW THEY AT THE ACTIVATIONS AND HOW THEY CHANGE. CHANGE. CHANGE. >> INSERTED OPS AND SO FORTH. >> INSERTED OPS AND SO FORTH. >> INSERTED OPS AND SO FORTH. >> WITH THE VISUALIZER YOU CAN >> WITH THE VISUALIZER YOU CAN >> WITH THE VISUALIZER YOU CAN SEE HOW THE GRAPH CHANGES. SEE HOW THE GRAPH CHANGES. SEE HOW THE GRAPH CHANGES. CORRECTNESS IS REALLY TRICKY CORRECTNESS IS REALLY TRICKY CORRECTNESS IS REALLY TRICKY BECAUSE IN MY EXPERIENCE THE BECAUSE IN MY EXPERIENCE THE BECAUSE IN MY EXPERIENCE THE ONLY THING THAT REALLY WORKS IS ONLY THING THAT REALLY WORKS IS ONLY THING THAT REALLY WORKS IS TO REALLY EVALUATE UNDER REAL TO REALLY EVALUATE UNDER REAL TO REALLY EVALUATE UNDER REAL DATA THAT YOU CARE TO RUN YOUR DATA THAT YOU CARE TO RUN YOUR DATA THAT YOU CARE TO RUN YOUR MODEL ON. MODEL ON. MODEL ON. WE TRIED TO DO THINGS LIKE -- WE TRIED TO DO THINGS LIKE -- WE TRIED TO DO THINGS LIKE -- VERSUS THE FULL PRECISION VERSUS THE FULL PRECISION VERSUS THE FULL PRECISION VERSES THE QUANTIZED ONE. VERSES THE QUANTIZED ONE. VERSES THE QUANTIZED ONE. SOME CATASTROPHIC NUMERICAL SOME CATASTROPHIC NUMERICAL SOME CATASTROPHIC NUMERICAL ERRORS BUT OTHERWISE JUST A ERRORS BUT OTHERWISE JUST A ERRORS BUT OTHERWISE JUST A GUESS PARTICULARLY DEPENDING ON GUESS PARTICULARLY DEPENDING ON GUESS PARTICULARLY DEPENDING ON THE OUTPUT LAYERS. THE OUTPUT LAYERS. THE OUTPUT LAYERS. THEY ARE EASIER TO QUANTIZE. THEY ARE EASIER TO QUANTIZE. THEY ARE EASIER TO QUANTIZE. REGRESSIONS ARE HARDER BECAUSE REGRESSIONS ARE HARDER BECAUSE REGRESSIONS ARE HARDER BECAUSE YOU NOW CARE ABOUT THE ACTUAL YOU NOW CARE ABOUT THE ACTUAL YOU NOW CARE ABOUT THE ACTUAL VALUES. VALUES. VALUES. IT'S AN OPEN -- IT'S AN OPEN -- IT'S AN OPEN -- >> THANK YOU. >> THANK YOU. >> THANK YOU. >> I HAVE A QUESTION ABOUT THE >> I HAVE A QUESTION ABOUT THE >> I HAVE A QUESTION ABOUT THE RESULTS FROM THE GNT TRAINING RESULTS FROM THE GNT TRAINING RESULTS FROM THE GNT TRAINING WITH INDUCED SPARSETY. WITH INDUCED SPARSETY. WITH INDUCED SPARSETY. I WAS WONDERING IF YOU HAD I WAS WONDERING IF YOU HAD I WAS WONDERING IF YOU HAD INSIGHTS ON WHY THE TRAINING INSIGHTS ON WHY THE TRAINING INSIGHTS ON WHY THE TRAINING WITH 80% SPARSETY BETTER THAN WITH 80% SPARSETY BETTER THAN WITH 80% SPARSETY BETTER THAN THE ORIGINAL VERSION. THE ORIGINAL VERSION. THE ORIGINAL VERSION. >> IN THIS CASE IT IS SOUND >> IN THIS CASE IT IS SOUND >> IN THIS CASE IT IS SOUND REGULARIZATION HAPPENS. REGULARIZATION HAPPENS. REGULARIZATION HAPPENS. I'VE SEEN THE SAME ARE SOME I'VE SEEN THE SAME ARE SOME I'VE SEEN THE SAME ARE SOME QUANTIZED MODELS. QUANTIZED MODELS. QUANTIZED MODELS. I NEVER SAT DOWN TO UNDERSTAND I NEVER SAT DOWN TO UNDERSTAND I NEVER SAT DOWN TO UNDERSTAND WHY THE REASONS ARE. WHY THE REASONS ARE. WHY THE REASONS ARE. SOMETIMES IT IS JUST WITHIN THE SOMETIMES IT IS JUST WITHIN THE SOMETIMES IT IS JUST WITHIN THE NOISE. NOISE. NOISE. IT ALL DEPENDS ON THE IT ALL DEPENDS ON THE IT ALL DEPENDS ON THE EVALUATION SET, RIGHT? EVALUATION SET, RIGHT? EVALUATION SET, RIGHT? IF IT'S REALLY NOT THAT BIG OR IF IT'S REALLY NOT THAT BIG OR IF IT'S REALLY NOT THAT BIG OR NOT THAT MEANINGFUL THESE JOBS NOT THAT MEANINGFUL THESE JOBS NOT THAT MEANINGFUL THESE JOBS ARE AT ALL POSSIBLE. ARE AT ALL POSSIBLE. ARE AT ALL POSSIBLE. I'VE SEEN SOME MODELS WHERE IT I'VE SEEN SOME MODELS WHERE IT I'VE SEEN SOME MODELS WHERE IT LOOKS GREAT AFTER YOU QUANTIZE LOOKS GREAT AFTER YOU QUANTIZE LOOKS GREAT AFTER YOU QUANTIZE IT AND YOU THROW A NEW DATASET IT AND YOU THROW A NEW DATASET IT AND YOU THROW A NEW DATASET AND NOISE HERE AND THEN YOU AND NOISE HERE AND THEN YOU AND NOISE HERE AND THEN YOU CLEARLY SEE THE DIFFERENCE CLEARLY SEE THE DIFFERENCE CLEARLY SEE THE DIFFERENCE BETWEEN ONE AND THE OTHER. BETWEEN ONE AND THE OTHER. BETWEEN ONE AND THE OTHER. IT IS A LOT OF IT CAN BE JUST IT IS A LOT OF IT CAN BE JUST IT IS A LOT OF IT CAN BE JUST NOISE. NOISE. NOISE. >> YOU MENTIONED EXPLAINABILITY >> YOU MENTIONED EXPLAINABILITY >> YOU MENTIONED EXPLAINABILITY AND TECHNIQUE COULD BE LIKE -- AND TECHNIQUE COULD BE LIKE -- AND TECHNIQUE COULD BE LIKE -- DO YOU HAVE ANY INSIGHTS ON HOW DO YOU HAVE ANY INSIGHTS ON HOW DO YOU HAVE ANY INSIGHTS ON HOW THESE TECHNIQUES EFFECT THE THESE TECHNIQUES EFFECT THE THESE TECHNIQUES EFFECT THE ABILITY TO CALCULATE THE ABILITY TO CALCULATE THE ABILITY TO CALCULATE THE GRADIENTS TO CALCULATE THE -- GRADIENTS TO CALCULATE THE -- GRADIENTS TO CALCULATE THE -- >> TRYING TO UNDERSTAND THE >> TRYING TO UNDERSTAND THE >> TRYING TO UNDERSTAND THE NETWORKS AND TRYING TO NETWORKS AND TRYING TO NETWORKS AND TRYING TO UNDERSTAND THE NEURAL NETWORKS. UNDERSTAND THE NEURAL NETWORKS. UNDERSTAND THE NEURAL NETWORKS. SO FAR I HAVEN'T GOT ANY LUCK SO FAR I HAVEN'T GOT ANY LUCK SO FAR I HAVEN'T GOT ANY LUCK TRYING TO GET THE PEOPLE ON TRYING TO GET THE PEOPLE ON TRYING TO GET THE PEOPLE ON THAT SIDE EXCITED ABOUT IT. THAT SIDE EXCITED ABOUT IT. THAT SIDE EXCITED ABOUT IT. I REALLY DON'T HAVE ANY I REALLY DON'T HAVE ANY I REALLY DON'T HAVE ANY MEANINGFUL THING TO SAY BECAUSE MEANINGFUL THING TO SAY BECAUSE MEANINGFUL THING TO SAY BECAUSE I HAVEN'T RUN MANY EXPERIMENTS I HAVEN'T RUN MANY EXPERIMENTS I HAVEN'T RUN MANY EXPERIMENTS AROUND IT. AROUND IT. AROUND IT. >> SO WHAT IS THE BEST WAY TO >> SO WHAT IS THE BEST WAY TO >> SO WHAT IS THE BEST WAY TO HANDLE FRAGMENTATION OF HANDLE FRAGMENTATION OF HANDLE FRAGMENTATION OF HARDWARE AND MORE OFTEN THERE HARDWARE AND MORE OFTEN THERE HARDWARE AND MORE OFTEN THERE AE MOVING PARTS AND HAVE -- AE MOVING PARTS AND HAVE -- AE MOVING PARTS AND HAVE -- WHAT ARE THE BEST WAYS THERE? WHAT ARE THE BEST WAYS THERE? WHAT ARE THE BEST WAYS THERE? >> ONE WAY WE TRIED TO DO IT >> ONE WAY WE TRIED TO DO IT >> ONE WAY WE TRIED TO DO IT WAS SPECIFICATIONS. WAS SPECIFICATIONS. WAS SPECIFICATIONS. AND YOU KNOW LIKE I DON'T KNOW AND YOU KNOW LIKE I DON'T KNOW AND YOU KNOW LIKE I DON'T KNOW IF IT MAKES OUR HARDWARE IF IT MAKES OUR HARDWARE IF IT MAKES OUR HARDWARE PARTNERS HAPPY. PARTNERS HAPPY. PARTNERS HAPPY. WE WOULD LIKE TO TARGET THEIR WE WOULD LIKE TO TARGET THEIR WE WOULD LIKE TO TARGET THEIR HARDWARE IN THE MOST PRECISE HARDWARE IN THE MOST PRECISE HARDWARE IN THE MOST PRECISE AND EFFICIENT WAY BUT IT IS ONE AND EFFICIENT WAY BUT IT IS ONE AND EFFICIENT WAY BUT IT IS ONE WAY WE TRIED TO ADDRESS IT. WAY WE TRIED TO ADDRESS IT. WAY WE TRIED TO ADDRESS IT. YOU KNOW, WITH OUR KNOWLEDGE OF YOU KNOW, WITH OUR KNOWLEDGE OF YOU KNOW, WITH OUR KNOWLEDGE OF SOME HARDWARE IS THERE AND WHAT SOME HARDWARE IS THERE AND WHAT SOME HARDWARE IS THERE AND WHAT IS SUPPORTED WE TRIED TO CREATE IS SUPPORTED WE TRIED TO CREATE IS SUPPORTED WE TRIED TO CREATE SPECIFICATIONS FOR EVERYBODY. SPECIFICATIONS FOR EVERYBODY. SPECIFICATIONS FOR EVERYBODY. LONGER TERM AND I DON'T WANT TO LONGER TERM AND I DON'T WANT TO LONGER TERM AND I DON'T WANT TO SAY TOO MUCH BECAUSE I REALLY SAY TOO MUCH BECAUSE I REALLY SAY TOO MUCH BECAUSE I REALLY HAVEN'T -- I DON'T HAVE A VERY HAVEN'T -- I DON'T HAVE A VERY HAVEN'T -- I DON'T HAVE A VERY CONCRETE PLAN TO SHARE. CONCRETE PLAN TO SHARE. CONCRETE PLAN TO SHARE. PART OF WHAT WE'RE BUILDING PART OF WHAT WE'RE BUILDING PART OF WHAT WE'RE BUILDING WITH THE INFRASTRUCTURE WE WANT WITH THE INFRASTRUCTURE WE WANT WITH THE INFRASTRUCTURE WE WANT TO BE ABLE TO BETTER TARGET THE TO BE ABLE TO BETTER TARGET THE TO BE ABLE TO BETTER TARGET THE HARDWARE AND BETTER PARTNER HARDWARE AND BETTER PARTNER HARDWARE AND BETTER PARTNER WITH HARDWARE VENDORS TO WITH HARDWARE VENDORS TO WITH HARDWARE VENDORS TO UNDERSTAND THEIR CAPABILITIES UNDERSTAND THEIR CAPABILITIES UNDERSTAND THEIR CAPABILITIES AND CREATE THE TRANSFORMATIONS AND CREATE THE TRANSFORMATIONS AND CREATE THE TRANSFORMATIONS THAT TARGET THAT HARDWARE. THAT TARGET THAT HARDWARE. THAT TARGET THAT HARDWARE. WE ARE GOING TO MAKE BETTER. WE ARE GOING TO MAKE BETTER. WE ARE GOING TO MAKE BETTER. >> YOU GO WITH THE LESS COMMON >> YOU GO WITH THE LESS COMMON >> YOU GO WITH THE LESS COMMON DENOMINATOR. DENOMINATOR. DENOMINATOR. (?) (?) (?) >> WE HAVE ALL THE DIFFERENT >> WE HAVE ALL THE DIFFERENT >> WE HAVE ALL THE DIFFERENT ONES LIKE WE HAVE THREE TYPES. ONES LIKE WE HAVE THREE TYPES. ONES LIKE WE HAVE THREE TYPES. TWO HOPEFULLY WE'LL BE ABLE TO TWO HOPEFULLY WE'LL BE ABLE TO TWO HOPEFULLY WE'LL BE ABLE TO MIX AND MATCH THOSE DIFFERENT MIX AND MATCH THOSE DIFFERENT MIX AND MATCH THOSE DIFFERENT TYPES. TYPES. TYPES. AT THE END OF THE DAY IT'S A AT THE END OF THE DAY IT'S A AT THE END OF THE DAY IT'S A VERY ARBITRARY AND IT'S THIS VERY ARBITRARY AND IT'S THIS VERY ARBITRARY AND IT'S THIS ONE IS HYBRID AND WE SHOULD BE ONE IS HYBRID AND WE SHOULD BE ONE IS HYBRID AND WE SHOULD BE ABLE TO TAKE ADVANTAGE OF ABLE TO TAKE ADVANTAGE OF ABLE TO TAKE ADVANTAGE OF MIXING AND MATCHING UP TO GET MIXING AND MATCHING UP TO GET MIXING AND MATCHING UP TO GET SOMETHING BETTER. SOMETHING BETTER. SOMETHING BETTER. >> THANK YOU. >> THANK YOU. >> THANK YOU. >> I HAVE A QUESTION ABOUT >> I HAVE A QUESTION ABOUT >> I HAVE A QUESTION ABOUT PRUNING. PRUNING. PRUNING. AS A GENERAL RULE IN LAYERS AS A GENERAL RULE IN LAYERS AS A GENERAL RULE IN LAYERS THINGS OPERATIONS ARE CONVERTED THINGS OPERATIONS ARE CONVERTED THINGS OPERATIONS ARE CONVERTED TO MATRIX MULTIPLY BECAUSE OF TO MATRIX MULTIPLY BECAUSE OF TO MATRIX MULTIPLY BECAUSE OF THEIR EFFICIENCY. THEIR EFFICIENCY. THEIR EFFICIENCY. WITH PRUNING YOUR PASSING AN WITH PRUNING YOUR PASSING AN WITH PRUNING YOUR PASSING AN INDIVIDUAL MULTIPLE OPERATIONS INDIVIDUAL MULTIPLE OPERATIONS INDIVIDUAL MULTIPLE OPERATIONS ONE-BY-ONE. ONE-BY-ONE. ONE-BY-ONE. THERE MUST BE SOME CROSSOVER THERE MUST BE SOME CROSSOVER THERE MUST BE SOME CROSSOVER POINT WHICH YOU NEED TO PRUNE POINT WHICH YOU NEED TO PRUNE POINT WHICH YOU NEED TO PRUNE BY 10, 15, 20% BEFORE YOU ARE BY 10, 15, 20% BEFORE YOU ARE BY 10, 15, 20% BEFORE YOU ARE CROSSING OVER AND ACTUALLY GET CROSSING OVER AND ACTUALLY GET CROSSING OVER AND ACTUALLY GET AN IMPROVEMENT. AN IMPROVEMENT. AN IMPROVEMENT. THOUGHTS ON WHERE THAT IS? THOUGHTS ON WHERE THAT IS? THOUGHTS ON WHERE THAT IS? >> I DON'T KNOW IF THIS IS >> I DON'T KNOW IF THIS IS >> I DON'T KNOW IF THIS IS EXACTLY WHAT YOU'RE ASKING BUT EXACTLY WHAT YOU'RE ASKING BUT EXACTLY WHAT YOU'RE ASKING BUT OUR PRUNING API SUPPORTS DEFINE OUR PRUNING API SUPPORTS DEFINE OUR PRUNING API SUPPORTS DEFINE WHAT THE PRUNING STRUCTURE IS. WHAT THE PRUNING STRUCTURE IS. WHAT THE PRUNING STRUCTURE IS. FOR EXAMPLE, WE KNOW THAT FOR FOR EXAMPLE, WE KNOW THAT FOR FOR EXAMPLE, WE KNOW THAT FOR CPU, THE INSTRUCTIONS WE HAVE CPU, THE INSTRUCTIONS WE HAVE CPU, THE INSTRUCTIONS WE HAVE CAN ACCOMMODATE 16 VALUE. CAN ACCOMMODATE 16 VALUE. CAN ACCOMMODATE 16 VALUE. WE WANT TO WORK ON CPU WE WE WANT TO WORK ON CPU WE WE WANT TO WORK ON CPU WE EXPECT TO SET THE SETTINGS TO EXPECT TO SET THE SETTINGS TO EXPECT TO SET THE SETTINGS TO SAY I WANT TO PRUNE IN BLOCKS SAY I WANT TO PRUNE IN BLOCKS SAY I WANT TO PRUNE IN BLOCKS OF 1 BY 16 AND HOW WE CAN GET OF 1 BY 16 AND HOW WE CAN GET OF 1 BY 16 AND HOW WE CAN GET THE SPEED OF CPU FOR EXAMPLE. THE SPEED OF CPU FOR EXAMPLE. THE SPEED OF CPU FOR EXAMPLE. UNFORTUNATELY RIGHT NOW BROADLY UNFORTUNATELY RIGHT NOW BROADLY UNFORTUNATELY RIGHT NOW BROADLY WILL BE HARDWARE DEPENDENT. WILL BE HARDWARE DEPENDENT. WILL BE HARDWARE DEPENDENT. BUT THAT'S ONE THING THAT YOU BUT THAT'S ONE THING THAT YOU BUT THAT'S ONE THING THAT YOU CAN DO RIGHT NOW. CAN DO RIGHT NOW. CAN DO RIGHT NOW. >> IS THERE SOME SORT OF >> IS THERE SOME SORT OF >> IS THERE SOME SORT OF MINIMUM? MINIMUM? MINIMUM? >> WE'RE OUT OF TIME. >> WE'RE OUT OF TIME. >> WE'RE OUT OF TIME. I'M SORRY. I'M SORRY. I'M SORRY. I'M SURE YOU ALL CAN TALK. I'M SURE YOU ALL CAN TALK. I'M SURE YOU ALL CAN TALK. THANK YOU ALL SO MUCH FOR THANK YOU ALL SO MUCH FOR THANK YOU ALL SO MUCH FOR COMING AND -- COMING AND -- COMING AND -- [APPLAUSE] AND HOW IT WORKS. AND HOW IT WORKS. AND HOW IT WORKS. YOU SAW SIMPLE EXAMPLE OF YOU SAW SIMPLE EXAMPLE OF YOU SAW SIMPLE EXAMPLE OF MATCHING NUMBERS TO EACH OTHER MATCHING NUMBERS TO EACH OTHER MATCHING NUMBERS TO EACH OTHER AND HOW USING PYTHON CODE A AND HOW USING PYTHON CODE A AND HOW USING PYTHON CODE A COMPUTER COULD LEARN THROUGH COMPUTER COULD LEARN THROUGH COMPUTER COULD LEARN THROUGH TRIAL AND ERROR WHAT THE TRIAL AND ERROR WHAT THE TRIAL AND ERROR WHAT THE RELATIONSHIP BETWEEN THE NUMBERS RELATIONSHIP BETWEEN THE NUMBERS RELATIONSHIP BETWEEN THE NUMBERS WAS. WAS. WAS. IN THIS EPISODE YOU'RE GOING TO IN THIS EPISODE YOU'RE GOING TO IN THIS EPISODE YOU'RE GOING TO TAKE IT A LITTLE FURTHER BY TAKE IT A LITTLE FURTHER BY TAKE IT A LITTLE FURTHER BY TEACHING YOUR COMPUTER HOW TO TEACHING YOUR COMPUTER HOW TO TEACHING YOUR COMPUTER HOW TO SEE AND RECOGNIZE DIFFERENT SEE AND RECOGNIZE DIFFERENT SEE AND RECOGNIZE DIFFERENT OBJECTS. OBJECTS. OBJECTS. FOR EXAMPLE, LOOK AT THOSE FOR EXAMPLE, LOOK AT THOSE FOR EXAMPLE, LOOK AT THOSE PICTURES. PICTURES. PICTURES. HOW MANY SHOES DO YOU SEE? HOW MANY SHOES DO YOU SEE? HOW MANY SHOES DO YOU SEE? YOU MIGHT SAY TWO; RIGHT? YOU MIGHT SAY TWO; RIGHT? YOU MIGHT SAY TWO; RIGHT? BUT HOW DO YOU KNOW THEY ARE BUT HOW DO YOU KNOW THEY ARE BUT HOW DO YOU KNOW THEY ARE SHOES? SHOES? SHOES? IMAGINE IF SOMEBODY HAD NEVER IMAGINE IF SOMEBODY HAD NEVER IMAGINE IF SOMEBODY HAD NEVER SEEN SHOES BEFORE, HOW WOULD YOU SEEN SHOES BEFORE, HOW WOULD YOU SEEN SHOES BEFORE, HOW WOULD YOU TELL THEM THAT DESPITE THE GREAT TELL THEM THAT DESPITE THE GREAT TELL THEM THAT DESPITE THE GREAT DIFFERENCE BETWEEN THE HIGH HEEL DIFFERENCE BETWEEN THE HIGH HEEL DIFFERENCE BETWEEN THE HIGH HEEL AND THE SPORTS SHOE THEY'RE AND THE SPORTS SHOE THEY'RE AND THE SPORTS SHOE THEY'RE STILL BOTH SHOES? STILL BOTH SHOES? STILL BOTH SHOES? WHICH IF YOU REMEMBER WAS THE WHICH IF YOU REMEMBER WAS THE WHICH IF YOU REMEMBER WAS THE SIZE OF THE OUR IMAGE. SIZE OF THE OUR IMAGE. SIZE OF THE OUR IMAGE. OUR LAST LAYER IS 10 WHICH IF OUR LAST LAYER IS 10 WHICH IF OUR LAST LAYER IS 10 WHICH IF YOU REMEMBER IS THE NUMBER OF YOU REMEMBER IS THE NUMBER OF YOU REMEMBER IS THE NUMBER OF DIFFERENT 2 OF THEM CLOSING DIFFERENT 2 OF THEM CLOSING DIFFERENT 2 OF THEM CLOSING DEFINITE 2 OF THEMITEMS OF CLOS DEFINITE 2 OF THEMITEMS OF CLOS DEFINITE 2 OF THEMITEMS OF CLOS DEFINITE ITEMS OF FILTER WHICH DEFINITE ITEMS OF FILTER WHICH DEFINITE ITEMS OF FILTER WHICH AND OUTPUTS ONE OF THE VALUES. AND OUTPUTS ONE OF THE VALUES. AND OUTPUTS ONE OF THE VALUES. SO WHAT ABOUT THIS NUMBER, 128? SO WHAT ABOUT THIS NUMBER, 128? SO WHAT ABOUT THIS NUMBER, 128? WHAT DOES THAT DO? WHAT DOES THAT DO? WHAT DOES THAT DO? WELL, THINK OF IT LIKE THIS. WELL, THINK OF IT LIKE THIS. WELL, THINK OF IT LIKE THIS. WE'RE GOING TO HAVE 128 WE'RE GOING TO HAVE 128 WE'RE GOING TO HAVE 128 FUNCTIONS, EACH ONE OF WHICH HAS FUNCTIONS, EACH ONE OF WHICH HAS FUNCTIONS, EACH ONE OF WHICH HAS PARAMETERS INSIDE OF IT. PARAMETERS INSIDE OF IT. PARAMETERS INSIDE OF IT. LET'S CALL THESE F0 THROUGH LET'S CALL THESE F0 THROUGH LET'S CALL THESE F0 THROUGH F127. F127. F127. WHAT WE WANT IS THAT WHEN THE WHAT WE WANT IS THAT WHEN THE WHAT WE WANT IS THAT WHEN THE PIXELS OF THE SHOES GET FED INTO PIXELS OF THE SHOES GET FED INTO PIXELS OF THE SHOES GET FED INTO THEM, 1 BY 1, THAT THE THEM, 1 BY 1, THAT THE THEM, 1 BY 1, THAT THE COMBINATION OF ALL OF THESE COMBINATION OF ALL OF THESE COMBINATION OF ALL OF THESE FUNCTIONS WILL OUTPUT THE FUNCTIONS WILL OUTPUT THE FUNCTIONS WILL OUTPUT THE CORRECT VALUE IN THIS CASE 9. CORRECT VALUE IN THIS CASE 9. CORRECT VALUE IN THIS CASE 9. IN ORDER TO DO THAT THE COMPUTER IN ORDER TO DO THAT THE COMPUTER IN ORDER TO DO THAT THE COMPUTER WILL NEED TO FIGURE OUT THE WILL NEED TO FIGURE OUT THE WILL NEED TO FIGURE OUT THE PARAMETERS INSIDE THESE PARAMETERS INSIDE THESE PARAMETERS INSIDE THESE FUNCTIONS TO GET THESE RESULTS. FUNCTIONS TO GET THESE RESULTS. FUNCTIONS TO GET THESE RESULTS. IT WILL EXTEND IT TO ALL THE IT WILL EXTEND IT TO ALL THE IT WILL EXTEND IT TO ALL THE OTHER ITEMS OF COLORS IN THE -- OTHER ITEMS OF COLORS IN THE -- OTHER ITEMS OF COLORS IN THE -- >> WHAT SOFTMAX DOES IT SETS AT >> WHAT SOFTMAX DOES IT SETS AT >> WHAT SOFTMAX DOES IT SETS AT 1 AND THE REST IS ZERO SO ALL WE 1 AND THE REST IS ZERO SO ALL WE 1 AND THE REST IS ZERO SO ALL WE HAVE TO DO IS FIND THE 1. HAVE TO DO IS FIND THE 1. HAVE TO DO IS FIND THE 1. TRAINING IS VERY SEARCH. TRAINING IS VERY SEARCH. TRAINING IS VERY SEARCH. WE FIT THE IMAGES TO THE LABELS. WE FIT THE IMAGES TO THE LABELS. WE FIT THE IMAGES TO THE LABELS. THIS TIME WE'LL TRY IT FOR JUST THIS TIME WE'LL TRY IT FOR JUST THIS TIME WE'LL TRY IT FOR JUST 5 ePOX. 5 ePOX. 5 ePOX. REMEMBER EARLIER WE HAD 10,000 REMEMBER EARLIER WE HAD 10,000 REMEMBER EARLIER WE HAD 10,000 IMAGES AND LABELS THAT WE DIDN'T IMAGES AND LABELS THAT WE DIDN'T IMAGES AND LABELS THAT WE DIDN'T TRAIN WITH. TRAIN WITH. TRAIN WITH. THESE ARE IMAGES THAT THE MODEL THESE ARE IMAGES THAT THE MODEL THESE ARE IMAGES THAT THE MODEL HASN'T PREVIOUSLY SEEN SO WE CAN HASN'T PREVIOUSLY SEEN SO WE CAN HASN'T PREVIOUSLY SEEN SO WE CAN USE THEM TO TEST HOW WELL OUR USE THEM TO TEST HOW WELL OUR USE THEM TO TEST HOW WELL OUR MODEL PERFORMS. MODEL PERFORMS. MODEL PERFORMS. WE CAN DO THAT TEST BY PASSING WE CAN DO THAT TEST BY PASSING WE CAN DO THAT TEST BY PASSING THEM TO THE EVALUATE METHOD LIKE THEM TO THE EVALUATE METHOD LIKE THEM TO THE EVALUATE METHOD LIKE THIS. THIS. THIS. AND THEN FINALLY WE CAN GET AND THEN FINALLY WE CAN GET AND THEN FINALLY WE CAN GET PRICKLETIONS BACK FROM -- PRICKLETIONS BACK FROM -- PRICKLETIONS BACK FROM -- PREDICTIONS FOR MODEL PREDICT PREDICTIONS FOR MODEL PREDICT PREDICTIONS FOR MODEL PREDICT LIKE THIS, AND THAT'S IT TAKES LIKE THIS, AND THAT'S IT TAKES LIKE THIS, AND THAT'S IT TAKES TO TEACH A COMPUTER TO SEE AND TO TEACH A COMPUTER TO SEE AND TO TEACH A COMPUTER TO SEE AND RECOGNIZE IMAGES. RECOGNIZE IMAGES. RECOGNIZE IMAGES. YOU CAN TRY THIS OUT IN THE LINK YOU CAN TRY THIS OUT IN THE LINK YOU CAN TRY THIS OUT IN THE LINK THAT I DESCRIBED BEFORE. THAT I DESCRIBED BEFORE. THAT I DESCRIBED BEFORE. HAVING GO THROUGH THIS I ♪ HAVING GO THROUGH THIS I ♪ HAVING GO THROUGH THIS I ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ >> THROUGH HIS TECHNICAL >> THROUGH HIS TECHNICAL >> THROUGH HIS TECHNICAL DIFFICULTIES -- THROUGH THOSE DIFFICULTIES -- THROUGH THOSE DIFFICULTIES -- THROUGH THOSE TECHNICAL DIFFICULTIES. TECHNICAL DIFFICULTIES. TECHNICAL DIFFICULTIES. IT'S A PLEASURE TO BRING UP MY IT'S A PLEASURE TO BRING UP MY IT'S A PLEASURE TO BRING UP MY TENSORFLOW FLOW TEAM AT GOOGLE. TENSORFLOW FLOW TEAM AT GOOGLE. TENSORFLOW FLOW TEAM AT GOOGLE. THEY WILL TALK ABOUT THEY WILL TALK ABOUT THEY WILL TALK ABOUT TENSORFLOW.JS. TENSORFLOW.JS. TENSORFLOW.JS. >> THANK YOU. >> THANK YOU. >> THANK YOU. HELLO, EVERYONE. HELLO, EVERYONE. HELLO, EVERYONE. THANK YOU FOR YOUR PATIENCE. THANK YOU FOR YOUR PATIENCE. THANK YOU FOR YOUR PATIENCE. WE ARE SUPER EXCITED TO BE HERE WE ARE SUPER EXCITED TO BE HERE WE ARE SUPER EXCITED TO BE HERE AND TALKING TO YOU ABOUT AND TALKING TO YOU ABOUT AND TALKING TO YOU ABOUT TENSORFLOW JS AND TO BRING -- TENSORFLOW JS AND TO BRING -- TENSORFLOW JS AND TO BRING -- AND HOW TO BRING HERE'S MY AND HOW TO BRING HERE'S MY AND HOW TO BRING HERE'S MY COLLEAGUE WHO'S A SOFTWARE COLLEAGUE WHO'S A SOFTWARE COLLEAGUE WHO'S A SOFTWARE ENGINEERING ON THE TENSORFLOW ENGINEERING ON THE TENSORFLOW ENGINEERING ON THE TENSORFLOW PROGRAM. PROGRAM. PROGRAM. HERE'S AN OUTLINE OF TENSORFLOW HERE'S AN OUTLINE OF TENSORFLOW HERE'S AN OUTLINE OF TENSORFLOW JS. JS. JS. AND WE WILL DO A CODE SE LEVELS- AND WE WILL DO A CODE SE LEVELS- AND WE WILL DO A CODE SE LEVELS- WALK-THROUGH OF THE DEVELOPMENT WALK-THROUGH OF THE DEVELOPMENT WALK-THROUGH OF THE DEVELOPMENT BUT TO USE ONE OF THESE BUT TO USE ONE OF THESE BUT TO USE ONE OF THESE PRETRAINED MODELS AS WELL AS, PRETRAINED MODELS AS WELL AS, PRETRAINED MODELS AS WELL AS, YOU KNOW, AND WE'D LOVE TO SHOW YOU KNOW, AND WE'D LOVE TO SHOW YOU KNOW, AND WE'D LOVE TO SHOW YOU SOME EXAMPLES AND FINALLY YOU SOME EXAMPLES AND FINALLY YOU SOME EXAMPLES AND FINALLY WE'LL POINT TO SOME RESOURCES WE'LL POINT TO SOME RESOURCES WE'LL POINT TO SOME RESOURCES THAT YOU CAN START EXPLORING. THAT YOU CAN START EXPLORING. THAT YOU CAN START EXPLORING. ALL RIGHT. SO WE HAVE A LOT OF ALL RIGHT. SO WE HAVE A LOT OF ALL RIGHT. SO WE HAVE A LOT OF EXCITING CONTENT TO COVER SO EXCITING CONTENT TO COVER SO EXCITING CONTENT TO COVER SO LET'S GET STARTED. LET'S GET STARTED. LET'S GET STARTED. YOU MAY HAVE HEARD AT THE -- AT YOU MAY HAVE HEARD AT THE -- AT YOU MAY HAVE HEARD AT THE -- AT THE TENSORFLOW JS KEYNOTE AN THE TENSORFLOW JS KEYNOTE AN THE TENSORFLOW JS KEYNOTE AN OVERVIEW OF THE TECHNOLOGY IN OVERVIEW OF THE TECHNOLOGY IN OVERVIEW OF THE TECHNOLOGY IN THE MORNING WE'RE GOING TO BUILD THE MORNING WE'RE GOING TO BUILD THE MORNING WE'RE GOING TO BUILD ON THAT HERE BUT FIRST A QUICK ON THAT HERE BUT FIRST A QUICK ON THAT HERE BUT FIRST A QUICK RECAP OF THE BASICS OF RECAP OF THE BASICS OF RECAP OF THE BASICS OF TENSORFLOW JS. TENSORFLOW JS. TENSORFLOW JS. SO IN A NUTSHELL, TENSORFLOW.JS SO IN A NUTSHELL, TENSORFLOW.JS SO IN A NUTSHELL, TENSORFLOW.JS IS MACHINE LEARNING FOR JAVA IS MACHINE LEARNING FOR JAVA IS MACHINE LEARNING FOR JAVA SCRIPT. SCRIPT. SCRIPT. IT'S BUILT FOR JAVA SCRIPT IT'S BUILT FOR JAVA SCRIPT IT'S BUILT FOR JAVA SCRIPT BUILDERS TO CREATE AND RUN BUILDERS TO CREATE AND RUN BUILDERS TO CREATE AND RUN MISSION MACHINE LEARNING MODELS MISSION MACHINE LEARNING MODELS MISSION MACHINE LEARNING MODELS WITH JAVASCRIPT TRAINING YOU CAN WITH JAVASCRIPT TRAINING YOU CAN WITH JAVASCRIPT TRAINING YOU CAN HAVE BROWSER PLATFORMS AND -- HAVE BROWSER PLATFORMS AND -- HAVE BROWSER PLATFORMS AND -- [INAUDIBLE] [INAUDIBLE] [INAUDIBLE] >> ML OPERATIONS ARE GPU >> ML OPERATIONS ARE GPU >> ML OPERATIONS ARE GPU ACCELERATED AND THE LIBRARY IS ACCELERATED AND THE LIBRARY IS ACCELERATED AND THE LIBRARY IS FULLY OPEN SOURCE AND ANYONE IS FULLY OPEN SOURCE AND ANYONE IS FULLY OPEN SOURCE AND ANYONE IS WELCOME TO CONTRIBUTE. WELCOME TO CONTRIBUTE. WELCOME TO CONTRIBUTE. OKAY. OKAY. OKAY. SO TENSORFLOW JS HAS YOU CAN USE SO TENSORFLOW JS HAS YOU CAN USE SO TENSORFLOW JS HAS YOU CAN USE OFF THE SHELF MODELS THAT THE OFF THE SHELF MODELS THAT THE OFF THE SHELF MODELS THAT THE LIBRARY PROVIDES AND WE'LL SEE LIBRARY PROVIDES AND WE'LL SEE LIBRARY PROVIDES AND WE'LL SEE THOSE IN A BIT. THOSE IN A BIT. THOSE IN A BIT. WE CAN ALSO USE YOUR PYTHON WE CAN ALSO USE YOUR PYTHON WE CAN ALSO USE YOUR PYTHON MODELS WITH OUR WITHOUT PLATFORM MODELS WITH OUR WITHOUT PLATFORM MODELS WITH OUR WITHOUT PLATFORM CONVERSION WITH THE PLATFORM CONVERSION WITH THE PLATFORM CONVERSION WITH THE PLATFORM YOU'RE USING OR YOU CAN RETRAIN YOU'RE USING OR YOU CAN RETRAIN YOU'RE USING OR YOU CAN RETRAIN EXISTING MODEL FOR TRAINING AND EXISTING MODEL FOR TRAINING AND EXISTING MODEL FOR TRAINING AND CUSTOMIZING IT TO YOUR DATASETS. CUSTOMIZING IT TO YOUR DATASETS. CUSTOMIZING IT TO YOUR DATASETS. THAT'S THE SECOND, YOU KNOW, THAT'S THE SECOND, YOU KNOW, THAT'S THE SECOND, YOU KNOW, STARTING POINTED -- AND THE STARTING POINTED -- AND THE STARTING POINTED -- AND THE THIRD STARTING MODEL YOU CAN THIRD STARTING MODEL YOU CAN THIRD STARTING MODEL YOU CAN BUILD YOUR ENTIRE MODEL WITH A BUILD YOUR ENTIRE MODEL WITH A BUILD YOUR ENTIRE MODEL WITH A KE DEFINITE KERAS LASER PLANNING KE DEFINITE KERAS LASER PLANNING KE DEFINITE KERAS LASER PLANNING KERAS WITH MANY A PLATFORMS LIKE KERAS WITH MANY A PLATFORMS LIKE KERAS WITH MANY A PLATFORMS LIKE WHICH APP WE HAVE JUST ADDED WHICH APP WE HAVE JUST ADDED WHICH APP WE HAVE JUST ADDED FIRST CLASS INTEGRATE WITH FIRST CLASS INTEGRATE WITH FIRST CLASS INTEGRATE WITH TENSORFLOW JS ON SERVER JS RUNS TENSORFLOW JS ON SERVER JS RUNS TENSORFLOW JS ON SERVER JS RUNS ON NOTE AND IN ADDITION IT CAN ON NOTE AND IN ADDITION IT CAN ON NOTE AND IN ADDITION IT CAN RUN ON DESKTOP APPLICATIONS RUN ON DESKTOP APPLICATIONS RUN ON DESKTOP APPLICATIONS USING THE ELECTRON FRAMEWORK. USING THE ELECTRON FRAMEWORK. USING THE ELECTRON FRAMEWORK. SO WHY TENSORFLOW JS? SO WHY TENSORFLOW JS? SO WHY TENSORFLOW JS? WHY RUN ML IN THE BROWSER WELL, WHY RUN ML IN THE BROWSER WELL, WHY RUN ML IN THE BROWSER WELL, WE WILL BE THERE ARE COMPELLING WE WILL BE THERE ARE COMPELLING WE WILL BE THERE ARE COMPELLING REASONS TO RUN ML AUSTIN BROWSER REASONS TO RUN ML AUSTIN BROWSER REASONS TO RUN ML AUSTIN BROWSER CLIENT ESPECIALLY FOR MODEL CLIENT ESPECIALLY FOR MODEL CLIENT ESPECIALLY FOR MODEL INFERENCING. INFERENCING. INFERENCING. LET'S LOOK AT SOME OF THESE LET'S LOOK AT SOME OF THESE LET'S LOOK AT SOME OF THESE MODELS. MODELS. MODELS. FIRSTLY THERE ARE NO DRIVERS AND FIRSTLY THERE ARE NO DRIVERS AND FIRSTLY THERE ARE NO DRIVERS AND NOTHING TO INSTALL. NOTHING TO INSTALL. NOTHING TO INSTALL. INCLUDE THE TENSORFLOW JS INCLUDE THE TENSORFLOW JS INCLUDE THE TENSORFLOW JS LIBRARY BY SCRIPT SOURCING IT LIBRARY BY SCRIPT SOURCING IT LIBRARY BY SCRIPT SOURCING IT INTO YOUR HTML PAGE OR BY INTO YOUR HTML PAGE OR BY INTO YOUR HTML PAGE OR BY BUNDLING INTO YOUR PACKAGE BUNDLING INTO YOUR PACKAGE BUNDLING INTO YOUR PACKAGE MANAGER IN YOUR CLIENT APP AND MANAGER IN YOUR CLIENT APP AND MANAGER IN YOUR CLIENT APP AND YOU'RE GOOD TO GO, AND THAT'S YOU'RE GOOD TO GO, AND THAT'S YOU'RE GOOD TO GO, AND THAT'S IT. IT. IT. THE -- THE SECOND ADVANTAGE IS THE -- THE SECOND ADVANTAGE IS THE -- THE SECOND ADVANTAGE IS YOU CAN UTILIZE A 1R5IVARIETY O YOU CAN UTILIZE A 1R5IVARIETY O YOU CAN UTILIZE A 1R5IVARIETY O VARIETY OF INPUTS SUCH AS VARIETY OF INPUTS SUCH AS VARIETY OF INPUTS SUCH AS CAMERA, MICROPHONE, GPS THROUGH CAMERA, MICROPHONE, GPS THROUGH CAMERA, MICROPHONE, GPS THROUGH STANDARD API HTML WEB API AND A STANDARD API HTML WEB API AND A STANDARD API HTML WEB API AND A SIMPLIFIED DATA API AND WE'RE SIMPLIFIED DATA API AND WE'RE SIMPLIFIED DATA API AND WE'RE GOING TO SEE SOME EXAMPLES GOING TO SEE SOME EXAMPLES GOING TO SEE SOME EXAMPLES TODAY. TODAY. TODAY. WHICH IS A GREAT CHOICE FOR WHICH IS A GREAT CHOICE FOR WHICH IS A GREAT CHOICE FOR PRIVACY SENSITIVE APPLICATIONS. PRIVACY SENSITIVE APPLICATIONS. PRIVACY SENSITIVE APPLICATIONS. AVOIDS ROUND TRIP LATENCY TO THE AVOIDS ROUND TRIP LATENCY TO THE AVOIDS ROUND TRIP LATENCY TO THE SERVER. SERVER. SERVER. IT'S ACCELERATED THESE FACTORS IT'S ACCELERATED THESE FACTORS IT'S ACCELERATED THESE FACTORS IN COMBINATION, YOU IN COMBINATION, YOU IN COMBINATION, YOU KNOW -- COMBINE TO MAKE FOR A KNOW -- COMBINE TO MAKE FOR A KNOW -- COMBINE TO MAKE FOR A MORE FLUID AND INTERACTIVE USER MORE FLUID AND INTERACTIVE USER MORE FLUID AND INTERACTIVE USER EXPERIENCE. EXPERIENCE. EXPERIENCE. ALSO RUNNING ML ON THE CLIENT ALSO RUNNING ML ON THE CLIENT ALSO RUNNING ML ON THE CLIENT HELPS REDUCE SERVICE SIDE COSTS HELPS REDUCE SERVICE SIDE COSTS HELPS REDUCE SERVICE SIDE COSTS AND SERVICE YOUR INFRASTRUCTURE, AND SERVICE YOUR INFRASTRUCTURE, AND SERVICE YOUR INFRASTRUCTURE, FOR EXAMPLE, NO ONLINE ML FOR EXAMPLE, NO ONLINE ML FOR EXAMPLE, NO ONLINE ML SERVING THAT NEEDS TO SCALE TO SERVING THAT NEEDS TO SCALE TO SERVING THAT NEEDS TO SCALE TO INCREASING TRAFFIC AND SO FORTH INCREASING TRAFFIC AND SO FORTH INCREASING TRAFFIC AND SO FORTH IS NEEDED BECAUSE YOU'RE IS NEEDED BECAUSE YOU'RE IS NEEDED BECAUSE YOU'RE OFFLOADING ALL YOUR COMPUTING TO OFFLOADING ALL YOUR COMPUTING TO OFFLOADING ALL YOUR COMPUTING TO THE CLIENT YOU HOST AN ML MODEL THE CLIENT YOU HOST AN ML MODEL THE CLIENT YOU HOST AN ML MODEL FROM A STATIC FILE LOCATION AND FROM A STATIC FILE LOCATION AND FROM A STATIC FILE LOCATION AND THAT'S IT. THAT'S IT. THAT'S IT. ON THE SERVER THERE ARE ALSO ON THE SERVER THERE ARE ALSO ON THE SERVER THERE ARE ALSO BENEFITS TO INTEGRATING BENEFITS TO INTEGRATING BENEFITS TO INTEGRATING TENSORFLOW JS INTO YOUR NORD JS TENSORFLOW JS INTO YOUR NORD JS TENSORFLOW JS INTO YOUR NORD JS ENVIRONMENT. ENVIRONMENT. ENVIRONMENT. IF YOU ARE USING A NODE JS IF YOU ARE USING A NODE JS IF YOU ARE USING A NODE JS SERVICE STACK IT ALLOWS YOU TO SERVICE STACK IT ALLOWS YOU TO SERVICE STACK IT ALLOWS YOU TO GO IN A STACK INSTEAD OF A GO IN A STACK INSTEAD OF A GO IN A STACK INSTEAD OF A PYTHON-BASED STACK. PYTHON-BASED STACK. PYTHON-BASED STACK. YOU CAN USE YOUR CORE INTENSIVE YOU CAN USE YOUR CORE INTENSIVE YOU CAN USE YOUR CORE INTENSIVE MODELS TO BRING THEM INTO JS NOT MODELS TO BRING THEM INTO JS NOT MODELS TO BRING THEM INTO JS NOT JUST THE PREBUILT MODELS YOUR JUST THE PREBUILT MODELS YOUR JUST THE PREBUILT MODELS YOUR CUSTOM MODELS BUILT WITH PYTHON CUSTOM MODELS BUILT WITH PYTHON CUSTOM MODELS BUILT WITH PYTHON TENSORFLOW CAN BE CONVERTED AND TENSORFLOW CAN BE CONVERTED AND TENSORFLOW CAN BE CONVERTED AND IN AN UPCOMING RELEASE YOU DON'T IN AN UPCOMING RELEASE YOU DON'T IN AN UPCOMING RELEASE YOU DON'T NEED THE CONVERSION PROCESS YOU NEED THE CONVERSION PROCESS YOU NEED THE CONVERSION PROCESS YOU CAN JUST USE THEM DIRECTLY IN CAN JUST USE THEM DIRECTLY IN CAN JUST USE THEM DIRECTLY IN NODE, AND FINALLY YOU CAN DO ALL NODE, AND FINALLY YOU CAN DO ALL NODE, AND FINALLY YOU CAN DO ALL OF THIS WITHOUT SACRIFICING OF THIS WITHOUT SACRIFICING OF THIS WITHOUT SACRIFICING PERFORMANCE. PERFORMANCE. PERFORMANCE. YOU GET CPU AND GPU ACCELERATION YOU GET CPU AND GPU ACCELERATION YOU GET CPU AND GPU ACCELERATION WITH THE UNDERLYING TENSORFLOW C WITH THE UNDERLYING TENSORFLOW C WITH THE UNDERLYING TENSORFLOW C LIBRARY BECAUSE THAT'S WHAT NODE LIBRARY BECAUSE THAT'S WHAT NODE LIBRARY BECAUSE THAT'S WHAT NODE USES AND WE'RE ALSO WORKING ON USES AND WE'RE ALSO WORKING ON USES AND WE'RE ALSO WORKING ON GPU ACCELERATION VIA OPEN GL SO GPU ACCELERATION VIA OPEN GL SO GPU ACCELERATION VIA OPEN GL SO THAT REMOVES THE NEED DEPENDING THAT REMOVES THE NEED DEPENDING THAT REMOVES THE NEED DEPENDING ON CORD DRIVERS AS WELL SO ON CORD DRIVERS AS WELL SO ON CORD DRIVERS AS WELL SO EFFECTIVELY, YOU GET PERFORMANCE EFFECTIVELY, YOU GET PERFORMANCE EFFECTIVELY, YOU GET PERFORMANCE THAT'S SIMILAR TO THE PYTHON THAT'S SIMILAR TO THE PYTHON THAT'S SIMILAR TO THE PYTHON LIBRARY. LIBRARY. LIBRARY. SO -- SO THESE ATTRIBUTES OF SO -- SO THESE ATTRIBUTES OF SO -- SO THESE ATTRIBUTES OF THE LIBRARY ENABLE A VARIETY OF THE LIBRARY ENABLE A VARIETY OF THE LIBRARY ENABLE A VARIETY OF USE CASES ACROSS THE CLIENT USE CASES ACROSS THE CLIENT USE CASES ACROSS THE CLIENT SERVER SPECTRUM SO LET'S TAKE A SERVER SPECTRUM SO LET'S TAKE A SERVER SPECTRUM SO LET'S TAKE A LOOK AT SOME OF THOSE ON THE LOOK AT SOME OF THOSE ON THE LOOK AT SOME OF THOSE ON THE CLIENT SIDE IT ENABLES YOU TO CLIENT SIDE IT ENABLES YOU TO CLIENT SIDE IT ENABLES YOU TO BUILD FEATURES THAT NEED HIGHER BUILD FEATURES THAT NEED HIGHER BUILD FEATURES THAT NEED HIGHER END ACTIVITY AUGMENTING HIGHER END ACTIVITY AUGMENTING HIGHER END ACTIVITY AUGMENTING HIGHER INTERACTIONS, SPEECH RECOGNITION INTERACTIONS, SPEECH RECOGNITION INTERACTIONS, SPEECH RECOGNITION AND ACCESSIBILITY AND SO FORTH AND ACCESSIBILITY AND SO FORTH AND ACCESSIBILITY AND SO FORTH ON THE SERVER SIDE OF THE ON THE SERVER SIDE OF THE ON THE SERVER SIDE OF THE SPECTRUM IT LETS YOU, YOU KNOW, SPECTRUM IT LETS YOU, YOU KNOW, SPECTRUM IT LETS YOU, YOU KNOW, HAVE YOUR MORE TRADITIONAL ML HAVE YOUR MORE TRADITIONAL ML HAVE YOUR MORE TRADITIONAL ML PIPELINES THAT SOLVE ENTERPRISE PIPELINES THAT SOLVE ENTERPRISE PIPELINES THAT SOLVE ENTERPRISE LIKE USE CASES AND IN THE MIDDLE LIKE USE CASES AND IN THE MIDDLE LIKE USE CASES AND IN THE MIDDLE THAT CAN LIVE EITHER ON THE THAT CAN LIVE EITHER ON THE THAT CAN LIVE EITHER ON THE SILVER ON THE CLIENT SILVER ON THE CLIENT SILVER ON THE CLIENT APPLICATIONS THAT THAT DO APPLICATIONS THAT THAT DO APPLICATIONS THAT THAT DO SENTIMENT ANALYSIS, SENTIMENT ANALYSIS, SENTIMENT ANALYSIS, CONVERSATIONAL AI, ML ASSISTED CONVERSATIONAL AI, ML ASSISTED CONVERSATIONAL AI, ML ASSISTED CONTENT ASSISTANCE AND SO FORTH CONTENT ASSISTANCE AND SO FORTH CONTENT ASSISTANCE AND SO FORTH SO YOU GET THE FLEXIBILITY OF SO YOU GET THE FLEXIBILITY OF SO YOU GET THE FLEXIBILITY OF CHOOSING WHERE YOU WANT YOUR ML CHOOSING WHERE YOU WANT YOUR ML CHOOSING WHERE YOU WANT YOUR ML TO RUN ON THE CLIENT ON THE TO RUN ON THE CLIENT ON THE TO RUN ON THE CLIENT ON THE SILVER EITHER/OR. SILVER EITHER/OR. SILVER EITHER/OR. SO WHAT ABOUT YOUR USE CASE -- SO WHAT ABOUT YOUR USE CASE -- SO WHAT ABOUT YOUR USE CASE -- WHATEVER YOUR USE CASE IS WHATEVER YOUR USE CASE IS WHATEVER YOUR USE CASE IS TENSORFLOW JS IS READY TO BE TENSORFLOW JS IS READY TO BE TENSORFLOW JS IS READY TO BE LEVERAGED. LEVERAGED. LEVERAGED. SO WITH THAT INTRO I'D LIKE TO SO WITH THAT INTRO I'D LIKE TO SO WITH THAT INTRO I'D LIKE TO DELVE DEEPER INTO THE READY TO DELVE DEEPER INTO THE READY TO DELVE DEEPER INTO THE READY TO USE MODELS AVAILABLE IN USE MODELS AVAILABLE IN USE MODELS AVAILABLE IN TENSORFLOW JS. TENSORFLOW JS. TENSORFLOW JS. OUR COLLECTION OF MODELS HAVE OUR COLLECTION OF MODELS HAVE OUR COLLECTION OF MODELS HAVE GROWN AND IS GROWING TO ADDRESS GROWN AND IS GROWING TO ADDRESS GROWN AND IS GROWING TO ADDRESS THE USE CASES THAT WE JUST THE USE CASES THAT WE JUST THE USE CASES THAT WE JUST MENTIONED NAMELY IMAGE MENTIONED NAMELY IMAGE MENTIONED NAMELY IMAGE CLASSIFICATION FOR CLASSIFYING CLASSIFICATION FOR CLASSIFYING CLASSIFICATION FOR CLASSIFYING WHOLE IMAGES, DETECTING WHOLE IMAGES, DETECTING WHOLE IMAGES, DETECTING SEGMENTING OBJECTS AND OBJECT SEGMENTING OBJECTS AND OBJECT SEGMENTING OBJECTS AND OBJECT BOUNDARIES DETECTING THE HUMAN BOUNDARIES DETECTING THE HUMAN BOUNDARIES DETECTING THE HUMAN BODY AND ESTIMATING POSE BODY AND ESTIMATING POSE BODY AND ESTIMATING POSE RECOGNIZING SPEECH COMMANDS AND RECOGNIZING SPEECH COMMANDS AND RECOGNIZING SPEECH COMMANDS AND COMMON WORDS FROM AUDIO DATA AND COMMON WORDS FROM AUDIO DATA AND COMMON WORDS FROM AUDIO DATA AND TEXT MODELS FOR TEXT TEXT MODELS FOR TEXT TEXT MODELS FOR TEXT CLASSIFICATION DEDUCTION. CLASSIFICATION DEDUCTION. CLASSIFICATION DEDUCTION. YOU CAN EXPLORE ALL OF THESE YOU CAN EXPLORE ALL OF THESE YOU CAN EXPLORE ALL OF THESE MODELS TODAY ON GITHUB. MODELS TODAY ON GITHUB. MODELS TODAY ON GITHUB. YOU CAN USE THEM BY INSTALLING YOU CAN USE THEM BY INSTALLING YOU CAN USE THEM BY INSTALLING THEM WITH MPM OR BY DIRECTLY THEM WITH MPM OR BY DIRECTLY THEM WITH MPM OR BY DIRECTLY INCLUDING FROM OUR HOLSTERED INCLUDING FROM OUR HOLSTERED INCLUDING FROM OUR HOLSTERED SCRIPTS. SCRIPTS. SCRIPTS. LET'S DIVE INTO SOME OF THESE LET'S DIVE INTO SOME OF THESE LET'S DIVE INTO SOME OF THESE MODELS AND GET MORE DETAILS OR MODELS AND GET MORE DETAILS OR MODELS AND GET MORE DETAILS OR THESE. THESE. THESE. SO THIS IS THE POST NET MODEL IT SO THIS IS THE POST NET MODEL IT SO THIS IS THE POST NET MODEL IT DETECTS 17 LANDMARK POINTS ON DETECTS 17 LANDMARK POINTS ON DETECTS 17 LANDMARK POINTS ON THE HUMAN BODY. THE HUMAN BODY. THE HUMAN BODY. IT SUPPORTS BOTH SINGLE PERSON IT SUPPORTS BOTH SINGLE PERSON IT SUPPORTS BOTH SINGLE PERSON AND MULTIPERSON DETECTION, YOU AND MULTIPERSON DETECTION, YOU AND MULTIPERSON DETECTION, YOU KNOW, WITH AN IMAGE. KNOW, WITH AN IMAGE. KNOW, WITH AN IMAGE. NOW, THERE ARE MULTIPLE VERSIONS NOW, THERE ARE MULTIPLE VERSIONS NOW, THERE ARE MULTIPLE VERSIONS OF THIS MODEL, VERSIONS THAT ARE OF THIS MODEL, VERSIONS THAT ARE OF THIS MODEL, VERSIONS THAT ARE BACKED BY MOBILE NET AND THOSE BACKED BY MOBILE NET AND THOSE BACKED BY MOBILE NET AND THOSE BACKED BY RES.NET THAT PROVIDE BACKED BY RES.NET THAT PROVIDE BACKED BY RES.NET THAT PROVIDE OPTIONS FOR BALANCING ACCURACY OPTIONS FOR BALANCING ACCURACY OPTIONS FOR BALANCING ACCURACY VERSUS MODEL SIZE VERSUS LATENCY VERSUS MODEL SIZE VERSUS LATENCY VERSUS MODEL SIZE VERSUS LATENCY DEPENDING ON YOUR NEEDS AND IT DEPENDING ON YOUR NEEDS AND IT DEPENDING ON YOUR NEEDS AND IT ENABLES USE CASES LIKE GESTURE ENABLES USE CASES LIKE GESTURE ENABLES USE CASES LIKE GESTURE BASE OR TRACTION AUGMENTED BASE OR TRACTION AUGMENTED BASE OR TRACTION AUGMENTED REALITY, ANIMATION AND SO ON, REALITY, ANIMATION AND SO ON, REALITY, ANIMATION AND SO ON, THINGS THAT, YOU KNOW -- THAT THINGS THAT, YOU KNOW -- THAT THINGS THAT, YOU KNOW -- THAT ARE RUNNING ML ON THE CLIENT BUT ARE RUNNING ML ON THE CLIENT BUT ARE RUNNING ML ON THE CLIENT BUT YOU CAN EXPLORE A DEMO OF THIS YOU CAN EXPLORE A DEMO OF THIS YOU CAN EXPLORE A DEMO OF THIS PARTICULAR MODEL ON BOTH THE PARTICULAR MODEL ON BOTH THE PARTICULAR MODEL ON BOTH THE EXPO HALL. EXPO HALL. EXPO HALL. ANOTHER HUMAN MODEL IS THE BODY ANOTHER HUMAN MODEL IS THE BODY ANOTHER HUMAN MODEL IS THE BODY PICK MODEL WHICH ENABLES PERSON PICK MODEL WHICH ENABLES PERSON PICK MODEL WHICH ENABLES PERSON SEGMENTATION AGAIN BOTH SINGLE SEGMENTATION AGAIN BOTH SINGLE SEGMENTATION AGAIN BOTH SINGLE AND MULTIPLE PERSONS IMAGE. AND MULTIPLE PERSONS IMAGE. AND MULTIPLE PERSONS IMAGE. IT IDENTIFIES BODY PARTS SUCH AS IT IDENTIFIES BODY PARTS SUCH AS IT IDENTIFIES BODY PARTS SUCH AS LEFT ARM, RIGHT ARM, TORSO, LEFT LEFT ARM, RIGHT ARM, TORSO, LEFT LEFT ARM, RIGHT ARM, TORSO, LEFT RIGHT LEDS AND SO FORTH IT ALSO RIGHT LEDS AND SO FORTH IT ALSO RIGHT LEDS AND SO FORTH IT ALSO PROVIDES CONVENIENCE API TO PROVIDES CONVENIENCE API TO PROVIDES CONVENIENCE API TO SEGMENT AND ASK EACH BODY PART SEGMENT AND ASK EACH BODY PART SEGMENT AND ASK EACH BODY PART IN A DIFFERENT COLOR WHICH IS IN A DIFFERENT COLOR WHICH IS IN A DIFFERENT COLOR WHICH IS WHAT YOU'RE SEEING IN THIS WHAT YOU'RE SEEING IN THIS WHAT YOU'RE SEEING IN THIS PARTICULAR GIF AND IN TERMS OF PARTICULAR GIF AND IN TERMS OF PARTICULAR GIF AND IN TERMS OF USE CASES YOU CAN BE USED FOR USE CASES YOU CAN BE USED FOR USE CASES YOU CAN BE USED FOR THINGS LIKE BLURRING FACES IN AN THINGS LIKE BLURRING FACES IN AN THINGS LIKE BLURRING FACES IN AN IMAGE OR BLURRED THE BACKGROUND IMAGE OR BLURRED THE BACKGROUND IMAGE OR BLURRED THE BACKGROUND TO PROTECT PRIVACY. TO PROTECT PRIVACY. TO PROTECT PRIVACY. THE SSID DETECTION MODEL ENABLES THE SSID DETECTION MODEL ENABLES THE SSID DETECTION MODEL ENABLES DETECTION OF MULTIPLE OBJECTS IN DETECTION OF MULTIPLE OBJECTS IN DETECTION OF MULTIPLE OBJECTS IN AN IMAGE. AN IMAGE. AN IMAGE. IT DETECTS ABOUT 90 CLASSES TO IT DETECTS ABOUT 90 CLASSES TO IT DETECTS ABOUT 90 CLASSES TO FIND IN THE DATASET AND THE NICE FIND IN THE DATASET AND THE NICE FIND IN THE DATASET AND THE NICE THING IS THAT IT TAKES THIS THING IS THAT IT TAKES THIS THING IS THAT IT TAKES THIS INPUT -- ANY BROWSER-BASED -- INPUT -- ANY BROWSER-BASED -- INPUT -- ANY BROWSER-BASED -- LIKE AN IMAGE OR VIDEO OR A LIKE AN IMAGE OR VIDEO OR A LIKE AN IMAGE OR VIDEO OR A CANVAS ELEMENT AND RETURNS A CANVAS ELEMENT AND RETURNS A CANVAS ELEMENT AND RETURNS A SCENARIO WITH THE DETECTOR CLASS SCENARIO WITH THE DETECTOR CLASS SCENARIO WITH THE DETECTOR CLASS AND THE ADVANCED LEVEL. AND THE ADVANCED LEVEL. AND THE ADVANCED LEVEL. IN THIS SAMPLE IMAGE YOU CAN SEE IN THIS SAMPLE IMAGE YOU CAN SEE IN THIS SAMPLE IMAGE YOU CAN SEE IT'S DETECTED WITH A HIGH IT'S DETECTED WITH A HIGH IT'S DETECTED WITH A HIGH CONFIDENCE SCORE AND YOU GET A CONFIDENCE SCORE AND YOU GET A CONFIDENCE SCORE AND YOU GET A BOUNDING BOX. BOUNDING BOX. BOUNDING BOX. THERE'S ALSO A DEEP LAB MODEL THERE'S ALSO A DEEP LAB MODEL THERE'S ALSO A DEEP LAB MODEL WHICH HAS SEMANICS SEGMENTATION WHICH HAS SEMANICS SEGMENTATION WHICH HAS SEMANICS SEGMENTATION THAT WILL BE COMING OUT SOON. THAT WILL BE COMING OUT SOON. THAT WILL BE COMING OUT SOON. WHAT I WOULD LIKE TO SHOW YOU WHAT I WOULD LIKE TO SHOW YOU WHAT I WOULD LIKE TO SHOW YOU HERE IS HOW EASY IT IS TO USE A HERE IS HOW EASY IT IS TO USE A HERE IS HOW EASY IT IS TO USE A MODEL LIKE COCOA SSD IN JAVA MODEL LIKE COCOA SSD IN JAVA MODEL LIKE COCOA SSD IN JAVA APPS AND YOU CAN MPM INSTALL AND APPS AND YOU CAN MPM INSTALL AND APPS AND YOU CAN MPM INSTALL AND USE A WEBPACK TO BUNDLE INTO A USE A WEBPACK TO BUNDLE INTO A USE A WEBPACK TO BUNDLE INTO A CLIENT FILE AND THEN YOU LOAD CLIENT FILE AND THEN YOU LOAD CLIENT FILE AND THEN YOU LOAD THE MODEL THAT'S THE DOT LOAD THE MODEL THAT'S THE DOT LOAD THE MODEL THAT'S THE DOT LOAD METHOD AND YOU -- AND YOU CALL METHOD AND YOU -- AND YOU CALL METHOD AND YOU -- AND YOU CALL THE MODEL METHOD ON -- ON THE THE MODEL METHOD ON -- ON THE THE MODEL METHOD ON -- ON THE IMAGE ELEMENT THAT YOU ARE IMAGE ELEMENT THAT YOU ARE IMAGE ELEMENT THAT YOU ARE TRYING TO ANALYZE AND THAT'S IT. TRYING TO ANALYZE AND THAT'S IT. TRYING TO ANALYZE AND THAT'S IT. WHAT YOU GET BACK IS A -- IS AN WHAT YOU GET BACK IS A -- IS AN WHAT YOU GET BACK IS A -- IS AN AREA OF OBJECTS THAT THAT THE AREA OF OBJECTS THAT THAT THE AREA OF OBJECTS THAT THAT THE BOUNDING BOXES AND THE CLASSES BOUNDING BOXES AND THE CLASSES BOUNDING BOXES AND THE CLASSES THAT ARE DETECTED SO REALLY 4 OR THAT ARE DETECTED SO REALLY 4 OR THAT ARE DETECTED SO REALLY 4 OR 5 LINES OF CODE TO END UP 5 LINES OF CODE TO END UP 5 LINES OF CODE TO END UP LEVERAGING A POWERFUL ML MODEL LEVERAGING A POWERFUL ML MODEL LEVERAGING A POWERFUL ML MODEL IN THE BROWSER. IN THE BROWSER. IN THE BROWSER. TEXT MODELS ARE ANOTHER USEFUL TEXT MODELS ARE ANOTHER USEFUL TEXT MODELS ARE ANOTHER USEFUL SET. SET. SET. MORE USER GENERATED SENSING D A- MORE USER GENERATED SENSING D A- MORE USER GENERATED SENSING D A- CODER MODEL AND LET'S USE THIS A CODER MODEL AND LET'S USE THIS A CODER MODEL AND LET'S USE THIS A LITTLE CLOSER. LITTLE CLOSER. LITTLE CLOSER. I WANT TO SHOW YOU A LIVE I WANT TO SHOW YOU A LIVE I WANT TO SHOW YOU A LIVE EXAMPLE OF THIS MODEL IN A WEB EXAMPLE OF THIS MODEL IN A WEB EXAMPLE OF THIS MODEL IN A WEB APP GIVE ME A SECOND. APP GIVE ME A SECOND. APP GIVE ME A SECOND. OKAY. OKAY. OKAY. THERE YOU GO. THERE YOU GO. THERE YOU GO. SO HERE IS A SUPER SIMPLE WEB SO HERE IS A SUPER SIMPLE WEB SO HERE IS A SUPER SIMPLE WEB APP THAT SIMPLY LOADS THE MODEL APP THAT SIMPLY LOADS THE MODEL APP THAT SIMPLY LOADS THE MODEL AND, YOU KNOW, PASSES A FEW AND, YOU KNOW, PASSES A FEW AND, YOU KNOW, PASSES A FEW SENTENCES TO IT AND WHAT THE SENTENCES TO IT AND WHAT THE SENTENCES TO IT AND WHAT THE MODEL DOES IS CLASSIFIED ON A MODEL DOES IS CLASSIFIED ON A MODEL DOES IS CLASSIFIED ON A FEW DIMENSIONS LIKE -- DOES IT FEW DIMENSIONS LIKE -- DOES IT FEW DIMENSIONS LIKE -- DOES IT SIGNIFY AN INSULT. SIGNIFY AN INSULT. SIGNIFY AN INSULT. IS THERE A TOXICITY OR -- SO IS THERE A TOXICITY OR -- SO IS THERE A TOXICITY OR -- SO LET'S TRY AN EXAMPLE HERE. LET'S TRY AN EXAMPLE HERE. LET'S TRY AN EXAMPLE HERE. SO I'M GOING TO SAY SOMETHING SO I'M GOING TO SAY SOMETHING SO I'M GOING TO SAY SOMETHING TOXIC. TOXIC. TOXIC. TYPING TYPING OKAY. TYPING TYPING OKAY. TYPING TYPING OKAY. SO SOMETHING THAT I WOULD THINK SO SOMETHING THAT I WOULD THINK SO SOMETHING THAT I WOULD THINK IS TOXIC SO THAT RETURNS AS IS TOXIC SO THAT RETURNS AS IS TOXIC SO THAT RETURNS AS TOXIC AS WELL AS REDUNDANCE AND TOXIC AS WELL AS REDUNDANCE AND TOXIC AS WELL AS REDUNDANCE AND INSULT HOWEVER THIS MODEL I WANT INSULT HOWEVER THIS MODEL I WANT INSULT HOWEVER THIS MODEL I WANT TO SHOW YOU THIS IS TO SHOW YOU THIS IS TO SHOW YOU THIS IS CONTEXT-BASED AND NOT KEYWORD CONTEXT-BASED AND NOT KEYWORD CONTEXT-BASED AND NOT KEYWORD BASED SO, YOU KNOW, YOU TYPE THE BASED SO, YOU KNOW, YOU TYPE THE BASED SO, YOU KNOW, YOU TYPE THE SAME WORD IN A -- IN A TOTALLY SAME WORD IN A -- IN A TOTALLY SAME WORD IN A -- IN A TOTALLY NONTOXIC CONCEPT -- CONTEXT NONTOXIC CONCEPT -- CONTEXT NONTOXIC CONCEPT -- CONTEXT YOU'RE GOING TO SEE A DIFFERENT YOU'RE GOING TO SEE A DIFFERENT YOU'RE GOING TO SEE A DIFFERENT ANSWER. ANSWER. ANSWER. AND THAT'S DETECTED AS, YOU AND THAT'S DETECTED AS, YOU AND THAT'S DETECTED AS, YOU KNOW, NOT AN INSULT SO THIS SORT KNOW, NOT AN INSULT SO THIS SORT KNOW, NOT AN INSULT SO THIS SORT OF MODEL CAN BE USED IN -- ON OF MODEL CAN BE USED IN -- ON OF MODEL CAN BE USED IN -- ON THE CLIENT SIDE IN PRODUCT THE CLIENT SIDE IN PRODUCT THE CLIENT SIDE IN PRODUCT REVIEWS OR IN CHAT TYPE OF REVIEWS OR IN CHAT TYPE OF REVIEWS OR IN CHAT TYPE OF HERE'S AN EXAMPLE I WANT TO SHOW HERE'S AN EXAMPLE I WANT TO SHOW HERE'S AN EXAMPLE I WANT TO SHOW YOU FROM A DEVELOPER WHO YOU FROM A DEVELOPER WHO YOU FROM A DEVELOPER WHO INTEGRATED TENSORFLOW JS INTO A INTEGRATED TENSORFLOW JS INTO A INTEGRATED TENSORFLOW JS INTO A CHAT APP AND, AGAIN, HERE IT'S CHAT APP AND, AGAIN, HERE IT'S CHAT APP AND, AGAIN, HERE IT'S ABLE TO DETECT THAT, YOU KNOW, ABLE TO DETECT THAT, YOU KNOW, ABLE TO DETECT THAT, YOU KNOW, TOXIC COMMENTS AND TWITTERED TOXIC COMMENTS AND TWITTERED TOXIC COMMENTS AND TWITTERED THEM RIGHT BEFORE SENDING. THEM RIGHT BEFORE SENDING. THEM RIGHT BEFORE SENDING. ALL RIGHT. ALL RIGHT. ALL RIGHT. SLIDE. SLIDE. SLIDE. ANOTHER INTERESTING MODEL IS ANOTHER INTERESTING MODEL IS ANOTHER INTERESTING MODEL IS FACE MESH IT TRACKING FACIAL FACE MESH IT TRACKING FACIAL FACE MESH IT TRACKING FACIAL FEATURES IT TRACKS 400 POINTS ON FEATURES IT TRACKS 400 POINTS ON FEATURES IT TRACKS 400 POINTS ON A APPEARANCE FACE SO WE BELIEVE A APPEARANCE FACE SO WE BELIEVE A APPEARANCE FACE SO WE BELIEVE THIS MODEL HAS GREAT POTENTIAL THIS MODEL HAS GREAT POTENTIAL THIS MODEL HAS GREAT POTENTIAL FOR REAL WORLD APPLICATIONS, FOR FOR REAL WORLD APPLICATIONS, FOR FOR REAL WORLD APPLICATIONS, FOR EXAMPLE, DETECTING FACIAL JES EXAMPLE, DETECTING FACIAL JES EXAMPLE, DETECTING FACIAL JES TOURS, EMOTION, RESEARCHING TOURS, EMOTION, RESEARCHING TOURS, EMOTION, RESEARCHING GREATER ACCESSIBILITY FEATURES GREATER ACCESSIBILITY FEATURES GREATER ACCESSIBILITY FEATURES AND SO FORTH SO NOW I WOULD LIKE AND SO FORTH SO NOW I WOULD LIKE AND SO FORTH SO NOW I WOULD LIKE TO SHOW YOU A COUPLE OF COOL TO SHOW YOU A COUPLE OF COOL TO SHOW YOU A COUPLE OF COOL DEMOS BOTH USING FACE MEASURE. DEMOS BOTH USING FACE MEASURE. DEMOS BOTH USING FACE MEASURE. FIRST UP -- AND YOU MIGHT HAVE FIRST UP -- AND YOU MIGHT HAVE FIRST UP -- AND YOU MIGHT HAVE SEEN THIS IN THE KEYNOTE SESSION SEEN THIS IN THE KEYNOTE SESSION SEEN THIS IN THE KEYNOTE SESSION IF YOU ATTENDED HERE'S AN IF YOU ATTENDED HERE'S AN IF YOU ATTENDED HERE'S AN APPLICATION BUILT BY ONE OF OUR APPLICATION BUILT BY ONE OF OUR APPLICATION BUILT BY ONE OF OUR PARTNER TEAMS AT GOOGLE. PARTNER TEAMS AT GOOGLE. PARTNER TEAMS AT GOOGLE. THIS APP IS CALLED LIP SYNC THIS APP IS CALLED LIP SYNC THIS APP IS CALLED LIP SYNC SINK. SINK. SINK. THIS IS A GAME THAT TRACKS HOW THIS IS A GAME THAT TRACKS HOW THIS IS A GAME THAT TRACKS HOW WELL YOU'RE LIP SYNC SINKING TOO WELL YOU'RE LIP SYNC SINKING TOO WELL YOU'RE LIP SYNC SINKING TOO SONG WHILE IN THE BROWSER. SONG WHILE IN THE BROWSER. SONG WHILE IN THE BROWSER. NOTICE IN PARTICULAR HOW THE NOTICE IN PARTICULAR HOW THE NOTICE IN PARTICULAR HOW THE DISPLAY CREATES WHEN THE LIP DISPLAY CREATES WHEN THE LIP DISPLAY CREATES WHEN THE LIP SYNC SINK DOESN'T MATCH AND HOW SYNC SINK DOESN'T MATCH AND HOW SYNC SINK DOESN'T MATCH AND HOW THE SCORE MATCHES HOW WELL THE THE SCORE MATCHES HOW WELL THE THE SCORE MATCHES HOW WELL THE PERSON IS LIP SYNC SIN. PERSON IS LIP SYNC SIN. PERSON IS LIP SYNC SIN. THERE YOU GO. THERE YOU GO. THERE YOU GO. I'M GOING TO USE MY MIC -- MOVE I'M GOING TO USE MY MIC -- MOVE I'M GOING TO USE MY MIC -- MOVE MY MIC HERE. MY MIC HERE. MY MIC HERE. OKAY. OKAY. OKAY. THERE'S NO AUDIO HERE. THERE'S NO AUDIO HERE. THERE'S NO AUDIO HERE. WE DON'T HAVE AUDIO SO YOU MIGHT WE DON'T HAVE AUDIO SO YOU MIGHT WE DON'T HAVE AUDIO SO YOU MIGHT IMAGINE IF YOU RECALL SO WHEN IMAGINE IF YOU RECALL SO WHEN IMAGINE IF YOU RECALL SO WHEN THE LIP SYNCING WHEN THE PERSON THE LIP SYNCING WHEN THE PERSON THE LIP SYNCING WHEN THE PERSON IS NOT LIP SYNC CORRECTLY IS NOT LIP SYNC CORRECTLY IS NOT LIP SYNC CORRECTLY THERE'S FEEDBACK GIVEN BACK TO THERE'S FEEDBACK GIVEN BACK TO THERE'S FEEDBACK GIVEN BACK TO THE USER, AND THIS YOU CAN SEE THE USER, AND THIS YOU CAN SEE THE USER, AND THIS YOU CAN SEE IS -- IS A TYPE OF EXAMPLE THAT, IS -- IS A TYPE OF EXAMPLE THAT, IS -- IS A TYPE OF EXAMPLE THAT, YOU KNOW, ONE CAN BUILD ENTIRELY YOU KNOW, ONE CAN BUILD ENTIRELY YOU KNOW, ONE CAN BUILD ENTIRELY ON THE CLIENT USING THIS AND LET ON THE CLIENT USING THIS AND LET ON THE CLIENT USING THIS AND LET YOUR IMAGINATION, YOU KNOW, GO YOUR IMAGINATION, YOU KNOW, GO YOUR IMAGINATION, YOU KNOW, GO FROM THERE. FROM THERE. FROM THERE. SO AGAIN THIS IS AVAILABLE AT SO AGAIN THIS IS AVAILABLE AT SO AGAIN THIS IS AVAILABLE AT THE TENSORFLOW JS BOOTH FEEL THE TENSORFLOW JS BOOTH FEEL THE TENSORFLOW JS BOOTH FEEL FREE TO TRY IT OUT AND SEE HOW FREE TO TRY IT OUT AND SEE HOW FREE TO TRY IT OUT AND SEE HOW YOU DO. YOU DO. YOU DO. THE NEXT APPLICATION WE WOULD THE NEXT APPLICATION WE WOULD THE NEXT APPLICATION WE WOULD LIKE TO SHOW USE A TRY ON APP LIKE TO SHOW USE A TRY ON APP LIKE TO SHOW USE A TRY ON APP WHICH IS A FACE. WHICH IS A FACE. WHICH IS A FACE. THIS IS WHAT THEY HAVE PUT ON THIS IS WHAT THEY HAVE PUT ON THIS IS WHAT THEY HAVE PUT ON THE PLATFORM I WOULD LIKE THE PLATFORM I WOULD LIKE THE PLATFORM I WOULD LIKE BRANDON TO COME ON STAGE AND BRANDON TO COME ON STAGE AND BRANDON TO COME ON STAGE AND SHOW YOU HOW THEY USE TENSORFLOW SHOW YOU HOW THEY USE TENSORFLOW SHOW YOU HOW THEY USE TENSORFLOW JS TO BUILD THIS APP. JS TO BUILD THIS APP. JS TO BUILD THIS APP. [APPLAUSE] [APPLAUSE] [APPLAUSE] >> THANKS, SO HI I'M BRENDON I'M >> THANKS, SO HI I'M BRENDON I'M >> THANKS, SO HI I'M BRENDON I'M RESEARCH SCIENTISTS AT MODDAFACE RESEARCH SCIENTISTS AT MODDAFACE RESEARCH SCIENTISTS AT MODDAFACE AND FIRST LET ME BRIEFLY AND FIRST LET ME BRIEFLY AND FIRST LET ME BRIEFLY INTRODUCE MODDAFACE. INTRODUCE MODDAFACE. INTRODUCE MODDAFACE. MODDAFACE IS A AUGMENTED REALITY MODDAFACE IS A AUGMENTED REALITY MODDAFACE IS A AUGMENTED REALITY FOR A BEAUTY COMPANY. FOR A BEAUTY COMPANY. FOR A BEAUTY COMPANY. IT WAS FOUNDED IN 2007 IN IT WAS FOUNDED IN 2007 IN IT WAS FOUNDED IN 2007 IN TORONTO, CANADA, AND ACQUIRED TORONTO, CANADA, AND ACQUIRED TORONTO, CANADA, AND ACQUIRED JUST LAST YEAR 100% BY L'OREAL. JUST LAST YEAR 100% BY L'OREAL. JUST LAST YEAR 100% BY L'OREAL. SO TODAY MODDAFACE COLLABORATES SO TODAY MODDAFACE COLLABORATES SO TODAY MODDAFACE COLLABORATES BEAUTY BRANDS L'OREAL PARIS AND BEAUTY BRANDS L'OREAL PARIS AND BEAUTY BRANDS L'OREAL PARIS AND MAYBE LINE. MAYBE LINE. MAYBE LINE. YOU CAN FIND OUR TECHNOLOGY ON YOU CAN FIND OUR TECHNOLOGY ON YOU CAN FIND OUR TECHNOLOGY ON SUCH ONLINE RETAIL GIANTS SUCH SUCH ONLINE RETAIL GIANTS SUCH SUCH ONLINE RETAIL GIANTS SUCH AS MACY, SEPHORA AND AMAZON. AS MACY, SEPHORA AND AMAZON. AS MACY, SEPHORA AND AMAZON. NOW I'M GOING TO TALK ABOUT HOW NOW I'M GOING TO TALK ABOUT HOW NOW I'M GOING TO TALK ABOUT HOW WE MAKE USE OF TENSORFLOW JS IN WE MAKE USE OF TENSORFLOW JS IN WE MAKE USE OF TENSORFLOW JS IN OUR VIRTUAL TRY-ON APPLICATIONS. OUR VIRTUAL TRY-ON APPLICATIONS. OUR VIRTUAL TRY-ON APPLICATIONS. SO IN ORDER TO INTRODUCE WHY WE SO IN ORDER TO INTRODUCE WHY WE SO IN ORDER TO INTRODUCE WHY WE NEED A FRAMEWORK LIKE TENSORFLOW NEED A FRAMEWORK LIKE TENSORFLOW NEED A FRAMEWORK LIKE TENSORFLOW JS I'M GOING TO USE WE CHAT MINI JS I'M GOING TO USE WE CHAT MINI JS I'M GOING TO USE WE CHAT MINI PROGRAM FOR MAKEUP VIRTUAL TRION PROGRAM FOR MAKEUP VIRTUAL TRION PROGRAM FOR MAKEUP VIRTUAL TRION TO SHOWCASE THE KIND OF TO SHOWCASE THE KIND OF TO SHOWCASE THE KIND OF CHALLENGES YOU'RE DEPLOYING CHALLENGES YOU'RE DEPLOYING CHALLENGES YOU'RE DEPLOYING VIRTUAL TRI-ON SYSTEMS. VIRTUAL TRI-ON SYSTEMS. VIRTUAL TRI-ON SYSTEMS. SECOND OF ALL WE WANTED TO PLAY SECOND OF ALL WE WANTED TO PLAY SECOND OF ALL WE WANTED TO PLAY OUR APPLICATIONS ON THE CLIENT OUR APPLICATIONS ON THE CLIENT OUR APPLICATIONS ON THE CLIENT WE WANTED -- USER USER PRIVACY WE WANTED -- USER USER PRIVACY WE WANTED -- USER USER PRIVACY TO AVOID THE LATENCY FROM DOING TO AVOID THE LATENCY FROM DOING TO AVOID THE LATENCY FROM DOING A ROUND TRIP TO BACK-END SERVER A ROUND TRIP TO BACK-END SERVER A ROUND TRIP TO BACK-END SERVER EVERY FRAME AND ALSO WE HAVE A EVERY FRAME AND ALSO WE HAVE A EVERY FRAME AND ALSO WE HAVE A SOFTWARE REQUIREMENT OF ABOUT 10 SOFTWARE REQUIREMENT OF ABOUT 10 SOFTWARE REQUIREMENT OF ABOUT 10 FRAME PER SECOND FRAME RATE IN FRAME PER SECOND FRAME RATE IN FRAME PER SECOND FRAME RATE IN ORDER FOR OUR OBLIGATIONS TO ORDER FOR OUR OBLIGATIONS TO ORDER FOR OUR OBLIGATIONS TO FEEL INTERACTIVE. FEEL INTERACTIVE. FEEL INTERACTIVE. SO THIS IS PARTICULARLY SO THIS IS PARTICULARLY SO THIS IS PARTICULARLY DIFFICULT ON THE WE CHAT DIFFICULT ON THE WE CHAT DIFFICULT ON THE WE CHAT PLATFORM BECAUSE YOU HAVE -- YOU PLATFORM BECAUSE YOU HAVE -- YOU PLATFORM BECAUSE YOU HAVE -- YOU RUN YOUR -- YOU RUN YOUR MINI RUN YOUR -- YOU RUN YOUR MINI RUN YOUR -- YOU RUN YOUR MINI PROGRAM IN WE CHAT JAVA SCRIPT PROGRAM IN WE CHAT JAVA SCRIPT PROGRAM IN WE CHAT JAVA SCRIPT ENVIRONMENT AND YOU INHERIT THE ENVIRONMENT AND YOU INHERIT THE ENVIRONMENT AND YOU INHERIT THE LIMITATIONS OF JAVA SCRIPT SO WE LIMITATIONS OF JAVA SCRIPT SO WE LIMITATIONS OF JAVA SCRIPT SO WE NEEDED TO DEVELOP A LIGHTWEIGHT NEEDED TO DEVELOP A LIGHTWEIGHT NEEDED TO DEVELOP A LIGHTWEIGHT SPARTAN MODEL FOR FACE TRACKING SPARTAN MODEL FOR FACE TRACKING SPARTAN MODEL FOR FACE TRACKING AND ALSO TO FIND A FRAMEWORK AND ALSO TO FIND A FRAMEWORK AND ALSO TO FIND A FRAMEWORK THAT RUNS QUICKLY IN JAVA SCRIPT THAT RUNS QUICKLY IN JAVA SCRIPT THAT RUNS QUICKLY IN JAVA SCRIPT AND IDEALLY MAKES SENSE OF WEB AND IDEALLY MAKES SENSE OF WEB AND IDEALLY MAKES SENSE OF WEB GL TO RUN -- TO RUN OPERATORS GL TO RUN -- TO RUN OPERATORS GL TO RUN -- TO RUN OPERATORS USING THE GPU HARBOR USING THE GPU HARBOR USING THE GPU HARBOR ACCELERATION. ACCELERATION. ACCELERATION. SO SECOND -- THE SECOND SO SECOND -- THE SECOND SO SECOND -- THE SECOND CHALLENGE WE RAN INTO THE WE CHALLENGE WE RAN INTO THE WE CHALLENGE WE RAN INTO THE WE CHAT -- WE CHAT MINI PROGRAMS CHAT -- WE CHAT MINI PROGRAMS CHAT -- WE CHAT MINI PROGRAMS HAVE A MINI -- IT HAS TO BE HAVE A MINI -- IT HAS TO BE HAVE A MINI -- IT HAS TO BE SMALL AND ALLOWS US TO DEVELOP A SMALL AND ALLOWS US TO DEVELOP A SMALL AND ALLOWS US TO DEVELOP A SMALL MODEL AND SO WE CAN LOAD SMALL MODEL AND SO WE CAN LOAD SMALL MODEL AND SO WE CAN LOAD IT AS QUICKLY AS POSSIBLE AS IT AS QUICKLY AS POSSIBLE AS IT AS QUICKLY AS POSSIBLE AS WELL. WELL. WELL. AND THIRDLY, WE -- BECAUSE OUR AND THIRDLY, WE -- BECAUSE OUR AND THIRDLY, WE -- BECAUSE OUR MODELS HAVE SOME CUSTOM MODELS HAVE SOME CUSTOM MODELS HAVE SOME CUSTOM OPERATORS WE NEEDED A FRAMEWORK OPERATORS WE NEEDED A FRAMEWORK OPERATORS WE NEEDED A FRAMEWORK THAT'S ON HIS TENSIBLE WITH THAT'S ON HIS TENSIBLE WITH THAT'S ON HIS TENSIBLE WITH CUSTOM OPERATORS AND WE ALSO CUSTOM OPERATORS AND WE ALSO CUSTOM OPERATORS AND WE ALSO NEEDED A FRAMEWORK THAT IS GOING NEEDED A FRAMEWORK THAT IS GOING NEEDED A FRAMEWORK THAT IS GOING TO SUPPORT ALL THE DIFFERENT TO SUPPORT ALL THE DIFFERENT TO SUPPORT ALL THE DIFFERENT MOBILE PHONES THAT ARE SUPPORTED MOBILE PHONES THAT ARE SUPPORTED MOBILE PHONES THAT ARE SUPPORTED BY -- MOBILE PHONE MODELS THAT BY -- MOBILE PHONE MODELS THAT BY -- MOBILE PHONE MODELS THAT ARE SUPPORTED BY WE CHAT ITSELF. ARE SUPPORTED BY WE CHAT ITSELF. ARE SUPPORTED BY WE CHAT ITSELF. SO NOW I'M GOING TO TELL YOU HOW SO NOW I'M GOING TO TELL YOU HOW SO NOW I'M GOING TO TELL YOU HOW TENSORFLOW JS FIT THE BILL AND TENSORFLOW JS FIT THE BILL AND TENSORFLOW JS FIT THE BILL AND WAS ABLE TO OVERCOME SOME OF WAS ABLE TO OVERCOME SOME OF WAS ABLE TO OVERCOME SOME OF THESE CHALLENGES THAT WE RAN THESE CHALLENGES THAT WE RAN THESE CHALLENGES THAT WE RAN INTO IN DEPLOYING OUR MAKEUP TRY INTO IN DEPLOYING OUR MAKEUP TRY INTO IN DEPLOYING OUR MAKEUP TRY ON. ON. ON. FIRST EXPO JS RUNS ON THE CLIENT FIRST EXPO JS RUNS ON THE CLIENT FIRST EXPO JS RUNS ON THE CLIENT SIDE BECAUSE IT MAKES SENSE OF SIDE BECAUSE IT MAKES SENSE OF SIDE BECAUSE IT MAKES SENSE OF WEB GL BACK-END IT'S ABLE TO WEB GL BACK-END IT'S ABLE TO WEB GL BACK-END IT'S ABLE TO HARNESS THE MOBILE PHONE GPU HARNESS THE MOBILE PHONE GPU HARNESS THE MOBILE PHONE GPU HARBOR ACCELERATION WHICH GIVES HARBOR ACCELERATION WHICH GIVES HARBOR ACCELERATION WHICH GIVES IT LIKE AN ORDER OF MAKING IT LIKE AN ORDER OF MAKING IT LIKE AN ORDER OF MAKING NEWSPAPER SPEEDUP OVER BROWSER NEWSPAPER SPEEDUP OVER BROWSER NEWSPAPER SPEEDUP OVER BROWSER SUCH AS WEB ASSEMBLY WHICH ARE SUCH AS WEB ASSEMBLY WHICH ARE SUCH AS WEB ASSEMBLY WHICH ARE LIMITED RIGHT NOW BY LACK OF LIMITED RIGHT NOW BY LACK OF LIMITED RIGHT NOW BY LACK OF MULTITHREADING. MULTITHREADING. MULTITHREADING. SO IT REALLY ALLOWS US TO FRAME SO IT REALLY ALLOWS US TO FRAME SO IT REALLY ALLOWS US TO FRAME REQUIREMENT AND SECOND OF ALL REQUIREMENT AND SECOND OF ALL REQUIREMENT AND SECOND OF ALL THE TENSORFLOW JS LIBRARY IS THE TENSORFLOW JS LIBRARY IS THE TENSORFLOW JS LIBRARY IS SMALL 700 KILOBYTES AND COMBINED SMALL 700 KILOBYTES AND COMBINED SMALL 700 KILOBYTES AND COMBINED WITH OUR 400 KILOBITE MODEL WITH OUR 400 KILOBITE MODEL WITH OUR 400 KILOBITE MODEL SIZES WE WERE ABLE TO FIT SIZES WE WERE ABLE TO FIT SIZES WE WERE ABLE TO FIT EVERYTHING WITHIN THE WE CHAT EVERYTHING WITHIN THE WE CHAT EVERYTHING WITHIN THE WE CHAT MINI PROGRAM FILE SIZE LIMIT. MINI PROGRAM FILE SIZE LIMIT. MINI PROGRAM FILE SIZE LIMIT. AND THIRDLY, TENSORFLOW JS BOTH AND THIRDLY, TENSORFLOW JS BOTH AND THIRDLY, TENSORFLOW JS BOTH HAS WIDESPREAD SUPPORT FOR HAS WIDESPREAD SUPPORT FOR HAS WIDESPREAD SUPPORT FOR BUILT-IN DEEP LEARNING OPERATORS BUILT-IN DEEP LEARNING OPERATORS BUILT-IN DEEP LEARNING OPERATORS AND ALSO ALLOWED TO US EXTEND IT AND ALSO ALLOWED TO US EXTEND IT AND ALSO ALLOWED TO US EXTEND IT WITH OUR CUSTOM OPERATORS SO WE WITH OUR CUSTOM OPERATORS SO WE WITH OUR CUSTOM OPERATORS SO WE NEED FOR OUR FACE TRACK AND NEED FOR OUR FACE TRACK AND NEED FOR OUR FACE TRACK AND FINALLY TENSORFLOW JS JS IS FINALLY TENSORFLOW JS JS IS FINALLY TENSORFLOW JS JS IS SUPPORTING A WIDE VARIETY OF SUPPORTING A WIDE VARIETY OF SUPPORTING A WIDE VARIETY OF MICROPHONE MODELS AND HAS MICROPHONE MODELS AND HAS MICROPHONE MODELS AND HAS CONTINUOUS SUPPORT FROM THE CONTINUOUS SUPPORT FROM THE CONTINUOUS SUPPORT FROM THE TENSORFLOW JS AND WE CHOSE THAT TENSORFLOW JS AND WE CHOSE THAT TENSORFLOW JS AND WE CHOSE THAT AS THE FRAMEWORK TO DEPLOY ON AS THE FRAMEWORK TO DEPLOY ON AS THE FRAMEWORK TO DEPLOY ON THE WE CHAT PLUG-IN. THE WE CHAT PLUG-IN. THE WE CHAT PLUG-IN. SO NOW I'D LIKE TO SHARE WITH SO NOW I'D LIKE TO SHARE WITH SO NOW I'D LIKE TO SHARE WITH YOU SOME OF OUR RESULTS. YOU SOME OF OUR RESULTS. YOU SOME OF OUR RESULTS. SO WITH THE FAITH OF TENSORFLOW SO WITH THE FAITH OF TENSORFLOW SO WITH THE FAITH OF TENSORFLOW JS WE WERE ABLE TO SUCCESSFULLY JS WE WERE ABLE TO SUCCESSFULLY JS WE WERE ABLE TO SUCCESSFULLY DEPLOY OUR -- DEPLOY TO WE CHAT DEPLOY OUR -- DEPLOY TO WE CHAT DEPLOY OUR -- DEPLOY TO WE CHAT OUR EASY TO USE REAL TIME SYSTEM OUR EASY TO USE REAL TIME SYSTEM OUR EASY TO USE REAL TIME SYSTEM FOR REALISTIC AR VIRTUAL TRY-ON FOR REALISTIC AR VIRTUAL TRY-ON FOR REALISTIC AR VIRTUAL TRY-ON AND OUR ENTIRE FINAL SOLUTION AND OUR ENTIRE FINAL SOLUTION AND OUR ENTIRE FINAL SOLUTION FIT INTO THE 1.8 MEGABITES FIT INTO THE 1.8 MEGABITES FIT INTO THE 1.8 MEGABITES INCLUDING THE CODE AND THE INCLUDING THE CODE AND THE INCLUDING THE CODE AND THE MODELS AND ON AN iPHONE 8S AND MODELS AND ON AN iPHONE 8S AND MODELS AND ON AN iPHONE 8S AND TRACKING IS 25 FRAMES PER TRACKING IS 25 FRAMES PER TRACKING IS 25 FRAMES PER SECOND. SECOND. SECOND. SO TENSORFLOW JS COUPLED WITH SO TENSORFLOW JS COUPLED WITH SO TENSORFLOW JS COUPLED WITH OUR TINY CNN FACE TRACKING MODEL OUR TINY CNN FACE TRACKING MODEL OUR TINY CNN FACE TRACKING MODEL ALLOWED US TO DEPLOY TO WEB -- ALLOWED US TO DEPLOY TO WEB -- ALLOWED US TO DEPLOY TO WEB -- OUR FASTEST, SMALLEST MAKEUP OUR FASTEST, SMALLEST MAKEUP OUR FASTEST, SMALLEST MAKEUP VIRTUAL TRY-ON APP TO DATE. VIRTUAL TRY-ON APP TO DATE. VIRTUAL TRY-ON APP TO DATE. SO NOW I'D LIKE TO BRIEFLY SO NOW I'D LIKE TO BRIEFLY SO NOW I'D LIKE TO BRIEFLY MENTION WHAT'S GOING ON. MENTION WHAT'S GOING ON. MENTION WHAT'S GOING ON. W'VE ALREADY USED TENSORFLOW JS W'VE ALREADY USED TENSORFLOW JS W'VE ALREADY USED TENSORFLOW JS TO CREATE WEB APPLICATION DEMOS TO CREATE WEB APPLICATION DEMOS TO CREATE WEB APPLICATION DEMOS FOR OUR MAKEUP -- MAKEUP TRY-ON, FOR OUR MAKEUP -- MAKEUP TRY-ON, FOR OUR MAKEUP -- MAKEUP TRY-ON, OUR HAIR COLOR TRY-ON AND NAIL OUR HAIR COLOR TRY-ON AND NAIL OUR HAIR COLOR TRY-ON AND NAIL TRY ON AND WE WERE ABLE TO TRY ON AND WE WERE ABLE TO TRY ON AND WE WERE ABLE TO ACHIEVE AN ORDER OF MAGNITUDE ACHIEVE AN ORDER OF MAGNITUDE ACHIEVE AN ORDER OF MAGNITUDE SPEED UP FOR A GUIDED OPERATOR SPEED UP FOR A GUIDED OPERATOR SPEED UP FOR A GUIDED OPERATOR BY TAKING OUR WEB ASSEMBLY BY TAKING OUR WEB ASSEMBLY BY TAKING OUR WEB ASSEMBLY IMPLEMENTION AND REIMPLEMENTING IMPLEMENTION AND REIMPLEMENTING IMPLEMENTION AND REIMPLEMENTING IT ON TENSORFLOW JS THIS IT ON TENSORFLOW JS THIS IT ON TENSORFLOW JS THIS AMOUNTED TO ABOUT A 20 AMOUNTED TO ABOUT A 20 AMOUNTED TO ABOUT A 20 MILLISECOND LATENT SI FOR THE MILLISECOND LATENT SI FOR THE MILLISECOND LATENT SI FOR THE WHOLE SYSTEM AND WE HAVE A WHOLE SYSTEM AND WE HAVE A WHOLE SYSTEM AND WE HAVE A NUMBER OF OTHER PRODUCTS GOING NUMBER OF OTHER PRODUCTS GOING NUMBER OF OTHER PRODUCTS GOING ON AT MODDAFACE, VIRTUAL AGING ON AT MODDAFACE, VIRTUAL AGING ON AT MODDAFACE, VIRTUAL AGING SIMULATION AND SKIN ANALYSIS SIMULATION AND SKIN ANALYSIS SIMULATION AND SKIN ANALYSIS THAT WE HOPE TO USE TENSORFLOW THAT WE HOPE TO USE TENSORFLOW THAT WE HOPE TO USE TENSORFLOW JS TO DEPLOY IN THE NEAR FUTURE. JS TO DEPLOY IN THE NEAR FUTURE. JS TO DEPLOY IN THE NEAR FUTURE. SO THANK YOU FOR EVERYONE FOR SO THANK YOU FOR EVERYONE FOR SO THANK YOU FOR EVERYONE FOR LISTENING AND I'LL PASS THE LISTENING AND I'LL PASS THE LISTENING AND I'LL PASS THE PRESENTATION ON. PRESENTATION ON. PRESENTATION ON. [APPLAUSE] [APPLAUSE] [APPLAUSE] AND I WANT TO SHOW YOU DETAILS AND I WANT TO SHOW YOU DETAILS AND I WANT TO SHOW YOU DETAILS WHICH USE AUGMENTED WHICH USE AUGMENTED WHICH USE AUGMENTED >> THROUGH THE CAMERA AND WEB >> THROUGH THE CAMERA AND WEB >> THROUGH THE CAMERA AND WEB PAGE THERE ARE FIRST WE NEED A PAGE THERE ARE FIRST WE NEED A PAGE THERE ARE FIRST WE NEED A MODEL THAT IS TRAINED TO FIND MODEL THAT IS TRAINED TO FIND MODEL THAT IS TRAINED TO FIND THE FACE. THE FACE. THE FACE. HEO ON AA WITH LAMENT. LAMENT. WE WERE SURPRISED HOW IT SCALES. WE WERE SURPRISED HOW IT SCALES. WE WERE SURPRISED HOW IT SCALES. 5,000 NOTES. 5,000 NOTES. 5,000 NOTES. >> THIS WAS THE FIRST TIME >> THIS WAS THE FIRST TIME >> THIS WAS THE FIRST TIME ANYBODY HAS RUN AN API ANYBODY HAS RUN AN API ANYBODY HAS RUN AN API APPLICATION AT THIS SCALE APPLICATION AT THIS SCALE APPLICATION AT THIS SCALE INSTEAD OF HAVING THE CLIMATE INSTEAD OF HAVING THE CLIMATE INSTEAD OF HAVING THE CLIMATE SCIENTISTS HOW TO WRITE HIGH SCIENTISTS HOW TO WRITE HIGH SCIENTISTS HOW TO WRITE HIGH TUNED CODE THEY COULD WRITE? TUNED CODE THEY COULD WRITE? TUNED CODE THEY COULD WRITE? -- GET ALL THE HIGH PERFORMANCE -- GET ALL THE HIGH PERFORMANCE -- GET ALL THE HIGH PERFORMANCE CODE MOST PEOPLE ARE USED TO CODE MOST PEOPLE ARE USED TO CODE MOST PEOPLE ARE USED TO WITHIN TENSORFLOW. WITHIN TENSORFLOW. WITHIN TENSORFLOW. >> WE'RE ENTERING THE SPACE >> WE'RE ENTERING THE SPACE >> WE'RE ENTERING THE SPACE WHERE IT CAN CONTRADICT TO THE WHERE IT CAN CONTRADICT TO THE WHERE IT CAN CONTRADICT TO THE EXTREME WEATHER EVENTS. EXTREME WEATHER EVENTS. EXTREME WEATHER EVENTS. >> WHEN YOU COMBINE >> WHEN YOU COMBINE >> WHEN YOU COMBINE TRADITIONALLY WITH AI FUSION TRADITIONALLY WITH AI FUSION TRADITIONALLY WITH AI FUSION REACTOR RESEARCH UNDERSTANDING REACTOR RESEARCH UNDERSTANDING REACTOR RESEARCH UNDERSTANDING DISEASES LIKE ALZHEIMER'S, DISEASES LIKE ALZHEIMER'S, DISEASES LIKE ALZHEIMER'S, CANCER, RIGHT, THAT'S LIKE CANCER, RIGHT, THAT'S LIKE CANCER, RIGHT, THAT'S LIKE INCREDIBLE. INCREDIBLE. INCREDIBLE. >> WE'VE SHOWN WITH A HIGH >> WE'VE SHOWN WITH A HIGH >> WE'VE SHOWN WITH A HIGH PRODUCTIVITY YOU CAN GET AWESOME PRODUCTIVITY YOU CAN GET AWESOME PRODUCTIVITY YOU CAN GET AWESOME PERFORMANCE AND ACCOMPLISH YOUR PERFORMANCE AND ACCOMPLISH YOUR PERFORMANCE AND ACCOMPLISH YOUR GOALS. GOALS. GOALS. >> GENETICS, COMESMOLOGIST, HIGH >> GENETICS, COMESMOLOGIST, HIGH >> GENETICS, COMESMOLOGIST, HIGH ENERGY PHYSICS THAT'S IMMENSELY ENERGY PHYSICS THAT'S IMMENSELY ENERGY PHYSICS THAT'S IMMENSELY EXCITING FOR ME. EXCITING FOR ME. EXCITING FOR ME. ♪ WORK A LOT ON HOW DO YOU DO DEEP WORK A LOT ON HOW DO YOU DO DEEP WORK A LOT ON HOW DO YOU DO DEEP NETWORKS AND BUILD, YOU KNOW, NETWORKS AND BUILD, YOU KNOW, NETWORKS AND BUILD, YOU KNOW, MACHINE LEARNING SYSTEMS THAT MACHINE LEARNING SYSTEMS THAT MACHINE LEARNING SYSTEMS THAT SCALE ON THE CLOUD WITH MINIMAL SCALE ON THE CLOUD WITH MINIMAL SCALE ON THE CLOUD WITH MINIMAL SUPERVISION. SUPERVISION. SUPERVISION. WE WORK IN LANGUAGE WE WORK IN LANGUAGE WE WORK IN LANGUAGE UNDERSTANDING MULTI-MODAL AM UNDERSTANDING MULTI-MODAL AM UNDERSTANDING MULTI-MODAL AM KEGS. KEGS. KEGS. HOW DO YOU TAKE ALL THESE ALGA HOW DO YOU TAKE ALL THESE ALGA HOW DO YOU TAKE ALL THESE ALGA RIT IMS AND COMPUTE THEM ON THE RIT IMS AND COMPUTE THEM ON THE RIT IMS AND COMPUTE THEM ON THE EDGE. EDGE. EDGE. I'M HERE TODAY WITH MY I'M HERE TODAY WITH MY I'M HERE TODAY WITH MY COLLEAGUE? COLLEAGUE? COLLEAGUE? >> I'M WORKING WITH SUJI AND ALL >> I'M WORKING WITH SUJI AND ALL >> I'M WORKING WITH SUJI AND ALL THE RELATED TOPICS. THE RELATED TOPICS. THE RELATED TOPICS. >> LET'S GET STARTED. >> LET'S GET STARTED. >> LET'S GET STARTED. YOU HAVE THE HONOR OF BEING HERE YOU HAVE THE HONOR OF BEING HERE YOU HAVE THE HONOR OF BEING HERE FOR THE LAST SESSION OF THE LAST FOR THE LAST SESSION OF THE LAST FOR THE LAST SESSION OF THE LAST DAY. DAY. DAY. KUDOS, AND BRAVO. KUDOS, AND BRAVO. KUDOS, AND BRAVO. I MEAN, YOU ARE REALLY I MEAN, YOU ARE REALLY I MEAN, YOU ARE REALLY DEDICATED. DEDICATED. DEDICATED. LET US BEGIN. LET US BEGIN. LET US BEGIN. WE ARE VERY EXCITED TO TALK TO WE ARE VERY EXCITED TO TALK TO WE ARE VERY EXCITED TO TALK TO YOU ABOUT NOODLE-STRUCTURED YOU ABOUT NOODLE-STRUCTURED YOU ABOUT NOODLE-STRUCTURED LEARNING, WHICH IS A NEW LEARNING, WHICH IS A NEW LEARNING, WHICH IS A NEW FRAMEWORK INTENSIVE FLOW THAT FRAMEWORK INTENSIVE FLOW THAT FRAMEWORK INTENSIVE FLOW THAT ALLOWS YOU TO TRAIN NOODLED ALLOWS YOU TO TRAIN NOODLED ALLOWS YOU TO TRAIN NOODLED FRAMEWORKS. FRAMEWORKS. FRAMEWORKS. FIRST, LET'S GO OVER SOME FIRST, LET'S GO OVER SOME FIRST, LET'S GO OVER SOME BASICS. BASICS. BASICS. IF YOU ARE IN THIS ROOM, AND YOU IF YOU ARE IN THIS ROOM, AND YOU IF YOU ARE IN THIS ROOM, AND YOU KNOW ABOUT DEEP LEARNING, AND KNOW ABOUT DEEP LEARNING, AND KNOW ABOUT DEEP LEARNING, AND YOU CARE DEEPLY ABOUT DEEP YOU CARE DEEPLY ABOUT DEEP YOU CARE DEEPLY ABOUT DEEP LEARNING YOU KNOW HOW TYPICAL LEARNING YOU KNOW HOW TYPICAL LEARNING YOU KNOW HOW TYPICAL NEURAL NETWORKS WORK. NEURAL NETWORKS WORK. NEURAL NETWORKS WORK. IF YOU WANT TO TAKE A NEURAL IF YOU WANT TO TAKE A NEURAL IF YOU WANT TO TAKE A NEURAL NETWORK AND TRAIN IT TO NETWORK AND TRAIN IT TO NETWORK AND TRAIN IT TO RECOGNIZE IMAGES AND DISTINGUISH RECOGNIZE IMAGES AND DISTINGUISH RECOGNIZE IMAGES AND DISTINGUISH BETWEEN CONCEPTS LIKE CATS AND BETWEEN CONCEPTS LIKE CATS AND BETWEEN CONCEPTS LIKE CATS AND DOGS, WHAT WOULD YOU DO? DOGS, WHAT WOULD YOU DO? DOGS, WHAT WOULD YOU DO? YOU FEED IMAGES ON THE LEFT SIDE YOU FEED IMAGES ON THE LEFT SIDE YOU FEED IMAGES ON THE LEFT SIDE AND DID GIVE THE LABEL DOG AND AND DID GIVE THE LABEL DOG AND AND DID GIVE THE LABEL DOG AND FEED IT TO THE NETWORK, AND THE FEED IT TO THE NETWORK, AND THE FEED IT TO THE NETWORK, AND THE PROCESS BY WHICH IT WORKS IS YOU PROCESS BY WHICH IT WORKS IS YOU PROCESS BY WHICH IT WORKS IS YOU ADJUST WEIGHTS IN THE NETWORKS ADJUST WEIGHTS IN THE NETWORKS ADJUST WEIGHTS IN THE NETWORKS SUCH THAT THE NETWORK LEARNS TO SUCH THAT THE NETWORK LEARNS TO SUCH THAT THE NETWORK LEARNS TO DISTINGUISH AND DISCRIINATE DISTINGUISH AND DISCRIINATE DISTINGUISH AND DISCRIINATE BETWEENS DIFFERENT CONAT THE BETWEENS DIFFERENT CONAT THE BETWEENS DIFFERENT CONAT THE PARTICULAR TIME AND TARGET THE PARTICULAR TIME AND TARGET THE PARTICULAR TIME AND TARGET THE IMAGE. IMAGE. IMAGE. THIS IS THE LAST SESSION, LAST THIS IS THE LAST SESSION, LAST THIS IS THE LAST SESSION, LAST DAY. DAY. DAY. STILL, VERY HAPPY THAT YOU ARE STILL, VERY HAPPY THAT YOU ARE STILL, VERY HAPPY THAT YOU ARE ALL WITH ME HERE. ALL WITH ME HERE. ALL WITH ME HERE. WHAT IS THE ONE THING? WHAT IS THE ONE THING? WHAT IS THE ONE THING? IT'S ALL GREAT. IT'S ALL GREAT. IT'S ALL GREAT. WE HAVE A LOT OF FANCY ALGA WE HAVE A LOT OF FANCY ALGA WE HAVE A LOT OF FANCY ALGA RHYTHMS AND FANCY NETWORKS. RHYTHMS AND FANCY NETWORKS. RHYTHMS AND FANCY NETWORKS. WHAT IS THE ONE CORPS INGREDIENT WHAT IS THE ONE CORPS INGREDIENT WHAT IS THE ONE CORPS INGREDIENT WE NEED WHEN WE BUILD A NETWORK? WE NEED WHEN WE BUILD A NETWORK? WE NEED WHEN WE BUILD A NETWORK? ALMOST MAJORITY OF THE ALMOST MAJORITY OF THE ALMOST MAJORITY OF THE APPLICATIONS WE WORK ON REQUIRE APPLICATIONS WE WORK ON REQUIRE APPLICATIONS WE WORK ON REQUIRE DATA. DATA. DATA. IT'S NOT ONE IMAGE YOU ARE IT'S NOT ONE IMAGE YOU ARE IT'S NOT ONE IMAGE YOU ARE FEEDING TO THIS NETWORK. FEEDING TO THIS NETWORK. FEEDING TO THIS NETWORK. YOU ARE TAKING A BUNCH OF IMAGES YOU ARE TAKING A BUNCH OF IMAGES YOU ARE TAKING A BUNCH OF IMAGES PAIRED WITH THE LABELS, CATS AND PAIRED WITH THE LABELS, CATS AND PAIRED WITH THE LABELS, CATS AND DOGS IN THIS CASE, BUT, OF DOGS IN THIS CASE, BUT, OF DOGS IN THIS CASE, BUT, OF COURSE, IT COULD BE WHATEVER, COURSE, IT COULD BE WHATEVER, COURSE, IT COULD BE WHATEVER, LIKE, DEPENDING ON THE LIKE, DEPENDING ON THE LIKE, DEPENDING ON THE APPLICATION. APPLICATION. APPLICATION. YOU FEED IT, LIKE, THOUSANDS OR YOU FEED IT, LIKE, THOUSANDS OR YOU FEED IT, LIKE, THOUSANDS OR HUNDREDS OF THANK YOUS THOUSANDS HUNDREDS OF THANK YOUS THOUSANDS HUNDREDS OF THANK YOUS THOUSANDS OR EVEN MILLIONS OF EXAMPLES OR EVEN MILLIONS OF EXAMPLES OR EVEN MILLIONS OF EXAMPLES INTO THE NETWORK TO TRAIN A GOOD INTO THE NETWORK TO TRAIN A GOOD INTO THE NETWORK TO TRAIN A GOOD CLASSIFIED. CLASSIFIED. CLASSIFIED. RIGHT? RIGHT? RIGHT? TODAY WE'RE GOING TO INTRODUCE TODAY WE'RE GOING TO INTRODUCE TODAY WE'RE GOING TO INTRODUCE NOODLE STRUCTURED LEARNING, NOODLE STRUCTURED LEARNING, NOODLE STRUCTURED LEARNING, WHICH IS A FRAMEWORK, WE'RE WHICH IS A FRAMEWORK, WE'RE WHICH IS A FRAMEWORK, WE'RE HAPPY TO TALK ABOUT IT INTENSIVE HAPPY TO TALK ABOUT IT INTENSIVE HAPPY TO TALK ABOUT IT INTENSIVE 2.0 AND IT ALLOWS YOU TO TRAIN 2.0 AND IT ALLOWS YOU TO TRAIN 2.0 AND IT ALLOWS YOU TO TRAIN BETTER AND MORROW BUST NETWORKS BETTER AND MORROW BUST NETWORKS BETTER AND MORROW BUST NETWORKS BY LEVERAGING STRUCTURE IN THE BY LEVERAGING STRUCTURE IN THE BY LEVERAGING STRUCTURE IN THE DATA. DATA. DATA. SO THE CORE IDEA BEHIND THE SO THE CORE IDEA BEHIND THE SO THE CORE IDEA BEHIND THE FRAMEWORK IS WE'RE GOING TO TAKE FRAMEWORK IS WE'RE GOING TO TAKE FRAMEWORK IS WE'RE GOING TO TAKE NOODLE NETWORKS AND FEED IS IN NOODLE NETWORKS AND FEED IS IN NOODLE NETWORKS AND FEED IS IN ADDITION TO FEATURE INPUTS, ADDITION TO FEATURE INPUTS, ADDITION TO FEATURE INPUTS, STRUCTURED SIGNALS. STRUCTURED SIGNALS. STRUCTURED SIGNALS. THINK OF THE IMAGE THAT I SHOWED THINK OF THE IMAGE THAT I SHOWED THINK OF THE IMAGE THAT I SHOWED YOU EARLIER. YOU EARLIER. YOU EARLIER. NOW, IN ADDITION TO THESE IMAGES NOW, IN ADDITION TO THESE IMAGES NOW, IN ADDITION TO THESE IMAGES PAIRED WITH LABELS, YOU ARE PAIRED WITH LABELS, YOU ARE PAIRED WITH LABELS, YOU ARE GOING TO FEED IT CONNECTIONS OR GOING TO FEED IT CONNECTIONS OR GOING TO FEED IT CONNECTIONS OR RELATIONSHIPS BETWEEN THE RELATIONSHIPS BETWEEN THE RELATIONSHIPS BETWEEN THE SAMPLES THEMSELVES. SAMPLES THEMSELVES. SAMPLES THEMSELVES. I WILL GET TO WHAT THESE I WILL GET TO WHAT THESE I WILL GET TO WHAT THESE RELATIONSHIPS MIGHT MEAN, BUT RELATIONSHIPS MIGHT MEAN, BUT RELATIONSHIPS MIGHT MEAN, BUT YOU HAVE THESE STRUCTURED YOU HAVE THESE STRUCTURED YOU HAVE THESE STRUCTURED SIGNALS, AND THE LABELS AND YOU SIGNALS, AND THE LABELS AND YOU SIGNALS, AND THE LABELS AND YOU FEED BOTH INTO THE NETWORK. FEED BOTH INTO THE NETWORK. FEED BOTH INTO THE NETWORK. YOU MIGHT ASK WHAT DO YOU MEAN YOU MIGHT ASK WHAT DO YOU MEAN YOU MIGHT ASK WHAT DO YOU MEAN BY STRUCTURE, RIGHT? BY STRUCTURE, RIGHT? BY STRUCTURE, RIGHT? STRUCTURE IS EVERYWHERE. STRUCTURE IS EVERYWHERE. STRUCTURE IS EVERYWHERE. IN THE EXAMPLE THAT I SHOWED YOU IN THE EXAMPLE THAT I SHOWED YOU IN THE EXAMPLE THAT I SHOWED YOU EARLIER, IF YOU LOOK AT IMAGES, EARLIER, IF YOU LOOK AT IMAGES, EARLIER, IF YOU LOOK AT IMAGES, JUST TAKE A LOOK AT THIS GRAPH JUST TAKE A LOOK AT THIS GRAPH JUST TAKE A LOOK AT THIS GRAPH HERE. HERE. HERE. THE IMAGE THAT IS ARE CONNECTED THE IMAGE THAT IS ARE CONNECTED THE IMAGE THAT IS ARE CONNECTED VIA EDGES IN THE PICTURE HERE VIA EDGES IN THE PICTURE HERE VIA EDGES IN THE PICTURE HERE BASICALLY REPRESENT THAT THERE'S BASICALLY REPRESENT THAT THERE'S BASICALLY REPRESENT THAT THERE'S SOME VISUAL SIMILARITIES BETWEEN SOME VISUAL SIMILARITIES BETWEEN SOME VISUAL SIMILARITIES BETWEEN THESE. THESE. THESE. IT IS ACTUALLY PRETTY EASY TO IT IS ACTUALLY PRETTY EASY TO IT IS ACTUALLY PRETTY EASY TO CONSTRUCT A STRUCTURED SIGNAL CONSTRUCT A STRUCTURED SIGNAL CONSTRUCT A STRUCTURED SIGNAL FROM DAY TO DAY SOURCES OF DATA. FROM DAY TO DAY SOURCES OF DATA. FROM DAY TO DAY SOURCES OF DATA. IN THE CASE OF IMAGES HERE, IT'S IN THE CASE OF IMAGES HERE, IT'S IN THE CASE OF IMAGES HERE, IT'S VISUAL SIMILARITY, BUT YOU COULD VISUAL SIMILARITY, BUT YOU COULD VISUAL SIMILARITY, BUT YOU COULD THINK OF WHAT IF YOU TAGGED YOUR THINK OF WHAT IF YOU TAGGED YOUR THINK OF WHAT IF YOU TAGGED YOUR IMAGES AND CREATED ALBUMS. IMAGES AND CREATED ALBUMS. IMAGES AND CREATED ALBUMS. THAT HAS SPECIFIC CONCEPTS. THAT HAS SPECIFIC CONCEPTS. THAT HAS SPECIFIC CONCEPTS. EVERYTHING WITHIN THAT REPTSZ EVERYTHING WITHIN THAT REPTSZ EVERYTHING WITHIN THAT REPTSZ ANOTHER TYPE OF STRUCTURE. ANOTHER TYPE OF STRUCTURE. ANOTHER TYPE OF STRUCTURE. IT'S NOT JUST FOR IMAGES. IT'S NOT JUST FOR IMAGES. IT'S NOT JUST FOR IMAGES. WE CAN GO TO, LIKE, MORE WE CAN GO TO, LIKE, MORE WE CAN GO TO, LIKE, MORE ADVANCED OR, LIKE, YOU KNOW, ADVANCED OR, LIKE, YOU KNOW, ADVANCED OR, LIKE, YOU KNOW, DIFFERENT -- COMPLETELY DIFFERENT -- COMPLETELY DIFFERENT -- COMPLETELY DIFFERENT APPLICATIONS LIKE IF DIFFERENT APPLICATIONS LIKE IF DIFFERENT APPLICATIONS LIKE IF YOU WANT TO TAKE, LIKE, YOU WANT TO TAKE, LIKE, YOU WANT TO TAKE, LIKE, PUBLICATIONS, SCIENTIFIC PUBLICATIONS, SCIENTIFIC PUBLICATIONS, SCIENTIFIC PUBLICATIONS OR NEWS ARTICLES, PUBLICATIONS OR NEWS ARTICLES, PUBLICATIONS OR NEWS ARTICLES, AND YOU WANT TO TAG THEM WITH AND YOU WANT TO TAG THEM WITH AND YOU WANT TO TAG THEM WITH THE TOPIC, ONE SIMPLE THING, THE TOPIC, ONE SIMPLE THING, THE TOPIC, ONE SIMPLE THING, TAKE BIOMEDICAL LITERATURE. TAKE BIOMEDICAL LITERATURE. TAKE BIOMEDICAL LITERATURE. ALL THE PAPERS THAT ARE ALL THE PAPERS THAT ARE ALL THE PAPERS THAT ARE PUBLISHED WHETHER IT'S NATURE OR PUBLISHED WHETHER IT'S NATURE OR PUBLISHED WHETHER IT'S NATURE OR ANY OF THE CONFERENCES, THEY ANY OF THE CONFERENCES, THEY ANY OF THE CONFERENCES, THEY HAVE REFERENCES AND CITATIONS TO HAVE REFERENCES AND CITATIONS TO HAVE REFERENCES AND CITATIONS TO OTHER PAPERS. OTHER PAPERS. OTHER PAPERS. RELATIONSHIPS THAT ARE MODELLED RELATIONSHIPS THAT ARE MODELLED RELATIONSHIPS THAT ARE MODELLED BETWEEN DIFFERENT TYPES OF BETWEEN DIFFERENT TYPES OF BETWEEN DIFFERENT TYPES OF OBJECTS. OBJECTS. OBJECTS. IN THE NATURAL LANGUAGE SPACE, IN THE NATURAL LANGUAGE SPACE, IN THE NATURAL LANGUAGE SPACE, THIS OCCURS EVERYWHERE. THIS OCCURS EVERYWHERE. THIS OCCURS EVERYWHERE. IF YOU ARE TALKING ABOUT DOING IF YOU ARE TALKING ABOUT DOING IF YOU ARE TALKING ABOUT DOING SEARCH, EVERYBODY HAS HEARD OF SEARCH, EVERYBODY HAS HEARD OF SEARCH, EVERYBODY HAS HEARD OF KNOWLEDGE GRAPH, WHICH IS A RICH KNOWLEDGE GRAPH, WHICH IS A RICH KNOWLEDGE GRAPH, WHICH IS A RICH SOURCE OF INFORMATION WHICH SOURCE OF INFORMATION WHICH SOURCE OF INFORMATION WHICH CAP CAP CAP CAPTURES. CAPTURES. CAPTURES. THE RELATIONSHIP IS ONE IS A THE RELATIONSHIP IS ONE IS A THE RELATIONSHIP IS ONE IS A CAPITAL OF THE OTHER. CAPITAL OF THE OTHER. CAPITAL OF THE OTHER. RIGHT? RIGHT? RIGHT? THESE SORT OF RELATIONSHIPS THESE SORT OF RELATIONSHIPS THESE SORT OF RELATIONSHIPS THESE ARE THE KIND OF THESE ARE THE KIND OF THESE ARE THE KIND OF RELATIONSHIPS WHICH ALREADY RELATIONSHIPS WHICH ALREADY RELATIONSHIPS WHICH ALREADY EXIST IN DAY TO DAY DATA EXIST IN DAY TO DAY DATA EXIST IN DAY TO DAY DATA SOURCES. SOURCES. SOURCES. WHY NOT LEVERAGE THEM. WHY NOT LEVERAGE THEM. WHY NOT LEVERAGE THEM. THAT IS WHAT YOU TRY TO DO WITH THAT IS WHAT YOU TRY TO DO WITH THAT IS WHAT YOU TRY TO DO WITH THE NOODLED STRUCTURED LEARNING. THE NOODLED STRUCTURED LEARNING. THE NOODLED STRUCTURED LEARNING. BEFORE WE TALK ABOUT WHAT IT BEFORE WE TALK ABOUT WHAT IT BEFORE WE TALK ABOUT WHAT IT DOES AND HOW WE DO IT, WHY DO DOES AND HOW WE DO IT, WHY DO DOES AND HOW WE DO IT, WHY DO YOU EVEN WANT TO CARE ABOUT IT? YOU EVEN WANT TO CARE ABOUT IT? YOU EVEN WANT TO CARE ABOUT IT? RIGHT? RIGHT? RIGHT? WHAT DOES IT BENEFIT. WHAT DOES IT BENEFIT. WHAT DOES IT BENEFIT. IT ALLOWS YOU TO TAKE THE IT ALLOWS YOU TO TAKE THE IT ALLOWS YOU TO TAKE THE STRUCTURE AND USE IT TO TRAIN STRUCTURE AND USE IT TO TRAIN STRUCTURE AND USE IT TO TRAIN LECTURES WITH LESS LABEL DATA, LECTURES WITH LESS LABEL DATA, LECTURES WITH LESS LABEL DATA, AND THAT'S THE COSTLY PROCESS. AND THAT'S THE COSTLY PROCESS. AND THAT'S THE COSTLY PROCESS. EVERY APPLICATION THAT YOU WANT EVERY APPLICATION THAT YOU WANT EVERY APPLICATION THAT YOU WANT TO TRAIN, IF YOU HAD TO COLLECT TO TRAIN, IF YOU HAD TO COLLECT TO TRAIN, IF YOU HAD TO COLLECT A LOT OF RICH ANNOTATED DATE A LOT OF RICH ANNOTATED DATE A LOT OF RICH ANNOTATED DATE WRA. WRA. WRA. IF YOU ARE ABLE TO USE A IF YOU ARE ABLE TO USE A IF YOU ARE ABLE TO USE A FRAMEWORK LIKE NEURAL STRUCTURED FRAMEWORK LIKE NEURAL STRUCTURED FRAMEWORK LIKE NEURAL STRUCTURED LEARNING, THAT AUTOMATICALLY LEARNING, THAT AUTOMATICALLY LEARNING, THAT AUTOMATICALLY CAPTURES THE DATA STRUCTURE AND CAPTURES THE DATA STRUCTURE AND CAPTURES THE DATA STRUCTURE AND RELATIONSHIP AND WITH MINIMAL RELATIONSHIP AND WITH MINIMAL RELATIONSHIP AND WITH MINIMAL SUPERVISION ARE YOU ABLE TO SUPERVISION ARE YOU ABLE TO SUPERVISION ARE YOU ABLE TO TRAIN PREDICTIVE SYSTEMS WITH TRAIN PREDICTIVE SYSTEMS WITH TRAIN PREDICTIVE SYSTEMS WITH THE SAME ACCURACY, THAT WOULD BE THE SAME ACCURACY, THAT WOULD BE THE SAME ACCURACY, THAT WOULD BE A HUGE THING. A HUGE THING. A HUGE THING. WHO WOULDN'T WANT THAT, RIGHT? WHO WOULDN'T WANT THAT, RIGHT? WHO WOULDN'T WANT THAT, RIGHT? THAT'S ONE TYPE OF BENEFIT. THAT'S ONE TYPE OF BENEFIT. THAT'S ONE TYPE OF BENEFIT. ANOTHER ONE IS THAT TYPICALLY ANOTHER ONE IS THAT TYPICALLY ANOTHER ONE IS THAT TYPICALLY WHEN YOU DEPLOY THESE SYSTEMS IN WHEN YOU DEPLOY THESE SYSTEMS IN WHEN YOU DEPLOY THESE SYSTEMS IN PRACTICE IN REAL WORLD PRACTICE IN REAL WORLD PRACTICE IN REAL WORLD APPLICATIONS, BE YOU WANT THE APPLICATIONS, BE YOU WANT THE APPLICATIONS, BE YOU WANT THE SYSTEMS OR NETWORKS TO BE SYSTEMS OR NETWORKS TO BE SYSTEMS OR NETWORKS TO BE REBUST. REBUST. REBUST. THAT MEANS YOU TRAIN THEM ONCE. THAT MEANS YOU TRAIN THEM ONCE. THAT MEANS YOU TRAIN THEM ONCE. YOU DON'T WANT -- IF THE INPUT YOU DON'T WANT -- IF THE INPUT YOU DON'T WANT -- IF THE INPUT DISTRIBUTION CHANGES OR DATA DISTRIBUTION CHANGES OR DATA DISTRIBUTION CHANGES OR DATA SUDDENLY CHANGES OR SOMEBODY, SUDDENLY CHANGES OR SOMEBODY, SUDDENLY CHANGES OR SOMEBODY, YOU KNOW -- SUDDENLY THEY FLIP YOU KNOW -- SUDDENLY THEY FLIP YOU KNOW -- SUDDENLY THEY FLIP THE PREDICTIONS AND GO THE PREDICTIONS AND GO THE PREDICTIONS AND GO BONNINGERS, RIGHT? BONNINGERS, RIGHT? BONNINGERS, RIGHT? THIS IS ANOTHER BENEFIT WHERE IF THIS IS ANOTHER BENEFIT WHERE IF THIS IS ANOTHER BENEFIT WHERE IF YOU USE NEURAL STRUCTURED YOU USE NEURAL STRUCTURED YOU USE NEURAL STRUCTURED LEARNING, YOU CAN ACTUALLY LEARNING, YOU CAN ACTUALLY LEARNING, YOU CAN ACTUALLY IMPROVE THE QUALITY OF THE IMPROVE THE QUALITY OF THE IMPROVE THE QUALITY OF THE NETWORK AND ALSO THE ROBUSTNESS NETWORK AND ALSO THE ROBUSTNESS NETWORK AND ALSO THE ROBUSTNESS OF THE NETWORK. OF THE NETWORK. OF THE NETWORK. LET ME DIVE A LITTLE DEEPER AND LET ME DIVE A LITTLE DEEPER AND LET ME DIVE A LITTLE DEEPER AND GIVE YOU A LITTLE MORE, YOU GIVE YOU A LITTLE MORE, YOU GIVE YOU A LITTLE MORE, YOU KNOW, INSIGHT INTO, RIGHT? KNOW, INSIGHT INTO, RIGHT? KNOW, INSIGHT INTO, RIGHT? TAKE DOCUMENT CLASSIFICATION AS TAKE DOCUMENT CLASSIFICATION AS TAKE DOCUMENT CLASSIFICATION AS AN EXAMPLE. AN EXAMPLE. AN EXAMPLE. SO I'LL GIVE YOU PROBABLY SOME SO I'LL GIVE YOU PROBABLY SOME SO I'LL GIVE YOU PROBABLY SOME EXAMPLE THAT YOU PROBABLY HAVE EXAMPLE THAT YOU PROBABLY HAVE EXAMPLE THAT YOU PROBABLY HAVE AT YOUR HOME; RIGHT? AT YOUR HOME; RIGHT? AT YOUR HOME; RIGHT? IMAGINE YOU HAVE A CATALOG OR A IMAGINE YOU HAVE A CATALOG OR A IMAGINE YOU HAVE A CATALOG OR A LIBRARY OF BOOKS IN YOUR HOME. LIBRARY OF BOOKS IN YOUR HOME. LIBRARY OF BOOKS IN YOUR HOME. THESE ARE DIGITIZED CONAT THE THESE ARE DIGITIZED CONAT THE THESE ARE DIGITIZED CONAT THE PARTICULAR TIME, AND YOU WANT TO PARTICULAR TIME, AND YOU WANT TO PARTICULAR TIME, AND YOU WANT TO NEATLY ARRANGE THEM INTO NEATLY ARRANGE THEM INTO NEATLY ARRANGE THEM INTO SPECIFIC TOPICS OR CATEGORIES. SPECIFIC TOPICS OR CATEGORIES. SPECIFIC TOPICS OR CATEGORIES. NOW, ONE PERSON MIGHT WANT TO NOW, ONE PERSON MIGHT WANT TO NOW, ONE PERSON MIGHT WANT TO CATEGORIZE THEM BASED ON THE CATEGORIZE THEM BASED ON THE CATEGORIZE THEM BASED ON THE GENRE. GENRE. GENRE. A DIFFERENT PERSON MIGHT SAY, A DIFFERENT PERSON MIGHT SAY, A DIFFERENT PERSON MIGHT SAY, OH, I WANT, LIKE, IT TO BELONG OH, I WANT, LIKE, IT TO BELONG OH, I WANT, LIKE, IT TO BELONG TO, YOU KNOW, THE SAME PERIOD. TO, YOU KNOW, THE SAME PERIOD. TO, YOU KNOW, THE SAME PERIOD. A THIRD PERSON MIGHT SAY I WANT A THIRD PERSON MIGHT SAY I WANT A THIRD PERSON MIGHT SAY I WANT TO CAPTURE DIFFERENT BOOKS THAT TO CAPTURE DIFFERENT BOOKS THAT TO CAPTURE DIFFERENT BOOKS THAT HAVE THE SAME WORDS THAT APPEAR HAVE THE SAME WORDS THAT APPEAR HAVE THE SAME WORDS THAT APPEAR IN THE SAME -- BY THE SAME IN THE SAME -- BY THE SAME IN THE SAME -- BY THE SAME AUTHOR OR BASED ON SOME CERTAIN AUTHOR OR BASED ON SOME CERTAIN AUTHOR OR BASED ON SOME CERTAIN ASPECT, LIKE THERE IS LIKE A ASPECT, LIKE THERE IS LIKE A ASPECT, LIKE THERE IS LIKE A PARTICULAR PLOT TWIST THAT YOU PARTICULAR PLOT TWIST THAT YOU PARTICULAR PLOT TWIST THAT YOU HAVE THAT IS CAPTURED IN THE HAVE THAT IS CAPTURED IN THE HAVE THAT IS CAPTURED IN THE DIFFERENT BOOKS, AND NOW I WANT DIFFERENT BOOKS, AND NOW I WANT DIFFERENT BOOKS, AND NOW I WANT TO ARRANGE THEM BASED ON THAT. TO ARRANGE THEM BASED ON THAT. TO ARRANGE THEM BASED ON THAT. YOU CAN THINK OF NOT EVERYBODY'S YOU CAN THINK OF NOT EVERYBODY'S YOU CAN THINK OF NOT EVERYBODY'S NEEDS ARE THE SAME. NEEDS ARE THE SAME. NEEDS ARE THE SAME. ARE YOU GOING TO COLLECT ENOUGH ARE YOU GOING TO COLLECT ENOUGH ARE YOU GOING TO COLLECT ENOUGH LABEL DATA FOR EACH OF THOSE LABEL DATA FOR EACH OF THOSE LABEL DATA FOR EACH OF THOSE TASKS, YOU KNOW, TO CREATE A TASKS, YOU KNOW, TO CREATE A TASKS, YOU KNOW, TO CREATE A NETWORK THAT CAN HAVE VERY HAYEK NETWORK THAT CAN HAVE VERY HAYEK NETWORK THAT CAN HAVE VERY HAYEK ARE ASSY AND CLASSIFY THEM INTO ARE ASSY AND CLASSIFY THEM INTO ARE ASSY AND CLASSIFY THEM INTO THE GENRE? THE GENRE? THE GENRE? PROBABLY NOT. PROBABLY NOT. PROBABLY NOT. ON THE ONE HAND YOU HAVE PLENTY ON THE ONE HAND YOU HAVE PLENTY ON THE ONE HAND YOU HAVE PLENTY OF DATA, THE RAW BOOK CONTENT OF DATA, THE RAW BOOK CONTENT OF DATA, THE RAW BOOK CONTENT THAT'S AVAILABLE TO YOU FOR RAW THAT'S AVAILABLE TO YOU FOR RAW THAT'S AVAILABLE TO YOU FOR RAW NEWS ARTICLES. NEWS ARTICLES. NEWS ARTICLES. THERE'S PLENTY OF RAW TEXT THERE'S PLENTY OF RAW TEXT THERE'S PLENTY OF RAW TEXT AVAILABLE TO YOU, BUT IT'S HARD AVAILABLE TO YOU, BUT IT'S HARD AVAILABLE TO YOU, BUT IT'S HARD TO CONSTRUCT THIS LABEL TO CONSTRUCT THIS LABEL TO CONSTRUCT THIS LABEL ANNOTATIVEL. ANNOTATIVEL. ANNOTATIVEL. WE CAN USE STRUCTURE. WE CAN USE STRUCTURE. WE CAN USE STRUCTURE. AGAIN, I WILL TELL YOU WHAT THE AGAIN, I WILL TELL YOU WHAT THE AGAIN, I WILL TELL YOU WHAT THE STRUCTURE MEANS, BUT PAIR IT STRUCTURE MEANS, BUT PAIR IT STRUCTURE MEANS, BUT PAIR IT WITH A FEW LABEL EXAMPLES, AND WITH A FEW LABEL EXAMPLES, AND WITH A FEW LABEL EXAMPLES, AND THEN FRAME A NETWORK THAT ALMOST THEN FRAME A NETWORK THAT ALMOST THEN FRAME A NETWORK THAT ALMOST AS GOOD AS THE NETWORK THAT IS AS GOOD AS THE NETWORK THAT IS AS GOOD AS THE NETWORK THAT IS TRAINED AND MILLIONS OF TRAINED AND MILLIONS OF TRAINED AND MILLIONS OF EXAMPLES. EXAMPLES. EXAMPLES. IMAGINE YOU COULD -- FOR YOUR IMAGINE YOU COULD -- FOR YOUR IMAGINE YOU COULD -- FOR YOUR APPLICATION -- ONLY USE 5% OR APPLICATION -- ONLY USE 5% OR APPLICATION -- ONLY USE 5% OR 10% OF THE LABEL DATA AND TRAIN 10% OF THE LABEL DATA AND TRAIN 10% OF THE LABEL DATA AND TRAIN AS GOOD A CLASSIFIER OR LIKE A AS GOOD A CLASSIFIER OR LIKE A AS GOOD A CLASSIFIER OR LIKE A PREDICTION SYSTEM. PREDICTION SYSTEM. PREDICTION SYSTEM. I'M GOING TO GIVE YOU A SOLID I'M GOING TO GIVE YOU A SOLID I'M GOING TO GIVE YOU A SOLID REFERENCE HERE, AND WE'RE GOING REFERENCE HERE, AND WE'RE GOING REFERENCE HERE, AND WE'RE GOING TO TALK MORE ABOUT THAT AS WELL. TO TALK MORE ABOUT THAT AS WELL. TO TALK MORE ABOUT THAT AS WELL. THERE IS A HANDS-ON TUTORIAL FOR THERE IS A HANDS-ON TUTORIAL FOR THERE IS A HANDS-ON TUTORIAL FOR THE EXACT EXAMPLE THAT I TOLD THE EXACT EXAMPLE THAT I TOLD THE EXACT EXAMPLE THAT I TOLD YOU. YOU. YOU. DOCUMENT CLASSIFICATION WITH A DOCUMENT CLASSIFICATION WITH A DOCUMENT CLASSIFICATION WITH A GRAPH, AND YOU CAN GO TO THE GRAPH, AND YOU CAN GO TO THE GRAPH, AND YOU CAN GO TO THE NEURAL STRUCTURED LEARNING NEURAL STRUCTURED LEARNING NEURAL STRUCTURED LEARNING WEBSITE AND TRY THIS OUT FOR WEBSITE AND TRY THIS OUT FOR WEBSITE AND TRY THIS OUT FOR YOURSELF. YOURSELF. YOURSELF. ALL WITHIN JUST A FEW LINES OF ALL WITHIN JUST A FEW LINES OF ALL WITHIN JUST A FEW LINES OF CODE. CODE. CODE. ALL YOU NEED TO DO IS CONSTRUCT ALL YOU NEED TO DO IS CONSTRUCT ALL YOU NEED TO DO IS CONSTRUCT THE DATA IN THE FORMAT, AND WITH THE DATA IN THE FORMAT, AND WITH THE DATA IN THE FORMAT, AND WITH A FEW LINES OF CODE, YOU SHOULD A FEW LINES OF CODE, YOU SHOULD A FEW LINES OF CODE, YOU SHOULD BE ABLE TO RUN THE SYSTEM BE ABLE TO RUN THE SYSTEM BE ABLE TO RUN THE SYSTEM END-TO-END. END-TO-END. END-TO-END. WE HAVE SEEN THE CAPACITY TO WE HAVE SEEN THE CAPACITY TO WE HAVE SEEN THE CAPACITY TO TRAIN QUALITY MODELS, ESPECIALLY TRAIN QUALITY MODELS, ESPECIALLY TRAIN QUALITY MODELS, ESPECIALLY AS THE NUMBER OF PERIMETERS, BUT AS THE NUMBER OF PERIMETERS, BUT AS THE NUMBER OF PERIMETERS, BUT ROBUSTNESS IS AN IMPORTANT ROBUSTNESS IS AN IMPORTANT ROBUSTNESS IS AN IMPORTANT CONCEPT, AND IT'S IMPORTANT WHEN CONCEPT, AND IT'S IMPORTANT WHEN CONCEPT, AND IT'S IMPORTANT WHEN WE DEPLOY THIS TO REAL WORLD WE DEPLOY THIS TO REAL WORLD WE DEPLOY THIS TO REAL WORLD SCENARIOS. SCENARIOS. SCENARIOS. FOR EXAMPLE, IF YOU HAVE AN FOR EXAMPLE, IF YOU HAVE AN FOR EXAMPLE, IF YOU HAVE AN IMAGE RECOGNITION SYSTEM AND IMAGE RECOGNITION SYSTEM AND IMAGE RECOGNITION SYSTEM AND SUDDENLY THE NET WORLD WAR I, SUDDENLY THE NET WORLD WAR I, SUDDENLY THE NET WORLD WAR I, THIS IS NOT A GOOD THING, AND THIS IS NOT A GOOD THING, AND THIS IS NOT A GOOD THING, AND THIS IS JUST -- NOT JUST THIS IS JUST -- NOT JUST THIS IS JUST -- NOT JUST CLASSIFICATION. CLASSIFICATION. CLASSIFICATION. IT COULD BE FOR TEXT OR ANY KIND IT COULD BE FOR TEXT OR ANY KIND IT COULD BE FOR TEXT OR ANY KIND OF SENSOR DATA. OF SENSOR DATA. OF SENSOR DATA. HERE'S AN EXAMPLE. HERE'S AN EXAMPLE. HERE'S AN EXAMPLE. TAKE THE IMAGE ON THE LEFT. TAKE THE IMAGE ON THE LEFT. TAKE THE IMAGE ON THE LEFT. WHAT DO YOU THINK IT IS? WHAT DO YOU THINK IT IS? WHAT DO YOU THINK IT IS? IT'S A PANDA. IT'S A PANDA. IT'S A PANDA. TAKE THE IMAGE ON THE RIGHT. TAKE THE IMAGE ON THE RIGHT. TAKE THE IMAGE ON THE RIGHT. WHAT DO YOU THINK IT IS? WHAT DO YOU THINK IT IS? WHAT DO YOU THINK IT IS? ANYONE WHO DISAGREES IT'S A ANYONE WHO DISAGREES IT'S A ANYONE WHO DISAGREES IT'S A PANDA? PANDA? PANDA? BASICALLY YOUR NEURAL NETWORK IS BASICALLY YOUR NEURAL NETWORK IS BASICALLY YOUR NEURAL NETWORK IS SAYING BOTH OF THESE IMAGES ARE SAYING BOTH OF THESE IMAGES ARE SAYING BOTH OF THESE IMAGES ARE PANDAS, BUT THAT'S NOT WHAT PANDAS, BUT THAT'S NOT WHAT PANDAS, BUT THAT'S NOT WHAT HAPPENS IF YOU ACTUALLY CREATE A HAPPENS IF YOU ACTUALLY CREATE A HAPPENS IF YOU ACTUALLY CREATE A NEURAL NETWORK. NEURAL NETWORK. NEURAL NETWORK. THE REASON IT HAPPENS IS YOU THE REASON IT HAPPENS IS YOU THE REASON IT HAPPENS IS YOU ZOOM IN REALLY CLOSE. ZOOM IN REALLY CLOSE. ZOOM IN REALLY CLOSE. THE SECOND IMAGE IS AN THE SECOND IMAGE IS AN THE SECOND IMAGE IS AN ADVERSARIAL EXAMPLE. ADVERSARIAL EXAMPLE. ADVERSARIAL EXAMPLE. IT'S ADDING NOISE TO THE IT'S ADDING NOISE TO THE IT'S ADDING NOISE TO THE ORIGINAL IMAGE, BUT THESE ARE SO ORIGINAL IMAGE, BUT THESE ARE SO ORIGINAL IMAGE, BUT THESE ARE SO TINY. TINY. TINY. IT'S IMPERCEIVABLE FOR THE HUMAN IT'S IMPERCEIVABLE FOR THE HUMAN IT'S IMPERCEIVABLE FOR THE HUMAN EYE, BUT THE NETWORK BASED ON EYE, BUT THE NETWORK BASED ON EYE, BUT THE NETWORK BASED ON THE PIXELS FLIPS IT COMPLETELY. THE PIXELS FLIPS IT COMPLETELY. THE PIXELS FLIPS IT COMPLETELY. IMAGINE THIS HAPPENING IN AWE IMAGINE THIS HAPPENING IN AWE IMAGINE THIS HAPPENING IN AWE LIVE SYSTEM. LIVE SYSTEM. LIVE SYSTEM. YOU WANT THE NETWORKS TO BE YOU WANT THE NETWORKS TO BE YOU WANT THE NETWORKS TO BE REBUST. REBUST. REBUST. THIS IS WHERE AN ASSAULT COMES THIS IS WHERE AN ASSAULT COMES THIS IS WHERE AN ASSAULT COMES AGAIN, AND, AGAIN, THEY'RE GOING AGAIN, AND, AGAIN, THEY'RE GOING AGAIN, AND, AGAIN, THEY'RE GOING TO USE THE SAME CONCEPT. TO USE THE SAME CONCEPT. TO USE THE SAME CONCEPT. USE THE STRUCTURE IN THE DATA TO USE THE STRUCTURE IN THE DATA TO USE THE STRUCTURE IN THE DATA TO TRAIN MORROW BUST MODELS, AND TRAIN MORROW BUST MODELS, AND TRAIN MORROW BUST MODELS, AND HERE THE STRUCTURE IS LIKELY HERE THE STRUCTURE IS LIKELY HERE THE STRUCTURE IS LIKELY DIFFERENT FROM THE ONE THAT I DIFFERENT FROM THE ONE THAT I DIFFERENT FROM THE ONE THAT I MENTIONED EARLIER. MENTIONED EARLIER. MENTIONED EARLIER. EARLIER WE'RE TALKING ABOUT EARLIER WE'RE TALKING ABOUT EARLIER WE'RE TALKING ABOUT MODELIZED GRAPHS. MODELIZED GRAPHS. MODELIZED GRAPHS. HERE WE ARE GOING TO CONSTRUCT HERE WE ARE GOING TO CONSTRUCT HERE WE ARE GOING TO CONSTRUCT THAT STRUCTURE. THAT STRUCTURE. THAT STRUCTURE. WE TAKE THE ORIGINAL IMAGE, AND WE TAKE THE ORIGINAL IMAGE, AND WE TAKE THE ORIGINAL IMAGE, AND WE ALSO GENERATE A ADVERSARIAL WE ALSO GENERATE A ADVERSARIAL WE ALSO GENERATE A ADVERSARIAL EXAMPLE FOR THE IMAGE. EXAMPLE FOR THE IMAGE. EXAMPLE FOR THE IMAGE. THESE TWO IMAGES ARE JOINED VIA THESE TWO IMAGES ARE JOINED VIA THESE TWO IMAGES ARE JOINED VIA A LINK. A LINK. A LINK. THERE'S A RELATIONSHIP BETWEEN THERE'S A RELATIONSHIP BETWEEN THERE'S A RELATIONSHIP BETWEEN THEM. THEM. THEM. THE DIFFERENCE IS THIS IS THE DIFFERENCE IS THIS IS THE DIFFERENCE IS THIS IS DYNAMICALLY GENERATED USING THE DYNAMICALLY GENERATED USING THE DYNAMICALLY GENERATED USING THE LEARNING PROCESS AS OPPOSED TO LEARNING PROCESS AS OPPOSED TO LEARNING PROCESS AS OPPOSED TO THE EARLIER CASE WHERE SOMEBODY THE EARLIER CASE WHERE SOMEBODY THE EARLIER CASE WHERE SOMEBODY GIVES YOU A KNOWLEDGE GRAPH OR GIVES YOU A KNOWLEDGE GRAPH OR GIVES YOU A KNOWLEDGE GRAPH OR THE STRUCTURED SIGNALS OR YOU THE STRUCTURED SIGNALS OR YOU THE STRUCTURED SIGNALS OR YOU CAN STRIKE THEM FROM A DATA CAN STRIKE THEM FROM A DATA CAN STRIKE THEM FROM A DATA SOURCE. SOURCE. SOURCE. WHAT WE TRY TO DO HERE IS USING WHAT WE TRY TO DO HERE IS USING WHAT WE TRY TO DO HERE IS USING THIS STRUCTURE WE ALREADY KNOW THIS STRUCTURE WE ALREADY KNOW THIS STRUCTURE WE ALREADY KNOW THE LABEL FOR THE FIRST IMAGE IS THE LABEL FOR THE FIRST IMAGE IS THE LABEL FOR THE FIRST IMAGE IS A PANDA. A PANDA. A PANDA. WE FORCE THE NETWORK TO LEARN WE FORCE THE NETWORK TO LEARN WE FORCE THE NETWORK TO LEARN THAT THE THIRD IMAGE ALSO SHOULD THAT THE THIRD IMAGE ALSO SHOULD THAT THE THIRD IMAGE ALSO SHOULD BE CLASSIFIED AS A PANDA. BE CLASSIFIED AS A PANDA. BE CLASSIFIED AS A PANDA. THAT'S AT A VERY HIGH LEVEL OF THAT'S AT A VERY HIGH LEVEL OF THAT'S AT A VERY HIGH LEVEL OF HOW THIS WORKS. HOW THIS WORKS. HOW THIS WORKS. AGAIN, IF YOU WANT TO TRY THIS AGAIN, IF YOU WANT TO TRY THIS AGAIN, IF YOU WANT TO TRY THIS OUT FOR ANY NETWORK OR IN ANY OUT FOR ANY NETWORK OR IN ANY OUT FOR ANY NETWORK OR IN ANY APPLICATION, YOU CAN GO. APPLICATION, YOU CAN GO. APPLICATION, YOU CAN GO. THERE'S A TUTORIAL THAT ALLOWS THERE'S A TUTORIAL THAT ALLOWS THERE'S A TUTORIAL THAT ALLOWS YOU TO DO THIS LOOSELY API. YOU TO DO THIS LOOSELY API. YOU TO DO THIS LOOSELY API. NSL ENABLES BOTH THE GRAPH -- NSL ENABLES BOTH THE GRAPH -- NSL ENABLES BOTH THE GRAPH -- NEURAL GRAPH TYPE OF LEARNING NEURAL GRAPH TYPE OF LEARNING NEURAL GRAPH TYPE OF LEARNING AND THE ADVERSARIAL TYPE OF AND THE ADVERSARIAL TYPE OF AND THE ADVERSARIAL TYPE OF LEARNING, AND YOU CAN GO TO THE LEARNING, AND YOU CAN GO TO THE LEARNING, AND YOU CAN GO TO THE WEBSITE AND RUN THROUGH THE CODE WEBSITE AND RUN THROUGH THE CODE WEBSITE AND RUN THROUGH THE CODE EXAMPLE, ALL WITH JUST A FEW EXAMPLE, ALL WITH JUST A FEW EXAMPLE, ALL WITH JUST A FEW LINES OF CODE. LINES OF CODE. LINES OF CODE. YOU'LL SEE MORE DETAILS LATER IN YOU'LL SEE MORE DETAILS LATER IN YOU'LL SEE MORE DETAILS LATER IN THE TALK. THE TALK. THE TALK. THIS IS HOW WEPTD WE WOULD WANT THIS IS HOW WEPTD WE WOULD WANT THIS IS HOW WEPTD WE WOULD WANT THE FRAMEWORK LIKE NSL, AND THE THE FRAMEWORK LIKE NSL, AND THE THE FRAMEWORK LIKE NSL, AND THE POWER OF USING IT TO ENABLE POWER OF USING IT TO ENABLE POWER OF USING IT TO ENABLE MORROW BUST NETWORKS AND ALSO MORROW BUST NETWORKS AND ALSO MORROW BUST NETWORKS AND ALSO BUILD NETWORKS THAT CAN BE BUILD NETWORKS THAT CAN BE BUILD NETWORKS THAT CAN BE TRAINED WITH MINIMAL TRAINED WITH MINIMAL TRAINED WITH MINIMAL SUPERVISION. SUPERVISION. SUPERVISION. THESE ARE VERY, VERY HANDY WHEN THESE ARE VERY, VERY HANDY WHEN THESE ARE VERY, VERY HANDY WHEN YOU WANT TO BUILD, YOU KNOW, YOU WANT TO BUILD, YOU KNOW, YOU WANT TO BUILD, YOU KNOW, APPLICATIONS ON THE FLY, AND APPLICATIONS ON THE FLY, AND APPLICATIONS ON THE FLY, AND VERY, VERY CUSTOM APPLICATION VERY, VERY CUSTOM APPLICATION VERY, VERY CUSTOM APPLICATION THAT IS DO NOT SPREAD THE THAT IS DO NOT SPREAD THE THAT IS DO NOT SPREAD THE REGULAR MOLD. REGULAR MOLD. REGULAR MOLD. LET US DIVE DEEPER INTO HOW WE LET US DIVE DEEPER INTO HOW WE LET US DIVE DEEPER INTO HOW WE KNOW WHAT THE FRAMEWORK IS KNOW WHAT THE FRAMEWORK IS KNOW WHAT THE FRAMEWORK IS DOING. DOING. DOING. STRUCTURE IS GOING TO BE USED TO STRUCTURE IS GOING TO BE USED TO STRUCTURE IS GOING TO BE USED TO MODEL AND AS AN INPUT AND MODEL AND AS AN INPUT AND MODEL AND AS AN INPUT AND NETWORK. NETWORK. NETWORK. WE CALL THIS PARADIGM NEURAL WE CALL THIS PARADIGM NEURAL WE CALL THIS PARADIGM NEURAL GRAPH LEARNING. GRAPH LEARNING. GRAPH LEARNING. THE CORE IDEA IS THAT SHE MIGHT THE CORE IDEA IS THAT SHE MIGHT THE CORE IDEA IS THAT SHE MIGHT ASK, BY THE WAY, AT THIS POINT, ASK, BY THE WAY, AT THIS POINT, ASK, BY THE WAY, AT THIS POINT, LIKE, WHERE IS THE GRAPH COMING LIKE, WHERE IS THE GRAPH COMING LIKE, WHERE IS THE GRAPH COMING FROM IN THIS SETTING? FROM IN THIS SETTING? FROM IN THIS SETTING? IN SOME CASES IT MIGHT BE GIVEN IN SOME CASES IT MIGHT BE GIVEN IN SOME CASES IT MIGHT BE GIVEN TO YOU. TO YOU. TO YOU. YOU HAVE TO SAY IT WOULD PROVIDE YOU HAVE TO SAY IT WOULD PROVIDE YOU HAVE TO SAY IT WOULD PROVIDE TOOLS. TOOLS. TOOLS. AGAIN, YOU WILL HEAR MORE ABOUT AGAIN, YOU WILL HEAR MORE ABOUT AGAIN, YOU WILL HEAR MORE ABOUT THIS THAT ALLOW YOU TO CONSTRUCT THIS THAT ALLOW YOU TO CONSTRUCT THIS THAT ALLOW YOU TO CONSTRUCT THESE GRAPHS FROM, YOU KNOW, THESE GRAPHS FROM, YOU KNOW, THESE GRAPHS FROM, YOU KNOW, SOURCES OF DATA LIKE WHAT SOURCES OF DATA LIKE WHAT SOURCES OF DATA LIKE WHAT EMBEDDINGS AND IMAGE EMBEDDINGS. EMBEDDINGS AND IMAGE EMBEDDINGS. EMBEDDINGS AND IMAGE EMBEDDINGS. THE PHOTOGRAPH HERE IS THE THE PHOTOGRAPH HERE IS THE THE PHOTOGRAPH HERE IS THE NETWORK IS GOING TO BE FORCED TO NETWORK IS GOING TO BE FORCED TO NETWORK IS GOING TO BE FORCED TO JOINTLY OPTIMIZE BOTH THE JOINTLY OPTIMIZE BOTH THE JOINTLY OPTIMIZE BOTH THE FEATURE INPUT AND THE STRUCTURED FEATURE INPUT AND THE STRUCTURED FEATURE INPUT AND THE STRUCTURED SIGNAL SIMULTANEOUSLY. SIGNAL SIMULTANEOUSLY. SIGNAL SIMULTANEOUSLY. LET'S SEE HOW THAT HAPPENS, LET'S SEE HOW THAT HAPPENS, LET'S SEE HOW THAT HAPPENS, LIKE, DIVING IN DEEPER IF YOU LIKE, DIVING IN DEEPER IF YOU LIKE, DIVING IN DEEPER IF YOU WERE TRYING TO LOOK AT WHAT WERE TRYING TO LOOK AT WHAT WERE TRYING TO LOOK AT WHAT EXACTLY THE NETWORK IS LEARNING. EXACTLY THE NETWORK IS LEARNING. EXACTLY THE NETWORK IS LEARNING. EVERY NETWORK IS TRYING TO EVERY NETWORK IS TRYING TO EVERY NETWORK IS TRYING TO OPTIMIZE SOME LOSS. OPTIMIZE SOME LOSS. OPTIMIZE SOME LOSS. IN IMAGE CLASSIFICATION, WHAT IS IN IMAGE CLASSIFICATION, WHAT IS IN IMAGE CLASSIFICATION, WHAT IS THE LOSS WHEN YOU TAKE THE THE LOSS WHEN YOU TAKE THE THE LOSS WHEN YOU TAKE THE PIXELS, PASS THROUGH THE PIXELS, PASS THROUGH THE PIXELS, PASS THROUGH THE NETWORK, GET PREDICTIONS, WHERE NETWORK, GET PREDICTIONS, WHERE NETWORK, GET PREDICTIONS, WHERE IS THE ERROR BETWEEN THE IS THE ERROR BETWEEN THE IS THE ERROR BETWEEN THE PREDICTIONS AND THE TRUE LABEL. PREDICTIONS AND THE TRUE LABEL. PREDICTIONS AND THE TRUE LABEL. IN NSL, THE NEURAL GRAPH SETTING IN NSL, THE NEURAL GRAPH SETTING IN NSL, THE NEURAL GRAPH SETTING LEARNING, WE CALL THIS TRAINED LEARNING, WE CALL THIS TRAINED LEARNING, WE CALL THIS TRAINED IN THIS MODE NEURAL GRAPH IN THIS MODE NEURAL GRAPH IN THIS MODE NEURAL GRAPH MACHINE. MACHINE. MACHINE. WE TRY TO OPTIMIZE TWO WE TRY TO OPTIMIZE TWO WE TRY TO OPTIMIZE TWO COMPONENTS. COMPONENTS. COMPONENTS. ONE IS THE STANDARD LOSS, WHICH ONE IS THE STANDARD LOSS, WHICH ONE IS THE STANDARD LOSS, WHICH IS IMAGE CLASSIFICATIONS. IS IMAGE CLASSIFICATIONS. IS IMAGE CLASSIFICATIONS. THE LOSS INCURRED WHEN YOU PASS THE LOSS INCURRED WHEN YOU PASS THE LOSS INCURRED WHEN YOU PASS THE PIXELS THROUGH THE NETWORK THE PIXELS THROUGH THE NETWORK THE PIXELS THROUGH THE NETWORK AND THE PREDICTIONS ARE MEASURED AND THE PREDICTIONS ARE MEASURED AND THE PREDICTIONS ARE MEASURED IN THE ERROR. IN THE ERROR. IN THE ERROR. THE OTHER COMPONENT IS GOING TO THE OTHER COMPONENT IS GOING TO THE OTHER COMPONENT IS GOING TO BE BUSINESSED ON THE STRUCTURED BE BUSINESSED ON THE STRUCTURED BE BUSINESSED ON THE STRUCTURED SIGNAL OR THE GRAPH THAT YOU SIGNAL OR THE GRAPH THAT YOU SIGNAL OR THE GRAPH THAT YOU PROVIDED. PROVIDED. PROVIDED. WHERE THAT COMES IS IF I ARE AN WHERE THAT COMES IS IF I ARE AN WHERE THAT COMES IS IF I ARE AN IMAGE THAT LOOKED LIKE THE PIT IMAGE THAT LOOKED LIKE THE PIT IMAGE THAT LOOKED LIKE THE PIT BULL DOG, THAT'S LABELED AS A BULL DOG, THAT'S LABELED AS A BULL DOG, THAT'S LABELED AS A PIT BULL. PIT BULL. PIT BULL. IF I HAVE A DIFFERENT IMAGE, IF I HAVE A DIFFERENT IMAGE, IF I HAVE A DIFFERENT IMAGE, WHICH THROUGH MY STRUCTURED WHICH THROUGH MY STRUCTURED WHICH THROUGH MY STRUCTURED SIGNAL HAS AN EDGE WITH THE SIGNAL HAS AN EDGE WITH THE SIGNAL HAS AN EDGE WITH THE ORIGINAL IMAGE, THEN THE NETWORK ORIGINAL IMAGE, THEN THE NETWORK ORIGINAL IMAGE, THEN THE NETWORK IS FORCED TO LEARN THAT THE IS FORCED TO LEARN THAT THE IS FORCED TO LEARN THAT THE SOURCE IMAGE AND ITS NEIGHBOR IN SOURCE IMAGE AND ITS NEIGHBOR IN SOURCE IMAGE AND ITS NEIGHBOR IN THE GRAPH SHOULD LEARN SIMILAR THE GRAPH SHOULD LEARN SIMILAR THE GRAPH SHOULD LEARN SIMILAR REPRESENTATIONS. REPRESENTATIONS. REPRESENTATIONS. THAT MEANS YOU'RE TRYING TO SAY THAT MEANS YOU'RE TRYING TO SAY THAT MEANS YOU'RE TRYING TO SAY THAT -- NOW THIS IS FLEXIBLE, AS THAT -- NOW THIS IS FLEXIBLE, AS THAT -- NOW THIS IS FLEXIBLE, AS YOU CAN MABLG. YOU CAN MABLG. YOU CAN MABLG. YOU CAN USUALLY CHANGE THE YOU CAN USUALLY CHANGE THE YOU CAN USUALLY CHANGE THE FORMULATIONS. FORMULATIONS. FORMULATIONS. THAT MEANS INSTEAD OF A THAT MEANS INSTEAD OF A THAT MEANS INSTEAD OF A SUPERVISED LOGS, IF YOU WANT TO SUPERVISED LOGS, IF YOU WANT TO SUPERVISED LOGS, IF YOU WANT TO DO UNSUPERVISED LEARNING OR A DO UNSUPERVISED LEARNING OR A DO UNSUPERVISED LEARNING OR A VERY DIFFERENT KIND OF LOSS, YOU VERY DIFFERENT KIND OF LOSS, YOU VERY DIFFERENT KIND OF LOSS, YOU CAN ACTUALLY CHANGE THE FIRST CAN ACTUALLY CHANGE THE FIRST CAN ACTUALLY CHANGE THE FIRST COMPONENT YOU ARE CAN USE IT COMPONENT YOU ARE CAN USE IT COMPONENT YOU ARE CAN USE IT XEBDING ON THE APPLICATIONS. XEBDING ON THE APPLICATIONS. XEBDING ON THE APPLICATIONS. YOU KNOW, VERY, VERY EASY FOR YOU KNOW, VERY, VERY EASY FOR YOU KNOW, VERY, VERY EASY FOR YOU TO HAVE A WIDE RANGE OF YOU TO HAVE A WIDE RANGE OF YOU TO HAVE A WIDE RANGE OF APPLICATIONS IN A DIFFERENT APPLICATIONS IN A DIFFERENT APPLICATIONS IN A DIFFERENT LEARNING SETTING, LIKE WHETHER LEARNING SETTING, LIKE WHETHER LEARNING SETTING, LIKE WHETHER IT'S UNSUPERVISED, SUPERVISED, IT'S UNSUPERVISED, SUPERVISED, IT'S UNSUPERVISED, SUPERVISED, OR RANKING OR CLASSIFICATION, OR RANKING OR CLASSIFICATION, OR RANKING OR CLASSIFICATION, BUT AT THE SAME TIME YOU WOULD BUT AT THE SAME TIME YOU WOULD BUT AT THE SAME TIME YOU WOULD BE ABLE TO PASS INFRASTRUCTURE BE ABLE TO PASS INFRASTRUCTURE BE ABLE TO PASS INFRASTRUCTURE IN A SEAMLESS MANNER. IN A SEAMLESS MANNER. IN A SEAMLESS MANNER. HERE'S AN EXAMPLE. HERE'S AN EXAMPLE. HERE'S AN EXAMPLE. TAKES IMAGE CLASSIFICATION WITH TAKES IMAGE CLASSIFICATION WITH TAKES IMAGE CLASSIFICATION WITH SOME SAMPLES, AS I SAID, THE SOME SAMPLES, AS I SAID, THE SOME SAMPLES, AS I SAID, THE PIXELS, AND YOU HAVE A PIXELS, AND YOU HAVE A PIXELS, AND YOU HAVE A STRUCTURE. STRUCTURE. STRUCTURE. IN THIS CASE, THE IMAGES ARE IN THIS CASE, THE IMAGES ARE IN THIS CASE, THE IMAGES ARE CONNECTED IN THE GRAPH BASED ON CONNECTED IN THE GRAPH BASED ON CONNECTED IN THE GRAPH BASED ON SOME USER AND TRAFFIC SIGNAL OR SOME USER AND TRAFFIC SIGNAL OR SOME USER AND TRAFFIC SIGNAL OR BASICALLY AN EXAMPLE AS I SHOWED BASICALLY AN EXAMPLE AS I SHOWED BASICALLY AN EXAMPLE AS I SHOWED YOU, LIKE, THEY BELONG TO THE YOU, LIKE, THEY BELONG TO THE YOU, LIKE, THEY BELONG TO THE SAME ALBUM OR THERE'S SOME SORT SAME ALBUM OR THERE'S SOME SORT SAME ALBUM OR THERE'S SOME SORT OF STRUCTURE TYING THEM OF STRUCTURE TYING THEM OF STRUCTURE TYING THEM TOGETHER. TOGETHER. TOGETHER. AS YOU MEAN, THIS IS GIVEN TO AS YOU MEAN, THIS IS GIVEN TO AS YOU MEAN, THIS IS GIVEN TO YOU. YOU. YOU. WE PASS THIS THROUGH THE WE PASS THIS THROUGH THE WE PASS THIS THROUGH THE NETWORK. NETWORK. NETWORK. BOTH THIS AND IT'S NEIGHBORS ARE BOTH THIS AND IT'S NEIGHBORS ARE BOTH THIS AND IT'S NEIGHBORS ARE PASSED SIMULTANEOUSLY THROUGH PASSED SIMULTANEOUSLY THROUGH PASSED SIMULTANEOUSLY THROUGH THE SAME NETWORK, AND THE THE SAME NETWORK, AND THE THE SAME NETWORK, AND THE NETWORK IS LEARNING TO OPTIMIZE NETWORK IS LEARNING TO OPTIMIZE NETWORK IS LEARNING TO OPTIMIZE WITHIN, YOU KNOW, EACH LAYER, WITHIN, YOU KNOW, EACH LAYER, WITHIN, YOU KNOW, EACH LAYER, AND THIS IS ALSO CONFIGURED, BY AND THIS IS ALSO CONFIGURED, BY AND THIS IS ALSO CONFIGURED, BY THE WAY, TO PUSH THE EMBEDDING THE WAY, TO PUSH THE EMBEDDING THE WAY, TO PUSH THE EMBEDDING FOR NEIGHBORS CLOSER TO EACH FOR NEIGHBORS CLOSER TO EACH FOR NEIGHBORS CLOSER TO EACH OTHER. OTHER. OTHER. THAT MEANS TWO IMAGES THAT ARE THAT MEANS TWO IMAGES THAT ARE THAT MEANS TWO IMAGES THAT ARE CONNECTED IN THE GRAPH SMUD CONNECTED IN THE GRAPH SMUD CONNECTED IN THE GRAPH SMUD LEARN SIMILAR EMBEDDINGS WHEN LEARN SIMILAR EMBEDDINGS WHEN LEARN SIMILAR EMBEDDINGS WHEN PASSED THROUGH THE NETWORK. PASSED THROUGH THE NETWORK. PASSED THROUGH THE NETWORK. SIMULTANEOUSY, YOU SHOULD ALSO SIMULTANEOUSY, YOU SHOULD ALSO SIMULTANEOUSY, YOU SHOULD ALSO OPTIMIZE THAT THEY SHOULD LEARN OPTIMIZE THAT THEY SHOULD LEARN OPTIMIZE THAT THEY SHOULD LEARN THE CORRECT PREDICTION, SO ONE THE CORRECT PREDICTION, SO ONE THE CORRECT PREDICTION, SO ONE OF THEM WAS LABEL AS A TIME, AND OF THEM WAS LABEL AS A TIME, AND OF THEM WAS LABEL AS A TIME, AND YOU ALSO WANT THE PREDICTION YOU ALSO WANT THE PREDICTION YOU ALSO WANT THE PREDICTION ERROR TO BE MINIMAL. ERROR TO BE MINIMAL. ERROR TO BE MINIMAL. BOTH HAVE THESE ARE BEING BOTH HAVE THESE ARE BEING BOTH HAVE THESE ARE BEING OPTIMIZED JOINTLY. OPTIMIZED JOINTLY. OPTIMIZED JOINTLY. HOPEFULLY THIS GIVES YOU AN IDEA HOPEFULLY THIS GIVES YOU AN IDEA HOPEFULLY THIS GIVES YOU AN IDEA OF HOW WE USE GRAPH LEARNING OR OF HOW WE USE GRAPH LEARNING OR OF HOW WE USE GRAPH LEARNING OR NEURAL GRAPH LEARNING, AND NEURAL GRAPH LEARNING, AND NEURAL GRAPH LEARNING, AND ENABLE THIS STRUCTURED LEARNING ENABLE THIS STRUCTURED LEARNING ENABLE THIS STRUCTURED LEARNING FRAMEWORK. FRAMEWORK. FRAMEWORK. AS I MENTIONED, STRUCTURE CAN AS I MENTIONED, STRUCTURE CAN AS I MENTIONED, STRUCTURE CAN COME IN DIFFERENT FORMS. COME IN DIFFERENT FORMS. COME IN DIFFERENT FORMS. THAT WAS AN EXPLICIT STRUCTURE THAT WAS AN EXPLICIT STRUCTURE THAT WAS AN EXPLICIT STRUCTURE AS A GRAPH INPUT, BUT WE CAN AS A GRAPH INPUT, BUT WE CAN AS A GRAPH INPUT, BUT WE CAN ALSO DO IMPLICIT STRUCTURES, AND ALSO DO IMPLICIT STRUCTURES, AND ALSO DO IMPLICIT STRUCTURES, AND THIS IS WHERE THE ADVERSARIAL THIS IS WHERE THE ADVERSARIAL THIS IS WHERE THE ADVERSARIAL LEARNING TYPE OF PARADIGMS ARE LEARNING TYPE OF PARADIGMS ARE LEARNING TYPE OF PARADIGMS ARE ENABLED USING THE NSL FRAMEWORK. ENABLED USING THE NSL FRAMEWORK. ENABLED USING THE NSL FRAMEWORK. YOU CREATE NXI PRIME AS AN YOU CREATE NXI PRIME AS AN YOU CREATE NXI PRIME AS AN ADVERSARIAL VERSION OF THAT. ADVERSARIAL VERSION OF THAT. ADVERSARIAL VERSION OF THAT. THERE IS SOME SORT OF WAIT BASED THERE IS SOME SORT OF WAIT BASED THERE IS SOME SORT OF WAIT BASED ON THE BUSINESS, AND THIS IS ON THE BUSINESS, AND THIS IS ON THE BUSINESS, AND THIS IS PASSED THROUGH THE NETWORK, AND PASSED THROUGH THE NETWORK, AND PASSED THROUGH THE NETWORK, AND THE NETWORK IS FORCED TO WANT ON THE NETWORK IS FORCED TO WANT ON THE NETWORK IS FORCED TO WANT ON MIZ BOTH OF THEM TO THE SAME MIZ BOTH OF THEM TO THE SAME MIZ BOTH OF THEM TO THE SAME EMBEDDINGS AND REPRESENTATIONS EMBEDDINGS AND REPRESENTATIONS EMBEDDINGS AND REPRESENTATIONS INSIDE. INSIDE. INSIDE. THESE ARE ALL GREAT. THESE ARE ALL GREAT. THESE ARE ALL GREAT. AS I MENTIONED, THIS SORT OF AS I MENTIONED, THIS SORT OF AS I MENTIONED, THIS SORT OF OPENS UP A HOST OF NEW KIND OF OPENS UP A HOST OF NEW KIND OF OPENS UP A HOST OF NEW KIND OF APPLICATION OR TRAINING CENTERS. APPLICATION OR TRAINING CENTERS. APPLICATION OR TRAINING CENTERS. THE BEST PART ABOUT THIS IS IF THE BEST PART ABOUT THIS IS IF THE BEST PART ABOUT THIS IS IF YOU ARE THINKING, OH, NOW HOW YOU ARE THINKING, OH, NOW HOW YOU ARE THINKING, OH, NOW HOW DOES THIS WORK WITH TRANSFORMERS DOES THIS WORK WITH TRANSFORMERS DOES THIS WORK WITH TRANSFORMERS OR DIFFERENT KINDS OF NETWORK OR DIFFERENT KINDS OF NETWORK OR DIFFERENT KINDS OF NETWORK STRUCTURE DOESN'T MATTER HERE. STRUCTURE DOESN'T MATTER HERE. STRUCTURE DOESN'T MATTER HERE. YOU CAN USE THIS WITH ANY TYPE YOU CAN USE THIS WITH ANY TYPE YOU CAN USE THIS WITH ANY TYPE OF NETWORK STRUCTURE. OF NETWORK STRUCTURE. OF NETWORK STRUCTURE. THAT'S THE BEST PART. THAT'S THE BEST PART. THAT'S THE BEST PART. RNNs, TRANSFORMERS, LIKE -- YOU RNNs, TRANSFORMERS, LIKE -- YOU RNNs, TRANSFORMERS, LIKE -- YOU HAVE A COMBINATIONS. HAVE A COMBINATIONS. HAVE A COMBINATIONS. IT DOESN'T MATTER. IT DOESN'T MATTER. IT DOESN'T MATTER. THESE ARE SORT OF A LEARNING THESE ARE SORT OF A LEARNING THESE ARE SORT OF A LEARNING STRATEGY WHERE YOU CAN ACTUALLY STRATEGY WHERE YOU CAN ACTUALLY STRATEGY WHERE YOU CAN ACTUALLY BUILD A NETWORK AND ENABLE NSL BUILD A NETWORK AND ENABLE NSL BUILD A NETWORK AND ENABLE NSL WORKING ADVERSARIAL AND THE WORKING ADVERSARIAL AND THE WORKING ADVERSARIAL AND THE NEURALRAPH SETTING VERY EASILY NEURALRAPH SETTING VERY EASILY NEURALRAPH SETTING VERY EASILY WITH VERY FEW LINES OF CODES IN WITH VERY FEW LINES OF CODES IN WITH VERY FEW LINES OF CODES IN 2.0, AND TO TELL YOU MORE ABOUT 2.0, AND TO TELL YOU MORE ABOUT 2.0, AND TO TELL YOU MORE ABOUT THAT I'M HANDING IT OVER THAT I'M HANDING IT OVER THAT I'M HANDING IT OVER >> ALL RIGHT. >> ALL RIGHT. >> ALL RIGHT. THANK YOU, SUJIT. THANK YOU, SUJIT. THANK YOU, SUJIT. NEXT, WE ARE GOING TO INTRODUCE NEXT, WE ARE GOING TO INTRODUCE NEXT, WE ARE GOING TO INTRODUCE THE LIBRARY TOOLS AND TRAINERS THE LIBRARY TOOLS AND TRAINERS THE LIBRARY TOOLS AND TRAINERS PROVIDED BY THE NEURAL PROVIDED BY THE NEURAL PROVIDED BY THE NEURAL STRUCTURED LEARNING FRAME WORLD STRUCTURED LEARNING FRAME WORLD STRUCTURED LEARNING FRAME WORLD WAR I. WAR I. WAR I. EVERYTHING HERE IS COMPATIBLE EVERYTHING HERE IS COMPATIBLE EVERYTHING HERE IS COMPATIBLE WITH TENSORFLOW 2.0. WITH TENSORFLOW 2.0. WITH TENSORFLOW 2.0. THIS IS THE TRAINING WORKFLOW WE THIS IS THE TRAINING WORKFLOW WE THIS IS THE TRAINING WORKFLOW WE JUST MENTIONED PREVIOUSLY. JUST MENTIONED PREVIOUSLY. JUST MENTIONED PREVIOUSLY. EVERY SEGMENT IN RED HERE IS A EVERY SEGMENT IN RED HERE IS A EVERY SEGMENT IN RED HERE IS A NEW STAT INTRODUCED TO THE NEW STAT INTRODUCED TO THE NEW STAT INTRODUCED TO THE WORKFLOW TO TRAIN WITH STRUCTURE WORKFLOW TO TRAIN WITH STRUCTURE WORKFLOW TO TRAIN WITH STRUCTURE SIGNALS. SIGNALS. SIGNALS. NEURAL STRUCTURE LEARNING NEURAL STRUCTURE LEARNING NEURAL STRUCTURE LEARNING PROVIDE LIBRARIES AND TOOLS FOR PROVIDE LIBRARIES AND TOOLS FOR PROVIDE LIBRARIES AND TOOLS FOR THESE SETS. THESE SETS. THESE SETS. LET'S FIRST TAKE A LOOK AT THE LET'S FIRST TAKE A LOOK AT THE LET'S FIRST TAKE A LOOK AT THE LAB PART OF THE WORKFLOW. LAB PART OF THE WORKFLOW. LAB PART OF THE WORKFLOW. THE TRAINING SAMPLES IN THE TRAINING SAMPLES IN THE TRAINING SAMPLES IN NEIGHBORS FROM THE SAME NEIGHBORS FROM THE SAME NEIGHBORS FROM THE SAME NEIGHBORHOOD ARE PACKED TO FORM NEIGHBORHOOD ARE PACKED TO FORM NEIGHBORHOOD ARE PACKED TO FORM THE NEW BATCH. THE NEW BATCH. THE NEW BATCH. NOTICE THAT IN THE BATCH EACH NOTICE THAT IN THE BATCH EACH NOTICE THAT IN THE BATCH EACH TRAINING SAMPLE IS EXTENDED TO TRAINING SAMPLE IS EXTENDED TO TRAINING SAMPLE IS EXTENDED TO INCLUDE THE NEIGHBORHOOD INCLUDE THE NEIGHBORHOOD INCLUDE THE NEIGHBORHOOD INFORMATION. INFORMATION. INFORMATION. TO ACHIEVE THIS IN THE NEURAL TO ACHIEVE THIS IN THE NEURAL TO ACHIEVE THIS IN THE NEURAL STRUCTURE LEARNING FRAMEWORK, WE STRUCTURE LEARNING FRAMEWORK, WE STRUCTURE LEARNING FRAMEWORK, WE PROVIDE TOOLS SUCH AS GRAPH AND PROVIDE TOOLS SUCH AS GRAPH AND PROVIDE TOOLS SUCH AS GRAPH AND PACK NEIGHBORS THAT A USING PACK NEIGHBORS THAT A USING PACK NEIGHBORS THAT A USING COULD USE DIRECTLY. COULD USE DIRECTLY. COULD USE DIRECTLY. WE ALSO PROVIDE FUNCTIONS THAT WE ALSO PROVIDE FUNCTIONS THAT WE ALSO PROVIDE FUNCTIONS THAT USERS COULD INTEGRATE INTO THEIR USERS COULD INTEGRATE INTO THEIR USERS COULD INTEGRATE INTO THEIR OWN CUSTOM PIPE LOOIRN, AND YOU OWN CUSTOM PIPE LOOIRN, AND YOU OWN CUSTOM PIPE LOOIRN, AND YOU MAY NOTICE NEW GRAPH AND PACK MAY NOTICE NEW GRAPH AND PACK MAY NOTICE NEW GRAPH AND PACK NEIGHBORS HERE ARE LISTED BOTH NEIGHBORS HERE ARE LISTED BOTH NEIGHBORS HERE ARE LISTED BOTH AS FINERIES AND FUNCTIONS. AS FINERIES AND FUNCTIONS. AS FINERIES AND FUNCTIONS. THIS IS NOT A TYPO. THIS IS NOT A TYPO. THIS IS NOT A TYPO. THIS MEANS THAT THEY COULD BE THIS MEANS THAT THEY COULD BE THIS MEANS THAT THEY COULD BE INVOLVED BOTH AS BINARY OR AS A INVOLVED BOTH AS BINARY OR AS A INVOLVED BOTH AS BINARY OR AS A FUNCTION. FUNCTION. FUNCTION. NEXT, LET'S TAKE A LOOK AT A NEXT, LET'S TAKE A LOOK AT A NEXT, LET'S TAKE A LOOK AT A RIGHT PART OF THE FIGURE. RIGHT PART OF THE FIGURE. RIGHT PART OF THE FIGURE. AGAIN, WE PROVIDE LIBRARIES FOR AGAIN, WE PROVIDE LIBRARIES FOR AGAIN, WE PROVIDE LIBRARIES FOR THESE NEW STATS INTRODUCED IN THESE NEW STATS INTRODUCED IN THESE NEW STATS INTRODUCED IN GRAPH -- BOTH THE TRAINING GRAPH -- BOTH THE TRAINING GRAPH -- BOTH THE TRAINING SAMPLE AND ITS NEIGHBOR WILL BE SAMPLE AND ITS NEIGHBOR WILL BE SAMPLE AND ITS NEIGHBOR WILL BE FED TO THE NEURAL NETWORK. FED TO THE NEURAL NETWORK. FED TO THE NEURAL NETWORK. IN ALL PACK NEIGHBOR FEATURES IS IN ALL PACK NEIGHBOR FEATURES IS IN ALL PACK NEIGHBOR FEATURES IS FOR THIS PURPOSE. FOR THIS PURPOSE. FOR THIS PURPOSE. IT CAN BE ANY TYPE OF NEURAL IT CAN BE ANY TYPE OF NEURAL IT CAN BE ANY TYPE OF NEURAL NETWORK NOT JUST LIMITED TO THE NETWORK NOT JUST LIMITED TO THE NETWORK NOT JUST LIMITED TO THE COMMERCIAL. COMMERCIAL. COMMERCIAL. THE DIFFERENCE BETWEEN THE THE DIFFERENCE BETWEEN THE THE DIFFERENCE BETWEEN THE SAMPLE AND ITS NEIGHBOR IS SAMPLE AND ITS NEIGHBOR IS SAMPLE AND ITS NEIGHBOR IS CALCULATED AND ADDED TO THE CALCULATED AND ADDED TO THE CALCULATED AND ADDED TO THE FINAL LOSS. FINAL LOSS. FINAL LOSS. IN ADDITION, WE ALSO PROVIDE IN ADDITION, WE ALSO PROVIDE IN ADDITION, WE ALSO PROVIDE LIBRARIES TO GENERATE THE LIBRARIES TO GENERATE THE LIBRARIES TO GENERATE THE ADVERSARIAL NEIGHBORS AS ADVERSARIAL NEIGHBORS AS ADVERSARIAL NEIGHBORS AS IMPLICIT STRUCTURE SIGNALS. IMPLICIT STRUCTURE SIGNALS. IMPLICIT STRUCTURE SIGNALS. FINALLY, WE ALSO PROVIDE SDPT FINALLY, WE ALSO PROVIDE SDPT FINALLY, WE ALSO PROVIDE SDPT ALL THREE TYPES OF MODELS EITHER ALL THREE TYPES OF MODELS EITHER ALL THREE TYPES OF MODELS EITHER VIA SEE CONTROVERSIAL OR VIA SEE CONTROVERSIAL OR VIA SEE CONTROVERSIAL OR FUNCTIONAL API OR VIA FUNCTIONAL API OR VIA FUNCTIONAL API OR VIA SUBCLASSING. SUBCLASSING. SUBCLASSING. THE FIRST STEP, IF YOU WANT TO THE FIRST STEP, IF YOU WANT TO THE FIRST STEP, IF YOU WANT TO USE NEURAL STRURGTAL LEARNING IS USE NEURAL STRURGTAL LEARNING IS USE NEURAL STRURGTAL LEARNING IS TO DO A PIT INSTALL. TO DO A PIT INSTALL. TO DO A PIT INSTALL. HERE WE PROVIDE A CODE EXAMPLE HERE WE PROVIDE A CODE EXAMPLE HERE WE PROVIDE A CODE EXAMPLE ILLUSTRATING THE API FROM THE ILLUSTRATING THE API FROM THE ILLUSTRATING THE API FROM THE NEURAL STRUCTURAL LEARNING NEURAL STRUCTURAL LEARNING NEURAL STRUCTURAL LEARNING LIBRARY. LIBRARY. LIBRARY. WE FIRST NEED TO READ THE WE FIRST NEED TO READ THE WE FIRST NEED TO READ THE TRAINING DATA. TRAINING DATA. TRAINING DATA. NOTE THAT THE DATA HERE ARE NOTE THAT THE DATA HERE ARE NOTE THAT THE DATA HERE ARE PROCESSED BY THE TOOLS OR PROCESSED BY THE TOOLS OR PROCESSED BY THE TOOLS OR FUNCTIONS TO INCORPORATE THE FUNCTIONS TO INCORPORATE THE FUNCTIONS TO INCORPORATE THE GRAPHS INTO TRAINING SAMPLES. GRAPHS INTO TRAINING SAMPLES. GRAPHS INTO TRAINING SAMPLES. NEXT, THE USER COULD BUILD NEXT, THE USER COULD BUILD NEXT, THE USER COULD BUILD CUSTOM MODELS, SUCH AS -- AND CUSTOM MODELS, SUCH AS -- AND CUSTOM MODELS, SUCH AS -- AND TREAT IT AS THE BASE MODEL. TREAT IT AS THE BASE MODEL. TREAT IT AS THE BASE MODEL. USING -- UT USER COULD BUILD USING -- UT USER COULD BUILD USING -- UT USER COULD BUILD THIS MODEL, THE BASE MODEL, USE THIS MODEL, THE BASE MODEL, USE THIS MODEL, THE BASE MODEL, USE ANY OF THEIR FAVORITE CURRENT ANY OF THEIR FAVORITE CURRENT ANY OF THEIR FAVORITE CURRENT API, LIKE WE JUST MENTIONED. API, LIKE WE JUST MENTIONED. API, LIKE WE JUST MENTIONED. SEE CONTROVERSIAL, FUNCTIONAL, SEE CONTROVERSIAL, FUNCTIONAL, SEE CONTROVERSIAL, FUNCTIONAL, OR SUBCLASING. OR SUBCLASING. OR SUBCLASING. AFTER THE BASE MODEL IS BUILT, AFTER THE BASE MODEL IS BUILT, AFTER THE BASE MODEL IS BUILT, WE USE THE API TO WRAP AROUND WE USE THE API TO WRAP AROUND WE USE THE API TO WRAP AROUND THE BASE MODEL TO ENABLE THE THE BASE MODEL TO ENABLE THE THE BASE MODEL TO ENABLE THE GRAPH REGULARIZATION. GRAPH REGULARIZATION. GRAPH REGULARIZATION. THERE ARE PERIMETERS WE NEED TO THERE ARE PERIMETERS WE NEED TO THERE ARE PERIMETERS WE NEED TO CONFIGURE. CONFIGURE. CONFIGURE. FOR EXAMPLE, WE NEED TO SPECIFY FOR EXAMPLE, WE NEED TO SPECIFY FOR EXAMPLE, WE NEED TO SPECIFY THE MAXIMUM NUMBER OF NEIGHBORS THE MAXIMUM NUMBER OF NEIGHBORS THE MAXIMUM NUMBER OF NEIGHBORS CONSIDERED DURING THE CONSIDERED DURING THE CONSIDERED DURING THE REGULARIZATION. REGULARIZATION. REGULARIZATION. ALSO, FOR EACH TYPE OF PROGRAM ALSO, FOR EACH TYPE OF PROGRAM ALSO, FOR EACH TYPE OF PROGRAM METER, WE PROVIDE DEEPER VALUES METER, WE PROVIDE DEEPER VALUES METER, WE PROVIDE DEEPER VALUES SET TO A CERTAIN NUMBER THAT WE SET TO A CERTAIN NUMBER THAT WE SET TO A CERTAIN NUMBER THAT WE KNOW IMPEERICLY THEY WORK WELL. KNOW IMPEERICLY THEY WORK WELL. KNOW IMPEERICLY THEY WORK WELL. AFTER WE ENABLE THE GRAPH, THE AFTER WE ENABLE THE GRAPH, THE AFTER WE ENABLE THE GRAPH, THE NEXT IS TO EXTEND OUR WORKFLOW, NEXT IS TO EXTEND OUR WORKFLOW, NEXT IS TO EXTEND OUR WORKFLOW, COMPILE, FIT, AND THEN EVAL. COMPILE, FIT, AND THEN EVAL. COMPILE, FIT, AND THEN EVAL. THAT'S IT. THAT'S IT. THAT'S IT. WITHIN FIVE LINES, WE ARE ABLE WITHIN FIVE LINES, WE ARE ABLE WITHIN FIVE LINES, WE ARE ABLE TO ENABLE GRAPH REGULARIZATION, TO ENABLE GRAPH REGULARIZATION, TO ENABLE GRAPH REGULARIZATION, AND IN FIVE LINES ACTUALLY AND IN FIVE LINES ACTUALLY AND IN FIVE LINES ACTUALLY INCLUDE ONE LINE AS A TOP. INCLUDE ONE LINE AS A TOP. INCLUDE ONE LINE AS A TOP. NOT AN ACTUAL LOGIC NOT AN ACTUAL LOGIC NOT AN ACTUAL LOGIC IMPLEMENTATION. IMPLEMENTATION. IMPLEMENTATION. HERE LET US SHOW SOME RESULTS OF HERE LET US SHOW SOME RESULTS OF HERE LET US SHOW SOME RESULTS OF A MODEL TREND WITH STRUCTURE A MODEL TREND WITH STRUCTURE A MODEL TREND WITH STRUCTURE SIGNALS. SIGNALS. SIGNALS. THE TASK IS TO CONDUCT A THE TASK IS TO CONDUCT A THE TASK IS TO CONDUCT A SENTIMENT ANALYSIS ON THE IMBB SENTIMENT ANALYSIS ON THE IMBB SENTIMENT ANALYSIS ON THE IMBB MANUFACTURE REVIEWS. MANUFACTURE REVIEWS. MANUFACTURE REVIEWS. WE WANT TO POINT OUT THAT THIS WE WANT TO POINT OUT THAT THIS WE WANT TO POINT OUT THAT THIS RESULT IS JUST FROM ONE OF OUR RESULT IS JUST FROM ONE OF OUR RESULT IS JUST FROM ONE OF OUR INTERNAL EXPERIMENTS. INTERNAL EXPERIMENTS. INTERNAL EXPERIMENTS. YOUR ACTUAL KNOWLEDGE MAY VARY YOUR ACTUAL KNOWLEDGE MAY VARY YOUR ACTUAL KNOWLEDGE MAY VARY FROM TASK TO TASK, TO DATA TO FROM TASK TO TASK, TO DATA TO FROM TASK TO TASK, TO DATA TO DATA OR MODEL TO MODEL. DATA OR MODEL TO MODEL. DATA OR MODEL TO MODEL. THE X EXIT HERE REPRESENTS THE THE X EXIT HERE REPRESENTS THE THE X EXIT HERE REPRESENTS THE AMOUNT OF SUPERVISION. AMOUNT OF SUPERVISION. AMOUNT OF SUPERVISION. WHICH COULD BE CONVERTED INTO WHICH COULD BE CONVERTED INTO WHICH COULD BE CONVERTED INTO THE AMOUNT OF LABEL EXAMPLES. THE AMOUNT OF LABEL EXAMPLES. THE AMOUNT OF LABEL EXAMPLES. AND THE Y EXIT HERE REPRESENTS AND THE Y EXIT HERE REPRESENTS AND THE Y EXIT HERE REPRESENTS THE MODEL ACCURACY. THE MODEL ACCURACY. THE MODEL ACCURACY. THE LAP FEATURE SHOWS THE THE LAP FEATURE SHOWS THE THE LAP FEATURE SHOWS THE PERFORMANCE OF A BIDIRECTIONAL PERFORMANCE OF A BIDIRECTIONAL PERFORMANCE OF A BIDIRECTIONAL DM, AND THE RIGHT FIGURE SHOWS DM, AND THE RIGHT FIGURE SHOWS DM, AND THE RIGHT FIGURE SHOWS THE PERFORMANCE OF A -- AS YOU THE PERFORMANCE OF A -- AS YOU THE PERFORMANCE OF A -- AS YOU CAN SEE, WHEN THERE -- WHEN WE CAN SEE, WHEN THERE -- WHEN WE CAN SEE, WHEN THERE -- WHEN WE HAVE LOTS OF TRAINING EXAMPLES, HAVE LOTS OF TRAINING EXAMPLES, HAVE LOTS OF TRAINING EXAMPLES, WHEN THE AMOUNT OF SUPERVISION WHEN THE AMOUNT OF SUPERVISION WHEN THE AMOUNT OF SUPERVISION IS HIGH, THERE IS NOT MUCH IS HIGH, THERE IS NOT MUCH IS HIGH, THERE IS NOT MUCH PERFORMANCE DIFFERENCE. PERFORMANCE DIFFERENCE. PERFORMANCE DIFFERENCE. AS SOON AS THE AMOUNT OF AS SOON AS THE AMOUNT OF AS SOON AS THE AMOUNT OF SUPERVISION DROPPED TO 5% OR SUPERVISION DROPPED TO 5% OR SUPERVISION DROPPED TO 5% OR EVEN 1%, TRAINING WITH STRUCTURE EVEN 1%, TRAINING WITH STRUCTURE EVEN 1%, TRAINING WITH STRUCTURE SIGNALS LEAD TO MORE ACCURATE SIGNALS LEAD TO MORE ACCURATE SIGNALS LEAD TO MORE ACCURATE MODELS, USUALLY BY THE MODELS, USUALLY BY THE MODELS, USUALLY BY THE IMPROVEMENT THAT'S MORE THAN IMPROVEMENT THAT'S MORE THAN IMPROVEMENT THAT'S MORE THAN 10%. 10%. 10%. >> WHAT SHOULD WE DO? >> WHAT SHOULD WE DO? >> WHAT SHOULD WE DO? NEURAL STRUCTURED LEARNING NEURAL STRUCTURED LEARNING NEURAL STRUCTURED LEARNING PROVIDES TWO METHODS. PROVIDES TWO METHODS. PROVIDES TWO METHODS. THE FIRST ONE IS TO CONSTRUCT THE FIRST ONE IS TO CONSTRUCT THE FIRST ONE IS TO CONSTRUCT THE GRAPH OR THE CONSTRUCT THE THE GRAPH OR THE CONSTRUCT THE THE GRAPH OR THE CONSTRUCT THE STRUCTURE VIA DATA PROCESSING, STRUCTURE VIA DATA PROCESSING, STRUCTURE VIA DATA PROCESSING, AND THE SECOND ONE IS TO AND THE SECOND ONE IS TO AND THE SECOND ONE IS TO CONSTRUCT SUCH STRUCTURE VIA CONSTRUCT SUCH STRUCTURE VIA CONSTRUCT SUCH STRUCTURE VIA ADVERSARIAL NEIGHBORS. ADVERSARIAL NEIGHBORS. ADVERSARIAL NEIGHBORS. LET'S FOCUS ON THE DATA LET'S FOCUS ON THE DATA LET'S FOCUS ON THE DATA PROCESSING ONE FIRST. PROCESSING ONE FIRST. PROCESSING ONE FIRST. AGAIN, LET'S TAKE DOCUMENT AGAIN, LET'S TAKE DOCUMENT AGAIN, LET'S TAKE DOCUMENT CLASSIFICATION AS AN EXAMPLE. CLASSIFICATION AS AN EXAMPLE. CLASSIFICATION AS AN EXAMPLE. GIVEN THE SAMPLE DOCUMENT, HOW GIVEN THE SAMPLE DOCUMENT, HOW GIVEN THE SAMPLE DOCUMENT, HOW DO WE KNOW IF ANOTHER DOCUMENT DO WE KNOW IF ANOTHER DOCUMENT DO WE KNOW IF ANOTHER DOCUMENT IS SIMILAR ENOUGH TO BE THE IS SIMILAR ENOUGH TO BE THE IS SIMILAR ENOUGH TO BE THE NEIGHBOR DOCUMENT? NEIGHBOR DOCUMENT? NEIGHBOR DOCUMENT? THESE DOCUMENTS WILL BE THESE DOCUMENTS WILL BE THESE DOCUMENTS WILL BE PROJECTED TO THE EMBEDDING PROJECTED TO THE EMBEDDING PROJECTED TO THE EMBEDDING SPACE. SPACE. SPACE. FOR EXAMPLE, WE COULD USE THE FOR EXAMPLE, WE COULD USE THE FOR EXAMPLE, WE COULD USE THE PRETREND BIRD EMBEDDING. PRETREND BIRD EMBEDDING. PRETREND BIRD EMBEDDING. THAT'S MENTIONED IN THE EARLIER THAT'S MENTIONED IN THE EARLIER THAT'S MENTIONED IN THE EARLIER ENS E TENSORFLOW TALK, AND TO ENS E TENSORFLOW TALK, AND TO ENS E TENSORFLOW TALK, AND TO GET ALL THESE DOCUMENTS GET ALL THESE DOCUMENTS GET ALL THESE DOCUMENTS EMBEDDED. EMBEDDED. EMBEDDED. DOCUMENTS THAT ARE CLOSER IN THE DOCUMENTS THAT ARE CLOSER IN THE DOCUMENTS THAT ARE CLOSER IN THE EMBEDDING SPACE WITH ASSUMED TO EMBEDDING SPACE WITH ASSUMED TO EMBEDDING SPACE WITH ASSUMED TO HAVE SIMILAR SCHEMATICS. HAVE SIMILAR SCHEMATICS. HAVE SIMILAR SCHEMATICS. NEXT, WE EXAMINE THE SIMILARITY NEXT, WE EXAMINE THE SIMILARITY NEXT, WE EXAMINE THE SIMILARITY BETWEEN TWO EMBEDS, PRECISE BETWEEN TWO EMBEDS, PRECISE BETWEEN TWO EMBEDS, PRECISE SIMILARITY OR OTHER METRIC TO BE SIMILARITY OR OTHER METRIC TO BE SIMILARITY OR OTHER METRIC TO BE USED HERE. USED HERE. USED HERE. IF THE SIMILARITY IS HIGHER THAN IF THE SIMILARITY IS HIGHER THAN IF THE SIMILARITY IS HIGHER THAN THE PREDEFINED THRESHOLD, WE THE PREDEFINED THRESHOLD, WE THE PREDEFINED THRESHOLD, WE TREAT THESE TWO DOCUMENTS ARE TREAT THESE TWO DOCUMENTS ARE TREAT THESE TWO DOCUMENTS ARE SIMILAR ENOUGH, AND, THEREFORE, SIMILAR ENOUGH, AND, THEREFORE, SIMILAR ENOUGH, AND, THEREFORE, WE ADD AN EDGE BETWEEN THESE TWO WE ADD AN EDGE BETWEEN THESE TWO WE ADD AN EDGE BETWEEN THESE TWO DOCUMENTS TO MAKE THEM DOCUMENTS TO MAKE THEM DOCUMENTS TO MAKE THEM NEIGHBORS. NEIGHBORS. NEIGHBORS. BY REPEATING THIS PROCESS, WE BY REPEATING THIS PROCESS, WE BY REPEATING THIS PROCESS, WE COULD CONSTRUCT A STRUCTURE OR COULD CONSTRUCT A STRUCTURE OR COULD CONSTRUCT A STRUCTURE OR CONSTRUCT A GRAPH ABOVE ALL THE CONSTRUCT A GRAPH ABOVE ALL THE CONSTRUCT A GRAPH ABOVE ALL THE DATA, VIA THE DATA FOR DATA, VIA THE DATA FOR DATA, VIA THE DATA FOR PROCESSING. PROCESSING. PROCESSING. AFTER WE HAVE THE GRAPH, THE AFTER WE HAVE THE GRAPH, THE AFTER WE HAVE THE GRAPH, THE REST OF THE TRAINING FLOW IS REST OF THE TRAINING FLOW IS REST OF THE TRAINING FLOW IS EXACTLY THE SAME AS WE MENTIONED EXACTLY THE SAME AS WE MENTIONED EXACTLY THE SAME AS WE MENTIONED BEFORE. BEFORE. BEFORE. LET'S, AGAIN, TAKE A LOOK AT THE LET'S, AGAIN, TAKE A LOOK AT THE LET'S, AGAIN, TAKE A LOOK AT THE ACTUAL CODE EXAMPLE. ACTUAL CODE EXAMPLE. ACTUAL CODE EXAMPLE. WE FIRST LOAD THE TRAINING DATA WE FIRST LOAD THE TRAINING DATA WE FIRST LOAD THE TRAINING DATA AND TEST SAMPLES FROM THE IMBD AND TEST SAMPLES FROM THE IMBD AND TEST SAMPLES FROM THE IMBD DATA SET. DATA SET. DATA SET. NEXT, WE LOAD THE PRETREND NEXT, WE LOAD THE PRETREND NEXT, WE LOAD THE PRETREND EMBEDDING MODEL FROM. EMBEDDING MODEL FROM. EMBEDDING MODEL FROM. THE EMBEDDING MODEL WE USE HERE THE EMBEDDING MODEL WE USE HERE THE EMBEDDING MODEL WE USE HERE IS A SWEEPER MODEL, BUT FEEL IS A SWEEPER MODEL, BUT FEEL IS A SWEEPER MODEL, BUT FEEL FREE TO REPLACE THEM WITH YOUR FREE TO REPLACE THEM WITH YOUR FREE TO REPLACE THEM WITH YOUR AVORITE PRETREND EMBEDDING AVORITE PRETREND EMBEDDING AVORITE PRETREND EMBEDDING MODEL SUCH AS WE CAN CALCULATE A MODEL SUCH AS WE CAN CALCULATE A MODEL SUCH AS WE CAN CALCULATE A SIMILARITY BETWEEN TWO REVIEWS SIMILARITY BETWEEN TWO REVIEWS SIMILARITY BETWEEN TWO REVIEWS IN AN EMBEDDING SPACE. IN AN EMBEDDING SPACE. IN AN EMBEDDING SPACE. REMEMBER, WHEN TWO REVIEWS ARE REMEMBER, WHEN TWO REVIEWS ARE REMEMBER, WHEN TWO REVIEWS ARE CLOSER IN THE EMBEDDING SPACE, CLOSER IN THE EMBEDDING SPACE, CLOSER IN THE EMBEDDING SPACE, WE ASSUME THEY SHARE SIMILAR WE ASSUME THEY SHARE SIMILAR WE ASSUME THEY SHARE SIMILAR SCHEMATICS. SCHEMATICS. SCHEMATICS. AFTER WE PROJECT TEXTS TO THE AFTER WE PROJECT TEXTS TO THE AFTER WE PROJECT TEXTS TO THE EMBEDDING, WE USE GRAPH EMBEDDING, WE USE GRAPH EMBEDDING, WE USE GRAPH FUNCTION, PROVIDED BY THE NEURAL FUNCTION, PROVIDED BY THE NEURAL FUNCTION, PROVIDED BY THE NEURAL STRUCTURAL LESCHING TO CONSTRUCT STRUCTURAL LESCHING TO CONSTRUCT STRUCTURAL LESCHING TO CONSTRUCT THE GRAPH. THE GRAPH. THE GRAPH. WE PROVIDE A THRESHOLD, WHICH IS WE PROVIDE A THRESHOLD, WHICH IS WE PROVIDE A THRESHOLD, WHICH IS 0.8 IN THIS SPACE. 0.8 IN THIS SPACE. 0.8 IN THIS SPACE. AFTER WE HAVE THE GRAPH, WE CALL AFTER WE HAVE THE GRAPH, WE CALL AFTER WE HAVE THE GRAPH, WE CALL PACK NEIGHBOR FUNCTION TO PACK NEIGHBOR FUNCTION TO PACK NEIGHBOR FUNCTION TO INCORPORATE THE NEIGHBOR SAMPLES INCORPORATE THE NEIGHBOR SAMPLES INCORPORATE THE NEIGHBOR SAMPLES INTO EACH TRAINING SAMPLE. INTO EACH TRAINING SAMPLE. INTO EACH TRAINING SAMPLE. HERE FOR EACH SAMPLE, THREE HERE FOR EACH SAMPLE, THREE HERE FOR EACH SAMPLE, THREE NEIGHBORS ARE CONSIDERED. NEIGHBORS ARE CONSIDERED. NEIGHBORS ARE CONSIDERED. AFTER WE AUGMENT IT WITH GRAPH AFTER WE AUGMENT IT WITH GRAPH AFTER WE AUGMENT IT WITH GRAPH SIGNALS, EVERYTHING IS JUST LIKE SIGNALS, EVERYTHING IS JUST LIKE SIGNALS, EVERYTHING IS JUST LIKE THE FIRST CODE EXAMPLE WE SHOW. THE FIRST CODE EXAMPLE WE SHOW. THE FIRST CODE EXAMPLE WE SHOW. DATA, BUILD A BASE MODEL VERSUS DATA, BUILD A BASE MODEL VERSUS DATA, BUILD A BASE MODEL VERSUS EITHER SEE CONTROVERSIAL, EITHER SEE CONTROVERSIAL, EITHER SEE CONTROVERSIAL, FUNCTIONAL API, OR SUBCLASSING. FUNCTIONAL API, OR SUBCLASSING. FUNCTIONAL API, OR SUBCLASSING. THEN WE USE THE NEURAL THEN WE USE THE NEURAL THEN WE USE THE NEURAL STRUCTURAL LEARNING API TO WRAP STRUCTURAL LEARNING API TO WRAP STRUCTURAL LEARNING API TO WRAP AROUND THE BASE MODEL. AROUND THE BASE MODEL. AROUND THE BASE MODEL. GERN THE REST OF THE WORKFLOW IS GERN THE REST OF THE WORKFLOW IS GERN THE REST OF THE WORKFLOW IS JUST TO EXTEND OUR FLOW, JUST TO EXTEND OUR FLOW, JUST TO EXTEND OUR FLOW, COMPILE, FIT, AND EVALVE. COMPILE, FIT, AND EVALVE. COMPILE, FIT, AND EVALVE. SO WE ALSO PROVIDE HANDS-ON SO WE ALSO PROVIDE HANDS-ON SO WE ALSO PROVIDE HANDS-ON TUTORIAL. TUTORIAL. TUTORIAL. STEP BY STEP TUTORIAL ON OUR STEP BY STEP TUTORIAL ON OUR STEP BY STEP TUTORIAL ON OUR WEBSITE. WEBSITE. WEBSITE. FEEL FREE TO VISIT THE WEBSITE FEEL FREE TO VISIT THE WEBSITE FEEL FREE TO VISIT THE WEBSITE AND PLAY IT BY YOURSELF. AND PLAY IT BY YOURSELF. AND PLAY IT BY YOURSELF. THE SECOND METHOD TO CONSTRUCT THE SECOND METHOD TO CONSTRUCT THE SECOND METHOD TO CONSTRUCT THE STRUCTURE OR CONSTRUCT A THE STRUCTURE OR CONSTRUCT A THE STRUCTURE OR CONSTRUCT A GRAPH SIGNAL IS TO BUILD A GRAPH GRAPH SIGNAL IS TO BUILD A GRAPH GRAPH SIGNAL IS TO BUILD A GRAPH DYNAMICALLY BY ADDING DYNAMICALLY BY ADDING DYNAMICALLY BY ADDING ADVERSARIAL NEIGHBORS. ADVERSARIAL NEIGHBORS. ADVERSARIAL NEIGHBORS. FOR EACH TRAINING SAMPLE WE FIND FOR EACH TRAINING SAMPLE WE FIND FOR EACH TRAINING SAMPLE WE FIND A MALICIOUS -- BASED ON THE A MALICIOUS -- BASED ON THE A MALICIOUS -- BASED ON THE REVERSE GRADIENT DIRECTION. REVERSE GRADIENT DIRECTION. REVERSE GRADIENT DIRECTION. IN OTHER WORDS, THAT IS DESIGND IN OTHER WORDS, THAT IS DESIGND IN OTHER WORDS, THAT IS DESIGND TO CONFUSE THE MODEL, WHICH TO CONFUSE THE MODEL, WHICH TO CONFUSE THE MODEL, WHICH MEANS TO MAXIMIZE THE LOSS. MEANS TO MAXIMIZE THE LOSS. MEANS TO MAXIMIZE THE LOSS. THEN THIS MALICIOUS -- IS -- IS THEN THIS MALICIOUS -- IS -- IS THEN THIS MALICIOUS -- IS -- IS ADDING ORIGINAL TRAINING SAMPLE ADDING ORIGINAL TRAINING SAMPLE ADDING ORIGINAL TRAINING SAMPLE TO CREATE AN ADVERSARIAL TO CREATE AN ADVERSARIAL TO CREATE AN ADVERSARIAL NEIGHBOR. NEIGHBOR. NEIGHBOR. THEN WE ADD IT BETWEEN THESE THEN WE ADD IT BETWEEN THESE THEN WE ADD IT BETWEEN THESE ADVERSARIAL NEIGHBORS AND THE ADVERSARIAL NEIGHBORS AND THE ADVERSARIAL NEIGHBORS AND THE ORIGINAL TRAINING EXAMPLE. ORIGINAL TRAINING EXAMPLE. ORIGINAL TRAINING EXAMPLE. THEREFORE, WE HAVE CONSTRUCTED A THEREFORE, WE HAVE CONSTRUCTED A THEREFORE, WE HAVE CONSTRUCTED A GRAPH OR CONSTRUCTED A GRAPH OR CONSTRUCTED A GRAPH OR CONSTRUCTED A STRUCTURE. STRUCTURE. STRUCTURE. THIS IS THE CODE EXAMPLE USING THIS IS THE CODE EXAMPLE USING THIS IS THE CODE EXAMPLE USING ADVERSARIAL MODEL FROM THE ADVERSARIAL MODEL FROM THE ADVERSARIAL MODEL FROM THE NEURAL STRUCTURAL LEARNING. NEURAL STRUCTURAL LEARNING. NEURAL STRUCTURAL LEARNING. AGAIN, FEEL FAMILIAR? AGAIN, FEEL FAMILIAR? AGAIN, FEEL FAMILIAR? IN ADDITION TO THESE THREE IN ADDITION TO THESE THREE IN ADDITION TO THESE THREE LINES, EVERYTHING ELSE FOLLOWED LINES, EVERYTHING ELSE FOLLOWED LINES, EVERYTHING ELSE FOLLOWED THE SAME WORKFLOW INTRODUCED THE SAME WORKFLOW INTRODUCED THE SAME WORKFLOW INTRODUCED BEFORE. BEFORE. BEFORE. IT HAS BEEN WIDELY USED IN MANY IT HAS BEEN WIDELY USED IN MANY IT HAS BEEN WIDELY USED IN MANY PRODUCTS AND SERVICES IN GOOGLE. PRODUCTS AND SERVICES IN GOOGLE. PRODUCTS AND SERVICES IN GOOGLE. FOR EXAMPLE, LEARNING IMAGE FOR EXAMPLE, LEARNING IMAGE FOR EXAMPLE, LEARNING IMAGE SCHEMATIC EMBEDDING. SCHEMATIC EMBEDDING. SCHEMATIC EMBEDDING. HERE WE PROVIDE SIX EXAMPLES. HERE WE PROVIDE SIX EXAMPLES. HERE WE PROVIDE SIX EXAMPLES. TWO FOR EACH SCHEMATIC TWO FOR EACH SCHEMATIC TWO FOR EACH SCHEMATIC GRANULARITY TO ILLUSTRATE A GRANULARITY TO ILLUSTRATE A GRANULARITY TO ILLUSTRATE A DIFFERENCE -- THE OBJECT IN THE DIFFERENCE -- THE OBJECT IN THE DIFFERENCE -- THE OBJECT IN THE RIGHT IS TO GOLDEN GATE BRIDGE. RIGHT IS TO GOLDEN GATE BRIDGE. RIGHT IS TO GOLDEN GATE BRIDGE. THE GOLDEN GATE BRIDGE IS THE THE GOLDEN GATE BRIDGE IS THE THE GOLDEN GATE BRIDGE IS THE STEEL RED BRIDGE. STEEL RED BRIDGE. STEEL RED BRIDGE. ALL THE STEEL RED BRIBLG, SUCH ALL THE STEEL RED BRIBLG, SUCH ALL THE STEEL RED BRIBLG, SUCH AS AN IMAGE IN THE MIDDLE OR THE AS AN IMAGE IN THE MIDDLE OR THE AS AN IMAGE IN THE MIDDLE OR THE GOLDEN GATE BRIDGE. GOLDEN GATE BRIDGE. GOLDEN GATE BRIDGE. LEARNING TO CAPTURE SCHEMATICS, LEARNING TO CAPTURE SCHEMATICS, LEARNING TO CAPTURE SCHEMATICS, HOWEVER, IS THE CORE OF MANY HOWEVER, IS THE CORE OF MANY HOWEVER, IS THE CORE OF MANY IMAGE-RELATED APPLICATION SUCH IMAGE-RELATED APPLICATION SUCH IMAGE-RELATED APPLICATION SUCH AS IMAGE SEARCH. AS IMAGE SEARCH. AS IMAGE SEARCH. EITHER CARRIED BY TRADITIONAL EITHER CARRIED BY TRADITIONAL EITHER CARRIED BY TRADITIONAL KEYWORDS OR THE EXAMPLE QUERY KEYWORDS OR THE EXAMPLE QUERY KEYWORDS OR THE EXAMPLE QUERY IMAGE. IMAGE. IMAGE. THIS IS THE OVERALL NEURAL THIS IS THE OVERALL NEURAL THIS IS THE OVERALL NEURAL ARCHITECTURE USED TO LEARN THE ARCHITECTURE USED TO LEARN THE ARCHITECTURE USED TO LEARN THE IMAGE EMBEDDING, AND AGAIN, FEEL IMAGE EMBEDDING, AND AGAIN, FEEL IMAGE EMBEDDING, AND AGAIN, FEEL FAMILIAR. FAMILIAR. FAMILIAR. THIS IS EXACTLY THE SAME THIS IS EXACTLY THE SAME THIS IS EXACTLY THE SAME WORKFLOW USED AGAIN AND AGAIN IN WORKFLOW USED AGAIN AND AGAIN IN WORKFLOW USED AGAIN AND AGAIN IN THIS TALK. THIS TALK. THIS TALK. SINCE THAT TALK, WE WILL NOT SINCE THAT TALK, WE WILL NOT SINCE THAT TALK, WE WILL NOT INTRODUCE IN DETAIL ABOUT OTHER INTRODUCE IN DETAIL ABOUT OTHER INTRODUCE IN DETAIL ABOUT OTHER TECHNIQUES SUCH AS SIMPLE USED TECHNIQUES SUCH AS SIMPLE USED TECHNIQUES SUCH AS SIMPLE USED TO TRAIN THE MODEL. TO TRAIN THE MODEL. TO TRAIN THE MODEL. IF YOU ARE INTERESTED, PLEASE IF YOU ARE INTERESTED, PLEASE IF YOU ARE INTERESTED, PLEASE REFER TO OUR PAPER. REFER TO OUR PAPER. REFER TO OUR PAPER. LET'S ZOOM IN INTO THE STRUCTURE LET'S ZOOM IN INTO THE STRUCTURE LET'S ZOOM IN INTO THE STRUCTURE PART. PART. PART. THE GRAPH USED HERE IS A CORE THE GRAPH USED HERE IS A CORE THE GRAPH USED HERE IS A CORE CURRENT GRAPH. CURRENT GRAPH. CURRENT GRAPH. ESSENTIALLY THE CORE CURRENTS ESSENTIALLY THE CORE CURRENTS ESSENTIALLY THE CORE CURRENTS GRAPH IS TRYING TO ANSWER THE GRAPH IS TRYING TO ANSWER THE GRAPH IS TRYING TO ANSWER THE FOLLOWING QUESTION. FOLLOWING QUESTION. FOLLOWING QUESTION. GIVEN ONE IMAGE IS SELECTED, GIVEN ONE IMAGE IS SELECTED, GIVEN ONE IMAGE IS SELECTED, WHAT ARE OTHER IMAGES THAT ARE WHAT ARE OTHER IMAGES THAT ARE WHAT ARE OTHER IMAGES THAT ARE SUFFICIENTLY SIMILAR THAT WOULD SUFFICIENTLY SIMILAR THAT WOULD SUFFICIENTLY SIMILAR THAT WOULD ALSO BE SELECTED AND SAY IT IS ALSO BE SELECTED AND SAY IT IS ALSO BE SELECTED AND SAY IT IS THE WHITE ENGLISH BULLDOG. THE WHITE ENGLISH BULLDOG. THE WHITE ENGLISH BULLDOG. IF TWO IMAGES ARE CALLED FOR IF TWO IMAGES ARE CALLED FOR IF TWO IMAGES ARE CALLED FOR MANY, MANY TIMES, WE ADD AN EDGE MANY, MANY TIMES, WE ADD AN EDGE MANY, MANY TIMES, WE ADD AN EDGE BETWEEN THESE TWO IMAGES MAKING BETWEEN THESE TWO IMAGES MAKING BETWEEN THESE TWO IMAGES MAKING THEM NEIGHBORS. THEM NEIGHBORS. THEM NEIGHBORS. SO HERE ARE SOME EXPERIMENTAL SO HERE ARE SOME EXPERIMENTAL SO HERE ARE SOME EXPERIMENTAL RESULTS FOR EACH QUERY IMAGE WE RESULTS FOR EACH QUERY IMAGE WE RESULTS FOR EACH QUERY IMAGE WE PROVIDE A TOP THREE NEIGHBOR PROVIDE A TOP THREE NEIGHBOR PROVIDE A TOP THREE NEIGHBOR BASED ON THE IMAGE EMBEDDING BASED ON THE IMAGE EMBEDDING BASED ON THE IMAGE EMBEDDING LEARNING. LEARNING. LEARNING. THE IMAGE COLOR TO BE STRONGLY THE IMAGE COLOR TO BE STRONGLY THE IMAGE COLOR TO BE STRONGLY SIMILAR WITH THE QUERY IMAGE, SIMILAR WITH THE QUERY IMAGE, SIMILAR WITH THE QUERY IMAGE, AND IT'S NOT SO SIMILAR. AND IT'S NOT SO SIMILAR. AND IT'S NOT SO SIMILAR. FOR EXAMPLE, IN THE LAST FIGURE FOR EXAMPLE, IN THE LAST FIGURE FOR EXAMPLE, IN THE LAST FIGURE WHEN THE IMAGE QUERY IS A WHITE WHEN THE IMAGE QUERY IS A WHITE WHEN THE IMAGE QUERY IS A WHITE SQUIRREL, ALL THE WHITE SQUIRREL SQUIRREL, ALL THE WHITE SQUIRREL SQUIRREL, ALL THE WHITE SQUIRREL USING AN EMBEDDING LEARNINGED USING AN EMBEDDING LEARNINGED USING AN EMBEDDING LEARNINGED FROM THE NEURAL STRUCTURAL FROM THE NEURAL STRUCTURAL FROM THE NEURAL STRUCTURAL LEARNING FRAMEWORK. LEARNING FRAMEWORK. LEARNING FRAMEWORK. IN OTHER WORDS, LEARNING WITH IN OTHER WORDS, LEARNING WITH IN OTHER WORDS, LEARNING WITH STRUCTURE IS ABLE TO CAPTURE STRUCTURE IS ABLE TO CAPTURE STRUCTURE IS ABLE TO CAPTURE IMAGE SCHEMATICS MUCH CLOSER TO IMAGE SCHEMATICS MUCH CLOSER TO IMAGE SCHEMATICS MUCH CLOSER TO ACTUAL HUMAN PERCEPTION. ACTUAL HUMAN PERCEPTION. ACTUAL HUMAN PERCEPTION. SO TO RECAP, TRAINING WITH SO TO RECAP, TRAINING WITH SO TO RECAP, TRAINING WITH STRUCTURE IS VERY USEFUL. STRUCTURE IS VERY USEFUL. STRUCTURE IS VERY USEFUL. LAST LABEL DATA TOO REQUIRE TO LAST LABEL DATA TOO REQUIRE TO LAST LABEL DATA TOO REQUIRE TO EFFECTIVELY TRAIN THE MODEL. EFFECTIVELY TRAIN THE MODEL. EFFECTIVELY TRAIN THE MODEL. ALSO, LEARNING WITH STRUCTURE ALSO, LEARNING WITH STRUCTURE ALSO, LEARNING WITH STRUCTURE SIGNALS LEAD TO THE MORROW BUST SIGNALS LEAD TO THE MORROW BUST SIGNALS LEAD TO THE MORROW BUST MODEL. MODEL. MODEL. ALSO, IT WORKS FOR ALL TYPES OF ALSO, IT WORKS FOR ALL TYPES OF ALSO, IT WORKS FOR ALL TYPES OF NEURAL NETS, EITHER COMMERCIAL NEURAL NETS, EITHER COMMERCIAL NEURAL NETS, EITHER COMMERCIAL NEURAL NET, OR RECURRENT NEURAL NEURAL NET, OR RECURRENT NEURAL NEURAL NET, OR RECURRENT NEURAL NET OR ANY CUSTOM NEURAL NET YOU NET OR ANY CUSTOM NEURAL NET YOU NET OR ANY CUSTOM NEURAL NET YOU DESIGN. DESIGN. DESIGN. THIS IS PROBABLY THE MOST THIS IS PROBABLY THE MOST THIS IS PROBABLY THE MOST INFORMATIVE SLIDE OF THIS TALK. INFORMATIVE SLIDE OF THIS TALK. INFORMATIVE SLIDE OF THIS TALK. YOU CAN LEARN TOOLS, LIBRARIES, YOU CAN LEARN TOOLS, LIBRARIES, YOU CAN LEARN TOOLS, LIBRARIES, OR HANDS-ON TUTORIALS IN DETAIL OR HANDS-ON TUTORIALS IN DETAIL OR HANDS-ON TUTORIALS IN DETAIL IN OUR WEBSITE. IN OUR WEBSITE. IN OUR WEBSITE. ALSO, PLEASE START. ALSO, PLEASE START. ALSO, PLEASE START. WE DO TAKE THE ISSUES, AND WE WE DO TAKE THE ISSUES, AND WE WE DO TAKE THE ISSUES, AND WE WOULD LOVE TO HEAR FROM YOU. WOULD LOVE TO HEAR FROM YOU. WOULD LOVE TO HEAR FROM YOU. WE ARE LOOKING FORWARD TO WE ARE LOOKING FORWARD TO WE ARE LOOKING FORWARD TO DEVELOPING THIS FRAMEWORK WITH DEVELOPING THIS FRAMEWORK WITH DEVELOPING THIS FRAMEWORK WITH ALL OF YOU TO MAKE THIS ALL OF YOU TO MAKE THIS ALL OF YOU TO MAKE THIS FRAMEWORK MORE COMPREHENSIVE. FRAMEWORK MORE COMPREHENSIVE. FRAMEWORK MORE COMPREHENSIVE. WE WILL BE WAITING FOR YOUR WE WILL BE WAITING FOR YOUR WE WILL BE WAITING FOR YOUR REQUEST. REQUEST. REQUEST. THANK YOU. THANK YOU. THANK YOU. (APP (APP (APP (APPLAUSE). (APPLAUSE). (APPLAUSE). >> DO WE STILL HAVE TIME >> DO WE STILL HAVE TIME >> DO WE STILL HAVE TIME FOR QUESTIONS? FOR QUESTIONS? FOR QUESTIONS? OKAY. OKAY. OKAY. I THINK WE STILL HAVE SOME TIME I THINK WE STILL HAVE SOME TIME I THINK WE STILL HAVE SOME TIME FOR QUESTIONS. FOR QUESTIONS. FOR QUESTIONS. >> YOU MAY HAVE TO USE THE MIK >> YOU MAY HAVE TO USE THE MIK >> YOU MAY HAVE TO USE THE MIK >> THE GRAPH FUNCTION THAT YOU >> THE GRAPH FUNCTION THAT YOU >> THE GRAPH FUNCTION THAT YOU MENTIONED, DOES IT DO, LIKE, MENTIONED, DOES IT DO, LIKE, MENTIONED, DOES IT DO, LIKE, COMPARISON ACROSS ALL THE, LIKE, COMPARISON ACROSS ALL THE, LIKE, COMPARISON ACROSS ALL THE, LIKE, ALL THE ITEMS ALL THE ITEMS ALL THE ITEMS >> ALL THE TRAINING -- ALL THE >> ALL THE TRAINING -- ALL THE >> ALL THE TRAINING -- ALL THE SAMPLES. SAMPLES. SAMPLES. NOT ONLY TRAINING, YEAH NOT ONLY TRAINING, YEAH NOT ONLY TRAINING, YEAH >> I SEE. >> I SEE. >> I SEE. SO FOR -- SO FOR -- SO FOR -- >> LABEL AND ALL LABEL >> LABEL AND ALL LABEL >> LABEL AND ALL LABEL CENTERS. CENTERS. CENTERS. >> I SEE. >> I SEE. >> I SEE. SO FOR -- DOES IT, LIKE -- HOW SO FOR -- DOES IT, LIKE -- HOW SO FOR -- DOES IT, LIKE -- HOW LONG DID IT TAKE TO BUILD GRAPH? LONG DID IT TAKE TO BUILD GRAPH? LONG DID IT TAKE TO BUILD GRAPH? >> IT'S A RELATIVELY SMALL DATA >> IT'S A RELATIVELY SMALL DATA >> IT'S A RELATIVELY SMALL DATA SET. SET. SET. IT WILL NOT TAKE TOO LONG? IT WILL NOT TAKE TOO LONG? IT WILL NOT TAKE TOO LONG? >> INSTEAD OF THAT, YOU THEN >> INSTEAD OF THAT, YOU THEN >> INSTEAD OF THAT, YOU THEN USE -- USE -- USE -- >> GREAT. >> GREAT. >> GREAT. >> WE'RE GOING TO RELEASE IT >> WE'RE GOING TO RELEASE IT >> WE'RE GOING TO RELEASE IT >> THANK YOU >> THANK YOU >> THANK YOU >> VERY NICE. >> VERY NICE. >> VERY NICE. JUST I UNDERSTAND WHAT YOU ARE JUST I UNDERSTAND WHAT YOU ARE JUST I UNDERSTAND WHAT YOU ARE DOING. DOING. DOING. IT LOOKS LIKE YOU'RE TAKING IT LOOKS LIKE YOU'RE TAKING IT LOOKS LIKE YOU'RE TAKING IMAGES AND THEN GATHERING IMAGES IMAGES AND THEN GATHERING IMAGES IMAGES AND THEN GATHERING IMAGES FROM THE GENERATOR ASSOCIATING FROM THE GENERATOR ASSOCIATING FROM THE GENERATOR ASSOCIATING THEM RECEIVING THE NEURAL THEM RECEIVING THE NEURAL THEM RECEIVING THE NEURAL NETWORK. NETWORK. NETWORK. IS THAT CORRECT? IS THAT CORRECT? IS THAT CORRECT? >> IT DOES NOT NECESSARILY >> IT DOES NOT NECESSARILY >> IT DOES NOT NECESSARILY HAVE TO BE THE STRUCTURE. HAVE TO BE THE STRUCTURE. HAVE TO BE THE STRUCTURE. THE IDEA IS ANY NETWORK THAT YOU THE IDEA IS ANY NETWORK THAT YOU THE IDEA IS ANY NETWORK THAT YOU ARE TRYING TO LEARN, WITH USING ARE TRYING TO LEARN, WITH USING ARE TRYING TO LEARN, WITH USING THE GRADIENTS AS A BACK DROP AND THE GRADIENTS AS A BACK DROP AND THE GRADIENTS AS A BACK DROP AND REVERSING THE GRADIENTS TO REVERSING THE GRADIENTS TO REVERSING THE GRADIENTS TO SKRUBLGT SORT OF A NOISY SKRUBLGT SORT OF A NOISY SKRUBLGT SORT OF A NOISY EXAMPLE, IF YOU WILL, AND THE EXAMPLE, IF YOU WILL, AND THE EXAMPLE, IF YOU WILL, AND THE IDEA -- THE REASON TO DO THIS IS IDEA -- THE REASON TO DO THIS IS IDEA -- THE REASON TO DO THIS IS IN A WHILE DOING THE TRAINING IS IN A WHILE DOING THE TRAINING IS IN A WHILE DOING THE TRAINING IS THAT NEXT TIME THE NETWORK SEES THAT NEXT TIME THE NETWORK SEES THAT NEXT TIME THE NETWORK SEES THIS NOISY EXAMPLE, RIGHT, IT THIS NOISY EXAMPLE, RIGHT, IT THIS NOISY EXAMPLE, RIGHT, IT WILL STILL LEARN TO CORRECT IT WILL STILL LEARN TO CORRECT IT WILL STILL LEARN TO CORRECT IT RATHER THAN, YOU KNOW, FLIPPING RATHER THAN, YOU KNOW, FLIPPING RATHER THAN, YOU KNOW, FLIPPING THE PREDICTIONS. THE PREDICTIONS. THE PREDICTIONS. WE TRY TO MAKE THAT INTERMEDIATE WE TRY TO MAKE THAT INTERMEDIATE WE TRY TO MAKE THAT INTERMEDIATE LAYER AND ALSO THE PREDICTIONS. LAYER AND ALSO THE PREDICTIONS. LAYER AND ALSO THE PREDICTIONS. >> IN THAT LAST FUNCTION, IT >> IN THAT LAST FUNCTION, IT >> IN THAT LAST FUNCTION, IT LOOKED LIKE WE HAD SOMETHING LOOKED LIKE WE HAD SOMETHING LOOKED LIKE WE HAD SOMETHING FROM A -- PLUS, THE GENERATOR FROM A -- PLUS, THE GENERATOR FROM A -- PLUS, THE GENERATOR >> THE EQUATION -- >> THE EQUATION -- >> THE EQUATION -- >> THAT LOSS THAT YOU HAD AND >> THAT LOSS THAT YOU HAD AND >> THAT LOSS THAT YOU HAD AND TRYING TO MINIMIZE TRYING TO MINIMIZE TRYING TO MINIMIZE >> OH, THE LOSS FUNCTION JUST >> OH, THE LOSS FUNCTION JUST >> OH, THE LOSS FUNCTION JUST HAS TWO COMPONENTS. HAS TWO COMPONENTS. HAS TWO COMPONENTS. THE FIRST IS THE SUPERVISED LOSS THE FIRST IS THE SUPERVISED LOSS THE FIRST IS THE SUPERVISED LOSS OR WHATEVER LOSS YOU HAD OR WHATEVER LOSS YOU HAD OR WHATEVER LOSS YOU HAD APPLICATION HAS. APPLICATION HAS. APPLICATION HAS. THE SECOND ONE IS FACTORIZING THE SECOND ONE IS FACTORIZING THE SECOND ONE IS FACTORIZING THE LOS OR NEIGHBORS, LIKE THE LOS OR NEIGHBORS, LIKE THE LOS OR NEIGHBORS, LIKE SOURCE IMAGES AND NEIGHBORS. SOURCE IMAGES AND NEIGHBORS. SOURCE IMAGES AND NEIGHBORS. AND IN THE CASE OF ADVERSARIAL AND IN THE CASE OF ADVERSARIAL AND IN THE CASE OF ADVERSARIAL NEIGHBOR IMAGE, IT'S BASICALLY NEIGHBOR IMAGE, IT'S BASICALLY NEIGHBOR IMAGE, IT'S BASICALLY CONSTRUCTED, AND THE WAY TO CONSTRUCTED, AND THE WAY TO CONSTRUCTED, AND THE WAY TO BASICALLY WHATEVER ASSIGN IT BASICALLY WHATEVER ASSIGN IT BASICALLY WHATEVER ASSIGN IT >> VERY GOOD. >> VERY GOOD. >> VERY GOOD. THANK YOU THANK YOU THANK YOU >> I THINK IN PAGE 54 GRAPH -- >> I THINK IN PAGE 54 GRAPH -- >> I THINK IN PAGE 54 GRAPH -- CAN YOU OPEN CAN YOU OPEN CAN YOU OPEN >> 34 >> 34 >> 34 >> YEAH, THIS ONE. >> YEAH, THIS ONE. >> YEAH, THIS ONE. YEAH. YEAH. YEAH. THIS GRAPH LOOKS LIKE IT'S THIS GRAPH LOOKS LIKE IT'S THIS GRAPH LOOKS LIKE IT'S CHANGED TO ME. CHANGED TO ME. CHANGED TO ME. HOW MANY SAMPLES AT THIS POINT HOW MANY SAMPLES AT THIS POINT HOW MANY SAMPLES AT THIS POINT HAVE YOU GENERATED HERE? HAVE YOU GENERATED HERE? HAVE YOU GENERATED HERE? >> BASICALLY, THIS IS -- >> BASICALLY, THIS IS -- >> BASICALLY, THIS IS -- IT'S -- IT'S NOT FROM THE -- IT'S -- IT'S NOT FROM THE -- IT'S -- IT'S NOT FROM THE -- >> HOW MANY TRAINING >> HOW MANY TRAINING >> HOW MANY TRAINING TRIALS -- HOW MANY TIMES DID YOU TRIALS -- HOW MANY TIMES DID YOU TRIALS -- HOW MANY TIMES DID YOU TRAIN? TRAIN? TRAIN? >> HOW MANY TRAINING >> HOW MANY TRAINING >> HOW MANY TRAINING TRIALS? TRIALS? TRIALS? I DO BELIEVE WE HAVE FIVE I DO BELIEVE WE HAVE FIVE I DO BELIEVE WE HAVE FIVE TRIALS. TRIALS. TRIALS. >> EACH POINT IS FIVE SAMPLES. >> EACH POINT IS FIVE SAMPLES. >> EACH POINT IS FIVE SAMPLES. EVEN IF THE NETWORK IS POWERFUL, EVEN IF THE NETWORK IS POWERFUL, EVEN IF THE NETWORK IS POWERFUL, LIKE FOR THE GAP TO INCREASE. LIKE FOR THE GAP TO INCREASE. LIKE FOR THE GAP TO INCREASE. THERE'S SOME 2% THING YOU SEE. THERE'S SOME 2% THING YOU SEE. THERE'S SOME 2% THING YOU SEE. THE GAP IS LOWER THAN WHAT THE THE GAP IS LOWER THAN WHAT THE THE GAP IS LOWER THAN WHAT THE 5% IS, BUT THAT'S JUST BASED ON 5% IS, BUT THAT'S JUST BASED ON 5% IS, BUT THAT'S JUST BASED ON THIS DATA CENTER. THIS DATA CENTER. THIS DATA CENTER. TYPICALLY WHAT YOU SEE IS THE TYPICALLY WHAT YOU SEE IS THE TYPICALLY WHAT YOU SEE IS THE GAP INCREASE AND SEPARATINGS GAP INCREASE AND SEPARATINGS GAP INCREASE AND SEPARATINGS RATIO GOES LOWER RATIO GOES LOWER RATIO GOES LOWER >> IF YOU HAVE A 2% -- >> IF YOU HAVE A 2% -- >> IF YOU HAVE A 2% -- >> YES. >> YES. >> YES. THIS IS JUST ONE EXAMPLE. THIS IS JUST ONE EXAMPLE. THIS IS JUST ONE EXAMPLE. >> WE WOULD RECOMMEND THAT YOU >> WE WOULD RECOMMEND THAT YOU >> WE WOULD RECOMMEND THAT YOU TRY IT ON YOUR OWN DATA SET AND TRY IT ON YOUR OWN DATA SET AND TRY IT ON YOUR OWN DATA SET AND ON ONE OF THE NETWORKS THAT YOU ON ONE OF THE NETWORKS THAT YOU ON ONE OF THE NETWORKS THAT YOU HAVE HAVE HAVE >> HI. >> HI. >> HI. HOW ARE YOU DOING? HOW ARE YOU DOING? HOW ARE YOU DOING? >> I WAS CURIOUS HOW YOU >> I WAS CURIOUS HOW YOU >> I WAS CURIOUS HOW YOU HAVE SEGMENTATION OR TO VIDEO HAVE SEGMENTATION OR TO VIDEO HAVE SEGMENTATION OR TO VIDEO CLASSIFICATION. CLASSIFICATION. CLASSIFICATION. >> I THINK THERE ARE A COUPLE OF >> I THINK THERE ARE A COUPLE OF >> I THINK THERE ARE A COUPLE OF DIFFERENT WAYS. DIFFERENT WAYS. DIFFERENT WAYS. LIKE, IN THE VIDEO LIKE, IN THE VIDEO LIKE, IN THE VIDEO CLASSIFICATION, YOU CAN LOOK AT CLASSIFICATION, YOU CAN LOOK AT CLASSIFICATION, YOU CAN LOOK AT VIDEOS THAT ARE SORT OF RELATED. VIDEOS THAT ARE SORT OF RELATED. VIDEOS THAT ARE SORT OF RELATED. RIGHT? RIGHT? RIGHT? LIKE I MEAN, SIMILAR, LET'S SAY, LIKE I MEAN, SIMILAR, LET'S SAY, LIKE I MEAN, SIMILAR, LET'S SAY, FROM THE SAME CHANNEL, AND YOU FROM THE SAME CHANNEL, AND YOU FROM THE SAME CHANNEL, AND YOU HAVE SIMILAR KIND OF CONTENT. HAVE SIMILAR KIND OF CONTENT. HAVE SIMILAR KIND OF CONTENT. OR THE METADATA KIND OF MATCHES. OR THE METADATA KIND OF MATCHES. OR THE METADATA KIND OF MATCHES. YOU CAN CREATE THESE LINKS YOU CAN CREATE THESE LINKS YOU CAN CREATE THESE LINKS BETWEEN DIFFERENT VIDEOS. BETWEEN DIFFERENT VIDEOS. BETWEEN DIFFERENT VIDEOS. THESE RELATED VIDEOS SHOULD BE THESE RELATED VIDEOS SHOULD BE THESE RELATED VIDEOS SHOULD BE OPTIMIZED TO LEARN THE SAME OPTIMIZED TO LEARN THE SAME OPTIMIZED TO LEARN THE SAME PREDICTION? PREDICTION? PREDICTION? >> THERE ARE MANY OTHER >> THERE ARE MANY OTHER >> THERE ARE MANY OTHER WAYS WE CAN TALK OFF LINE. WAYS WE CAN TALK OFF LINE. WAYS WE CAN TALK OFF LINE. >> THANK YOU >> THANK YOU >> THANK YOU >> IF WE'RE TALKING ABOUT -- >> IF WE'RE TALKING ABOUT -- >> IF WE'RE TALKING ABOUT -- >> LAST QUESTION. >> LAST QUESTION. >> LAST QUESTION. >> LAST QUESTION >> LAST QUESTION >> LAST QUESTION >> CLASSICAL PRINCIPLES, LIKE >> CLASSICAL PRINCIPLES, LIKE >> CLASSICAL PRINCIPLES, LIKE DISTORTION AND BLURRING, DISTORTION AND BLURRING, DISTORTION AND BLURRING, STRETCHING, AND ALL THIS KIND OF STRETCHING, AND ALL THIS KIND OF STRETCHING, AND ALL THIS KIND OF STUFF COMPARED TO THE STUFF COMPARED TO THE STUFF COMPARED TO THE ADVERSARIAL GENERATED SAMPLES. ADVERSARIAL GENERATED SAMPLES. ADVERSARIAL GENERATED SAMPLES. IS THERE ANY BENEFIT OF USING IS THERE ANY BENEFIT OF USING IS THERE ANY BENEFIT OF USING ONE AUTO OVER ANOTHER IN THIS ONE AUTO OVER ANOTHER IN THIS ONE AUTO OVER ANOTHER IN THIS CASE? CASE? CASE? >> SO -- >> SO -- >> SO -- >> WHY ADVERSARIAL EXAMPLES. >> WHY ADVERSARIAL EXAMPLES. >> WHY ADVERSARIAL EXAMPLES. THERE ARE SOME CLASSICAL THERE ARE SOME CLASSICAL THERE ARE SOME CLASSICAL PRINCIPLES PRINCIPLES PRINCIPLES >> BECAUSE THE DISTORTION AND >> BECAUSE THE DISTORTION AND >> BECAUSE THE DISTORTION AND ROTATION ARE PREDEFINED, LIKE, ROTATION ARE PREDEFINED, LIKE, ROTATION ARE PREDEFINED, LIKE, TRANSMISSION FUNCTIONS THAT YOU TRANSMISSION FUNCTIONS THAT YOU TRANSMISSION FUNCTIONS THAT YOU HAVE, RIGHT? HAVE, RIGHT? HAVE, RIGHT? THE ADVERSARIAL -- IT DEPENDS ON THE ADVERSARIAL -- IT DEPENDS ON THE ADVERSARIAL -- IT DEPENDS ON THE DATA AND THE NETWORK, AND THE DATA AND THE NETWORK, AND THE DATA AND THE NETWORK, AND YOU ARE USING THE GREATENTS FOR YOU ARE USING THE GREATENTS FOR YOU ARE USING THE GREATENTS FOR THE BACK DROPS TO CREATE AN THE BACK DROPS TO CREATE AN THE BACK DROPS TO CREATE AN EQUAL EXAMPLE. EQUAL EXAMPLE. EQUAL EXAMPLE. THE HARD EXAMPLE, IF YOU WILL. THE HARD EXAMPLE, IF YOU WILL. THE HARD EXAMPLE, IF YOU WILL. THIS IS THE ROTATION OR BRURING, THIS IS THE ROTATION OR BRURING, THIS IS THE ROTATION OR BRURING, RIGHT? RIGHT? RIGHT? WHAT IF SOMEBODY SAYS IT'S NOT A WHAT IF SOMEBODY SAYS IT'S NOT A WHAT IF SOMEBODY SAYS IT'S NOT A ROTATED IMAGE, BUT SAYS I TAKE, ROTATED IMAGE, BUT SAYS I TAKE, ROTATED IMAGE, BUT SAYS I TAKE, LIKE, PIXELS HERE OR THERE AND LIKE, PIXELS HERE OR THERE AND LIKE, PIXELS HERE OR THERE AND TRANSFORM THEM OR BLUR THEM TRANSFORM THEM OR BLUR THEM TRANSFORM THEM OR BLUR THEM >> WHEN NETWORK HAS THE FEEDBACK >> WHEN NETWORK HAS THE FEEDBACK >> WHEN NETWORK HAS THE FEEDBACK FROM THE GRAPH THAT ACTUALLY FROM THE GRAPH THAT ACTUALLY FROM THE GRAPH THAT ACTUALLY ADVERSARIAL NEIGHBOR GIVES YOU A ADVERSARIAL NEIGHBOR GIVES YOU A ADVERSARIAL NEIGHBOR GIVES YOU A COMPLETELY, LIKE, LOW SCORE ON COMPLETELY, LIKE, LOW SCORE ON COMPLETELY, LIKE, LOW SCORE ON THE EXAMPLE, BUT THE NETWORK THE EXAMPLE, BUT THE NETWORK THE EXAMPLE, BUT THE NETWORK ITSELF IS PRETTY SURE THAT THE ITSELF IS PRETTY SURE THAT THE ITSELF IS PRETTY SURE THAT THE CLASSES IS THERE. CLASSES IS THERE. CLASSES IS THERE. HOW DOES IT RESULT ON THIS LEVEL HOW DOES IT RESULT ON THIS LEVEL HOW DOES IT RESULT ON THIS LEVEL WHEN YOU HAVE A PRETTY CONVINCED WHEN YOU HAVE A PRETTY CONVINCED WHEN YOU HAVE A PRETTY CONVINCED SCENARIO VERSUS ADVERSARIAL. SCENARIO VERSUS ADVERSARIAL. SCENARIO VERSUS ADVERSARIAL. >> AS AN EXAMPLE, THERE'S >> AS AN EXAMPLE, THERE'S >> AS AN EXAMPLE, THERE'S NOT NECESSARILY HAVE THAT WRONG NOT NECESSARILY HAVE THAT WRONG NOT NECESSARILY HAVE THAT WRONG LABEL DURING THE LEARNING LABEL DURING THE LEARNING LABEL DURING THE LEARNING PROCESS. PROCESS. PROCESS. -- -- -- >> YOU TRY TO REGULARIZE >> YOU TRY TO REGULARIZE >> YOU TRY TO REGULARIZE IT, AND BY THIS MOMENT THEN, THE IT, AND BY THIS MOMENT THEN, THE IT, AND BY THIS MOMENT THEN, THE NETWORK IS ALREADY CONVINCED NETWORK IS ALREADY CONVINCED NETWORK IS ALREADY CONVINCED THAT IT'S NOT GIVEN, FOR THAT IT'S NOT GIVEN, FOR THAT IT'S NOT GIVEN, FOR EXAMPLE. EXAMPLE. EXAMPLE. THE GRAPH TELLS THAT IT'S STILL, THE GRAPH TELLS THAT IT'S STILL, THE GRAPH TELLS THAT IT'S STILL, LIKE, THE CLASS IS WRONG. LIKE, THE CLASS IS WRONG. LIKE, THE CLASS IS WRONG. HOW DOES THIS RESULT. HOW DOES THIS RESULT. HOW DOES THIS RESULT. >> A DMREET ADD ON THIS THAT IN >> A DMREET ADD ON THIS THAT IN >> A DMREET ADD ON THIS THAT IN SOME EXPERIMENTS, WE APPLY THE SOME EXPERIMENTS, WE APPLY THE SOME EXPERIMENTS, WE APPLY THE CORRECT LABEL TO THE ADVERSARIAL CORRECT LABEL TO THE ADVERSARIAL CORRECT LABEL TO THE ADVERSARIAL EXAMPLE. EXAMPLE. EXAMPLE. IN A PREVIOUS LIFE, ACTUALLY, IN A PREVIOUS LIFE, ACTUALLY, IN A PREVIOUS LIFE, ACTUALLY, THE ADVERSARY EXAMPLE, WE WOULD THE ADVERSARY EXAMPLE, WE WOULD THE ADVERSARY EXAMPLE, WE WOULD HAVE THE LABEL. HAVE THE LABEL. HAVE THE LABEL. YOU COULD THINK IN THIS WAY. YOU COULD THINK IN THIS WAY. YOU COULD THINK IN THIS WAY. THIS SAMPLE IS VERY CONFUSING TO THIS SAMPLE IS VERY CONFUSING TO THIS SAMPLE IS VERY CONFUSING TO THE MODEL. THE MODEL. THE MODEL. WE STILL SUPERVISE MODELS TO WE STILL SUPERVISE MODELS TO WE STILL SUPERVISE MODELS TO LEARN THE LABELING, AND LATER ON LEARN THE LABELING, AND LATER ON LEARN THE LABELING, AND LATER ON IT WILL BE MORROW BUST IT WILL BE MORROW BUST IT WILL BE MORROW BUST >> OKAY. >> OKAY. >> OKAY. CLEAR. CLEAR. CLEAR. THANK YOU VERY MUCH.
B1 model quantization training graph network neural TensorFlow World 2019 | Day 2 Livestream PM 2 0 林宜悉 posted on 2020/04/04 More Share Save Report Video vocabulary