TensorFlow World 2019 | Day 2 Livestream PM - VoiceTube: Learn English through videos!

Subtitles section Play video

FOR THINGS LIKE CALCULATING THE
FOR THINGS LIKE CALCULATING THE
FOR THINGS LIKE CALCULATING THE MEDIAN FOR STANDARD DEVIATION
MEDIAN FOR STANDARD DEVIATION
MEDIAN FOR STANDARD DEVIATION OF A FEATURE, NUMBERS THAT ARE
OF A FEATURE, NUMBERS THAT ARE
OF A FEATURE, NUMBERS THAT ARE THE SAME FOR ALL EXAMPLES,
THE SAME FOR ALL EXAMPLES,
THE SAME FOR ALL EXAMPLES, TRANSFORM WILL OUTPUT A
TRANSFORM WILL OUTPUT A
TRANSFORM WILL OUTPUT A CONSTANT.
CONSTANT.
CONSTANT. FOR THINGS LIKE NORMALIZING A
FOR THINGS LIKE NORMALIZING A
FOR THINGS LIKE NORMALIZING A VALUE, VALUES WHICH WILL BE
VALUE, VALUES WHICH WILL BE
VALUE, VALUES WHICH WILL BE DIFFERENT FOR DIFFERENT
DIFFERENT FOR DIFFERENT
DIFFERENT FOR DIFFERENT EXAMPLES, TRANSFORM WILL OUTPUT
EXAMPLES, TRANSFORM WILL OUTPUT
EXAMPLES, TRANSFORM WILL OUTPUT TENSORFLOW OPS, IT WILL THEN
TENSORFLOW OPS, IT WILL THEN
TENSORFLOW OPS, IT WILL THEN PUT AN OUTFLOW GRAPH WITH THE
PUT AN OUTFLOW GRAPH WITH THE
PUT AN OUTFLOW GRAPH WITH THE CONSTANTS AND OPS.
CONSTANTS AND OPS.
CONSTANTS AND OPS. THAT IS HER MEDIC.
THAT IS HER MEDIC.
THAT IS HER MEDIC. IT CONTAINS ALL THE INFORMATION
IT CONTAINS ALL THE INFORMATION
IT CONTAINS ALL THE INFORMATION YOU NEED TO APPLY THOSE
YOU NEED TO APPLY THOSE
YOU NEED TO APPLY THOSE TRANSFORMATIONS AND FORM THE
TRANSFORMATIONS AND FORM THE
TRANSFORMATIONS AND FORM THE INPUT STAGE FOR YOUR MODEL.
INPUT STAGE FOR YOUR MODEL.
INPUT STAGE FOR YOUR MODEL. THAT MEANS THAT THE SAME
THAT MEANS THAT THE SAME
THAT MEANS THAT THE SAME TRANSFORMATIONS ARE APPLIED
TRANSFORMATIONS ARE APPLIED
TRANSFORMATIONS ARE APPLIED CONSISTENTLY BETWEEN TRAINING
CONSISTENTLY BETWEEN TRAINING
CONSISTENTLY BETWEEN TRAINING AND SERVING, WHICH ELIMINATES
AND SERVING, WHICH ELIMINATES
AND SERVING, WHICH ELIMINATES TRAINING/SERVING SKEW.
TRAINING/SERVING SKEW.
TRAINING/SERVING SKEW. IF INSTEAD YOU ARE MOVING YOUR
IF INSTEAD YOU ARE MOVING YOUR
IF INSTEAD YOU ARE MOVING YOUR MODEL FROM A TRAINING
MODEL FROM A TRAINING
MODEL FROM A TRAINING ENVIRONMENT INTO A SERVING
ENVIRONMENT INTO A SERVING
ENVIRONMENT INTO A SERVING ENVIRONMENT OR APPLICATION AND
ENVIRONMENT OR APPLICATION AND
ENVIRONMENT OR APPLICATION AND TRYING TO APPLY THE SAME
TRYING TO APPLY THE SAME
TRYING TO APPLY THE SAME FEATURE ENGINEERING IN BOTH
FEATURE ENGINEERING IN BOTH
FEATURE ENGINEERING IN BOTH PLACES, YOU HOPE THAT THE
PLACES, YOU HOPE THAT THE
PLACES, YOU HOPE THAT THE TRANSFORMATIONS ARE THE SAME
TRANSFORMATIONS ARE THE SAME
TRANSFORMATIONS ARE THE SAME BUT SOMETIMES YOU FIND THAT
BUT SOMETIMES YOU FIND THAT
BUT SOMETIMES YOU FIND THAT THEY'RE NOT.
THEY'RE NOT.
THEY'RE NOT. WE CALL THAT TRAINING SERVING
WE CALL THAT TRAINING SERVING
WE CALL THAT TRAINING SERVING SKEW AND TRANSFORM ELIMINATES
SKEW AND TRANSFORM ELIMINATES
SKEW AND TRANSFORM ELIMINATES IT BY USING EXACTLY THE SAME
IT BY USING EXACTLY THE SAME
IT BY USING EXACTLY THE SAME CODE ANYWHERE YOU RUN YOUR
CODE ANYWHERE YOU RUN YOUR
CODE ANYWHERE YOU RUN YOUR MODEL.
MODEL.
MODEL. NOW WE'RE FINALLY READY TO
NOW WE'RE FINALLY READY TO
NOW WE'RE FINALLY READY TO TRAIN OUR MODEL.
TRAIN OUR MODEL.
TRAIN OUR MODEL. THE PART OF THE PROCESS THAT
THE PART OF THE PROCESS THAT
THE PART OF THE PROCESS THAT YOU OFTEN THINK ABOUT WHEN YOU
YOU OFTEN THINK ABOUT WHEN YOU
YOU OFTEN THINK ABOUT WHEN YOU THINK ABOUT MACHINE LEARNING.
THINK ABOUT MACHINE LEARNING.
THINK ABOUT MACHINE LEARNING. TRAINER TAKES IN THE TRANSFORM
TRAINER TAKES IN THE TRANSFORM
TRAINER TAKES IN THE TRANSFORM GRAPH AND DATA FROM TRANSFORM
GRAPH AND DATA FROM TRANSFORM
GRAPH AND DATA FROM TRANSFORM AND ESCHEMA FROM SCHEMA GEN AND
AND ESCHEMA FROM SCHEMA GEN AND
AND ESCHEMA FROM SCHEMA GEN AND TRAINS THE MODEL USING YOUR
TRAINS THE MODEL USING YOUR
TRAINS THE MODEL USING YOUR MODELLING CODE.
MODELLING CODE.
MODELLING CODE. NORMAL MODEL TRAINING.
NORMAL MODEL TRAINING.
NORMAL MODEL TRAINING. BUT WHEN TRAINING IS COMPLETE,
BUT WHEN TRAINING IS COMPLETE,
BUT WHEN TRAINING IS COMPLETE, TRAINER WILL SAVE TWO DIFFERENT
TRAINER WILL SAVE TWO DIFFERENT
TRAINER WILL SAVE TWO DIFFERENT SAVED MODELS.
SAVED MODELS.
SAVED MODELS. ONE IS A NORMAL SAVE MODEL THAT
ONE IS A NORMAL SAVE MODEL THAT
ONE IS A NORMAL SAVE MODEL THAT WILL BE DEPLOYED TO PRODUCTION.
WILL BE DEPLOYED TO PRODUCTION.
WILL BE DEPLOYED TO PRODUCTION. AND THE OTHER IS AN EVAL SAVE
AND THE OTHER IS AN EVAL SAVE
AND THE OTHER IS AN EVAL SAVE MODEL THAT WILL BE USED FOR
MODEL THAT WILL BE USED FOR
MODEL THAT WILL BE USED FOR ANALYZING THE PERFORMANCE OF
ANALYZING THE PERFORMANCE OF
ANALYZING THE PERFORMANCE OF YOUR MODEL.
YOUR MODEL.
YOUR MODEL. THE CONFIGURATION FOR TRAINER
THE CONFIGURATION FOR TRAINER
THE CONFIGURATION FOR TRAINER IS WHAT YOU WOULD EXPECT.
IS WHAT YOU WOULD EXPECT.
IS WHAT YOU WOULD EXPECT. THINGS LIKE THE NUMBER OF STEPS
THINGS LIKE THE NUMBER OF STEPS
THINGS LIKE THE NUMBER OF STEPS AND WHETHER OR NOT TO USE WARM
AND WHETHER OR NOT TO USE WARM
AND WHETHER OR NOT TO USE WARM STARTING.
STARTING.
STARTING. THE CODE THAT YOU CREATE FOR
THE CODE THAT YOU CREATE FOR
THE CODE THAT YOU CREATE FOR TRAINER IS YOUR MODELING CODE.
TRAINER IS YOUR MODELING CODE.
TRAINER IS YOUR MODELING CODE. SO IT CAN BE AS SIMPLE OR
SO IT CAN BE AS SIMPLE OR
SO IT CAN BE AS SIMPLE OR COMPLEX AS YOU NEED IT TO BE.
COMPLEX AS YOU NEED IT TO BE.
COMPLEX AS YOU NEED IT TO BE. TO MONITOR AND ANALYZE THE
TO MONITOR AND ANALYZE THE
TO MONITOR AND ANALYZE THE TRAINING PROCESS YOU CAN USE
TRAINING PROCESS YOU CAN USE
TRAINING PROCESS YOU CAN USE TENSOR BOARD JUST LIKE YOU
TENSOR BOARD JUST LIKE YOU
TENSOR BOARD JUST LIKE YOU WOULD NORMALLY.
WOULD NORMALLY.
WOULD NORMALLY. IN THIS CASE, YOU CAN LOOK AT
IN THIS CASE, YOU CAN LOOK AT
IN THIS CASE, YOU CAN LOOK AT THE CURRENT MODEL TRAINING RUN
THE CURRENT MODEL TRAINING RUN
THE CURRENT MODEL TRAINING RUN OR COMPARE THE RESULTS FROM
OR COMPARE THE RESULTS FROM
OR COMPARE THE RESULTS FROM MULTIPLE MODEL TRAINING RUNS.
MULTIPLE MODEL TRAINING RUNS.
MULTIPLE MODEL TRAINING RUNS. THIS IS ONLY POSSIBLE BECAUSE
THIS IS ONLY POSSIBLE BECAUSE
THIS IS ONLY POSSIBLE BECAUSE OF THE ML METADATA STORE THAT
OF THE ML METADATA STORE THAT
OF THE ML METADATA STORE THAT WE TALKED ABOUT IN OUR LAST
WE TALKED ABOUT IN OUR LAST
WE TALKED ABOUT IN OUR LAST EPISODE.
EPISODE.
EPISODE. TFX MAKES IT FAIRLY EASY TO DO
TFX MAKES IT FAIRLY EASY TO DO
TFX MAKES IT FAIRLY EASY TO DO THIS KIND OF COMPARISON WHICH
THIS KIND OF COMPARISON WHICH
THIS KIND OF COMPARISON WHICH IS OFTEN REVEALING.
IS OFTEN REVEALING.
IS OFTEN REVEALING. NOW THAT WE'VE TRAINED OUR
NOW THAT WE'VE TRAINED OUR
NOW THAT WE'VE TRAINED OUR MODEL, HOW DO THE RESULTS LOOK?
MODEL, HOW DO THE RESULTS LOOK?
MODEL, HOW DO THE RESULTS LOOK? THE EVALUATOR COMPONENT WILL
THE EVALUATOR COMPONENT WILL
THE EVALUATOR COMPONENT WILL TAKE THE MODEL TRAINER CREATED
TAKE THE MODEL TRAINER CREATED
TAKE THE MODEL TRAINER CREATED AND USE DEEP ANALYSIS USING
AND USE DEEP ANALYSIS USING
AND USE DEEP ANALYSIS USING BEAM AND THE MODEL ANALYSIS
BEAM AND THE MODEL ANALYSIS
BEAM AND THE MODEL ANALYSIS LIBRARY.
LIBRARY.
LIBRARY. NOT JUST LOOKING AT THE TOP
NOT JUST LOOKING AT THE TOP
NOT JUST LOOKING AT THE TOP LEVEL RESULTS ACROSS THE WHOLE
LEVEL RESULTS ACROSS THE WHOLE
LEVEL RESULTS ACROSS THE WHOLE DATASET, IT IS LOOKING DEEPER
DATASET, IT IS LOOKING DEEPER
DATASET, IT IS LOOKING DEEPER THAN THAT.
THAN THAT.
THAN THAT. AT INDIVIDUAL SLICES OF OUR
AT INDIVIDUAL SLICES OF OUR
AT INDIVIDUAL SLICES OF OUR DATASET.
DATASET.
DATASET. THAT'S IMPORTANT BECAUSE THE
THAT'S IMPORTANT BECAUSE THE
THAT'S IMPORTANT BECAUSE THE EXPERIENCE THAT EACH USER OF
EXPERIENCE THAT EACH USER OF
EXPERIENCE THAT EACH USER OF OUR MODEL HAS WILL DEPEND ON
OUR MODEL HAS WILL DEPEND ON
OUR MODEL HAS WILL DEPEND ON THEIR INDIVIDUAL DATA POINT.
THEIR INDIVIDUAL DATA POINT.
THEIR INDIVIDUAL DATA POINT. OUR MODEL MAY DO WELL OVER OUR
OUR MODEL MAY DO WELL OVER OUR
OUR MODEL MAY DO WELL OVER OUR ENTIRE DATASET, BUT IF IT DOES
ENTIRE DATASET, BUT IF IT DOES
ENTIRE DATASET, BUT IF IT DOES POORLY ON A DATA POINT THAT A
POORLY ON A DATA POINT THAT A
POORLY ON A DATA POINT THAT A USER GIVES IT.
USER GIVES IT.
USER GIVES IT. THAT USER'S EXPERIENCE IS POOR.
THAT USER'S EXPERIENCE IS POOR.
THAT USER'S EXPERIENCE IS POOR. WE'LL TALK ABOUT THIS MORE IN
WE'LL TALK ABOUT THIS MORE IN
WE'LL TALK ABOUT THIS MORE IN OUR NEXT EPISODE.
OUR NEXT EPISODE.
OUR NEXT EPISODE. SO NOW THAT WE'VE LOOKED AT OUR
SO NOW THAT WE'VE LOOKED AT OUR
SO NOW THAT WE'VE LOOKED AT OUR MODEL'S PERFORMANCE, SHOULD WE
MODEL'S PERFORMANCE, SHOULD WE
MODEL'S PERFORMANCE, SHOULD WE PUSH IT TO PRODUCTION?
PUSH IT TO PRODUCTION?
PUSH IT TO PRODUCTION? IS IT BETTER OR WORSE THAN WHAT
IS IT BETTER OR WORSE THAN WHAT
IS IT BETTER OR WORSE THAN WHAT WE ALREADY HAVE IN PRODUCTION?
WE ALREADY HAVE IN PRODUCTION?
WE ALREADY HAVE IN PRODUCTION? WE DON'T WANT TO PUSH A WORSE
WE DON'T WANT TO PUSH A WORSE
WE DON'T WANT TO PUSH A WORSE MODEL JUST BECAUSE IT'S NEW.
MODEL JUST BECAUSE IT'S NEW.
MODEL JUST BECAUSE IT'S NEW. SO THE MODEL VALIDATOR
SO THE MODEL VALIDATOR
SO THE MODEL VALIDATOR COMPONENT USES BEAM TO DO THAT
COMPONENT USES BEAM TO DO THAT
COMPONENT USES BEAM TO DO THAT COMPARISON USING CRITERIA THAT
COMPARISON USING CRITERIA THAT
COMPARISON USING CRITERIA THAT WE DEFINE TO DECIDE WHETHER OR
WE DEFINE TO DECIDE WHETHER OR
WE DEFINE TO DECIDE WHETHER OR NOT TO PUSH THE NEW MODEL TO
NOT TO PUSH THE NEW MODEL TO
NOT TO PUSH THE NEW MODEL TO PRODUCTION.
PRODUCTION.
PRODUCTION. IF MODEL VALIDATOR DECIDES THAT
IF MODEL VALIDATOR DECIDES THAT
IF MODEL VALIDATOR DECIDES THAT OUR NEW MODEL IS READY FOR
OUR NEW MODEL IS READY FOR
OUR NEW MODEL IS READY FOR PRODUCTION, THEN PUSHER DOES
PRODUCTION, THEN PUSHER DOES
PRODUCTION, THEN PUSHER DOES THE WORK OF ACTUALLY PUSHING IT
THE WORK OF ACTUALLY PUSHING IT
THE WORK OF ACTUALLY PUSHING IT TO OUR DEPLOYMENT TARGETS.
TO OUR DEPLOYMENT TARGETS.
TO OUR DEPLOYMENT TARGETS. THOSE TARGETS COULD BE
THOSE TARGETS COULD BE
THOSE TARGETS COULD BE TENSORFLOW LIGHT IN WE'RE DOING
TENSORFLOW LIGHT IN WE'RE DOING
TENSORFLOW LIGHT IN WE'RE DOING A MOBILE APPLICATION OR
A MOBILE APPLICATION OR
A MOBILE APPLICATION OR TENSORFLOW JS IF WE'RE
TENSORFLOW JS IF WE'RE
TENSORFLOW JS IF WE'RE DEPLOYING TO A JAVASCRIPT
DEPLOYING TO A JAVASCRIPT
DEPLOYING TO A JAVASCRIPT ENVIRONMENT OR TENSORFLOW
ENVIRONMENT OR TENSORFLOW
ENVIRONMENT OR TENSORFLOW SERVING IF WE ARE DEPLOYING TO
SERVING IF WE ARE DEPLOYING TO
SERVING IF WE ARE DEPLOYING TO A SERVER FARM OR ALL OF THE
A SERVER FARM OR ALL OF THE
A SERVER FARM OR ALL OF THE ABOVE.
ABOVE.
ABOVE. SO NOW HOPEFULLY WE'VE GIVEN
SO NOW HOPEFULLY WE'VE GIVEN
SO NOW HOPEFULLY WE'VE GIVEN YOU A BASIC UNDERSTANDING OF
YOU A BASIC UNDERSTANDING OF
YOU A BASIC UNDERSTANDING OF TFX AND ML PIPELINES IN GENERAL
TFX AND ML PIPELINES IN GENERAL
TFX AND ML PIPELINES IN GENERAL AND SOME OF THE ISSUES AROUND
AND SOME OF THE ISSUES AROUND
AND SOME OF THE ISSUES AROUND PUTTING AN ML APPLICATION INTO
PUTTING AN ML APPLICATION INTO
PUTTING AN ML APPLICATION INTO PRODUCTION.
PRODUCTION.
PRODUCTION. AND REMEMBER, TFX IS OPEN
AND REMEMBER, TFX IS OPEN
AND REMEMBER, TFX IS OPEN SOURCE.
SOURCE.
SOURCE. WE WANT YOU TO HELP CONTRIBUTE
WE WANT YOU TO HELP CONTRIBUTE
WE WANT YOU TO HELP CONTRIBUTE TO MAKING TFX BETTER.
TO MAKING TFX BETTER.
TO MAKING TFX BETTER. IN OUR NEXT EPISODE, WE'LL LOOK
IN OUR NEXT EPISODE, WE'LL LOOK
IN OUR NEXT EPISODE, WE'LL LOOK AT A REAL WORLD EXAMPLE OF WHY
AT A REAL WORLD EXAMPLE OF WHY
AT A REAL WORLD EXAMPLE OF WHY ANALYZING MODEL PERFORMANCE IS
ANALYZING MODEL PERFORMANCE IS
ANALYZING MODEL PERFORMANCE IS IMPORTANT TO YOUR BUSINESS.
IMPORTANT TO YOUR BUSINESS.
IMPORTANT TO YOUR BUSINESS. FOR MORE INFORMATION ON TFX,
FOR MORE INFORMATION ON TFX,
FOR MORE INFORMATION ON TFX, VISIT US AS TENSORFLOW.ORG
VISIT US AS TENSORFLOW.ORG
VISIT US AS TENSORFLOW.ORG SLASH TFX AND DON'T FORGET TO
SLASH TFX AND DON'T FORGET TO
SLASH TFX AND DON'T FORGET TO COMMENT AND LIKE US BELOW AND
COMMENT AND LIKE US BELOW AND
COMMENT AND LIKE US BELOW AND THANKS FOR WATCHING.
>> GOOD AFTERNOON.
>> GOOD AFTERNOON.
>> GOOD AFTERNOON. I WOULD LIKE TO WELCOME YOU TO
I WOULD LIKE TO WELCOME YOU TO
I WOULD LIKE TO WELCOME YOU TO TENSORFLOW MODEL OPTIMIZATION,
TENSORFLOW MODEL OPTIMIZATION,
TENSORFLOW MODEL OPTIMIZATION, QUAUNTIZATION AND PRUNING.
QUAUNTIZATION AND PRUNING.
QUAUNTIZATION AND PRUNING. A TALK GIVEN MY MY COLLEAGUE AT
A TALK GIVEN MY MY COLLEAGUE AT
A TALK GIVEN MY MY COLLEAGUE AT OOG WILL, RAZIEL ALVEREZ.
OOG WILL, RAZIEL ALVEREZ.
OOG WILL, RAZIEL ALVEREZ. >> HI.
>> HI.
>> HI. MY NAME IS RAZIEL.
MY NAME IS RAZIEL.
MY NAME IS RAZIEL. I AM TALKING ABOUT OPTIMIZATION
I AM TALKING ABOUT OPTIMIZATION
I AM TALKING ABOUT OPTIMIZATION AND TALK IN PARTICULAR AROUND
AND TALK IN PARTICULAR AROUND
AND TALK IN PARTICULAR AROUND TECHNIQUES AROUND QUANTIZATION
TECHNIQUES AROUND QUANTIZATION
TECHNIQUES AROUND QUANTIZATION AND PRUNING.
AND PRUNING.
AND PRUNING. SO FIRST I WILL INTRODUCE WHY
SO FIRST I WILL INTRODUCE WHY
SO FIRST I WILL INTRODUCE WHY WHAT IS MORE OPTIMIZATION AND
WHAT IS MORE OPTIMIZATION AND
WHAT IS MORE OPTIMIZATION AND PRUNING AND WHY WE THINK IT'S
PRUNING AND WHY WE THINK IT'S
PRUNING AND WHY WE THINK IT'S IMPORTANT WHY WE'RE INVESTING
IMPORTANT WHY WE'RE INVESTING
IMPORTANT WHY WE'RE INVESTING IN THIS AREA.
IN THIS AREA.
IN THIS AREA. I WILL COVER THE TOOLS THAT WE
I WILL COVER THE TOOLS THAT WE
I WILL COVER THE TOOLS THAT WE HAVE AVAILABLE AND AT THE END I
HAVE AVAILABLE AND AT THE END I
HAVE AVAILABLE AND AT THE END I WILL GIVE AN OVERVIEW ON OUR
WILL GIVE AN OVERVIEW ON OUR
WILL GIVE AN OVERVIEW ON OUR ROADMAP FOR THE SHORT TERM AND
ROADMAP FOR THE SHORT TERM AND
ROADMAP FOR THE SHORT TERM AND THE LONGER TERM.
THE LONGER TERM.
THE LONGER TERM. AND AT THE END OF THE
AND AT THE END OF THE
AND AT THE END OF THE PRESENTATION WE HAVE SOME
PRESENTATION WE HAVE SOME
PRESENTATION WE HAVE SOME MINUTES TO GO OVER Q & A.
MINUTES TO GO OVER Q & A.
MINUTES TO GO OVER Q & A. SO IT SHOULD ALLOW YOU TO
SO IT SHOULD ALLOW YOU TO
SO IT SHOULD ALLOW YOU TO OPTIMIZE MACHINE LEARNING MODEL
OPTIMIZE MACHINE LEARNING MODEL
OPTIMIZE MACHINE LEARNING MODEL FOR DEPLOYMENT AND EXECUTION.
FOR DEPLOYMENT AND EXECUTION.
FOR DEPLOYMENT AND EXECUTION. WE THINK THIS IS IMPORTANT
WE THINK THIS IS IMPORTANT
WE THINK THIS IS IMPORTANT BECAUSE MACHINE LEARNING IS A
BECAUSE MACHINE LEARNING IS A
BECAUSE MACHINE LEARNING IS A VERY IMPORTANT FIELD AND WE
VERY IMPORTANT FIELD AND WE
VERY IMPORTANT FIELD AND WE THINK THAT THERE IS A LOT OF
THINK THAT THERE IS A LOT OF
THINK THAT THERE IS A LOT OF ROOM TO MAKE IT MORE EFFICIENT
ROOM TO MAKE IT MORE EFFICIENT
ROOM TO MAKE IT MORE EFFICIENT AND THIS HAS SOME IMPLICATIONS
AND THIS HAS SOME IMPLICATIONS
AND THIS HAS SOME IMPLICATIONS BOTH ECONOMY.
BOTH ECONOMY.
BOTH ECONOMY. YOU CAN MAKE ALL THESE
YOU CAN MAKE ALL THESE
YOU CAN MAKE ALL THESE APPLICATIONS MUCH BETTER, RIGHT?
APPLICATIONS MUCH BETTER, RIGHT?
APPLICATIONS MUCH BETTER, RIGHT? LIKE THE QUALITY OR CHEAPER TO
LIKE THE QUALITY OR CHEAPER TO
LIKE THE QUALITY OR CHEAPER TO EXECUTE.
EXECUTE.
EXECUTE. WE CAN ENABLE SOME NEW MODELS
WE CAN ENABLE SOME NEW MODELS
WE CAN ENABLE SOME NEW MODELS AND NEW DEPLOYMENT AND NEW
AND NEW DEPLOYMENT AND NEW
AND NEW DEPLOYMENT AND NEW PROTOCOL EVEN IF YOU JUST TRIED
PROTOCOL EVEN IF YOU JUST TRIED
PROTOCOL EVEN IF YOU JUST TRIED TO DO -- EXECUTE THIS MUCH
TO DO -- EXECUTE THIS MUCH
TO DO -- EXECUTE THIS MUCH LEARNING MODE ON THE SERVER.
LEARNING MODE ON THE SERVER.
LEARNING MODE ON THE SERVER. SO CURRENTLY MACHINE LEARNING
SO CURRENTLY MACHINE LEARNING
SO CURRENTLY MACHINE LEARNING RUNS EITHER ON THE SERVER OR ON
RUNS EITHER ON THE SERVER OR ON
RUNS EITHER ON THE SERVER OR ON THE EDGE.
THE EDGE.
THE EDGE. ON THE SERVER YOU MAY THINK
ON THE SERVER YOU MAY THINK
ON THE SERVER YOU MAY THINK THERE IS A LOT OF CAPACITY AND
THERE IS A LOT OF CAPACITY AND
THERE IS A LOT OF CAPACITY AND A LOT OF COMPUTER MEMORY.
A LOT OF COMPUTER MEMORY.
A LOT OF COMPUTER MEMORY. WHAT IS THE BENEFIT OF THESE
WHAT IS THE BENEFIT OF THESE
WHAT IS THE BENEFIT OF THESE MODELS?
MODELS?
MODELS? APPLICATIONS ARE STILL BOUND BY
APPLICATIONS ARE STILL BOUND BY
APPLICATIONS ARE STILL BOUND BY LATENCY.
LATENCY.
LATENCY. STILL A VERY IMPORTANT METRIC
STILL A VERY IMPORTANT METRIC
STILL A VERY IMPORTANT METRIC FOR A LOT OF APPLICATIONS OR
FOR A LOT OF APPLICATIONS OR
FOR A LOT OF APPLICATIONS OR YOU WANT TO IMPROVE HOW MANY
YOU WANT TO IMPROVE HOW MANY
YOU WANT TO IMPROVE HOW MANY TASKS CAN RUN ON YOUR SERVER.
TASKS CAN RUN ON YOUR SERVER.
TASKS CAN RUN ON YOUR SERVER. AND THESE TWO ARE ALSO RELATED
AND THESE TWO ARE ALSO RELATED
AND THESE TWO ARE ALSO RELATED TO MONEY.
TO MONEY.
TO MONEY. EVERYBODY WILL WANT TO SAVE
EVERYBODY WILL WANT TO SAVE
EVERYBODY WILL WANT TO SAVE MONEY AND POTENTIALLY WE'RE
MONEY AND POTENTIALLY WE'RE
MONEY AND POTENTIALLY WE'RE TALKING A LOT OF MONEY.
TALKING A LOT OF MONEY.
TALKING A LOT OF MONEY. NOW ON THE EDGE IS A LITTLE
NOW ON THE EDGE IS A LITTLE
NOW ON THE EDGE IS A LITTLE MORE OBVIOUS WHEN YOU HAVE
MORE OBVIOUS WHEN YOU HAVE
MORE OBVIOUS WHEN YOU HAVE OPTIMIZATION.
OPTIMIZATION.
OPTIMIZATION. CONSTRAINED ENVIRONMENT AS YOU
CONSTRAINED ENVIRONMENT AS YOU
CONSTRAINED ENVIRONMENT AS YOU TALK ABOUT APPLICATIONS IN
TALK ABOUT APPLICATIONS IN
TALK ABOUT APPLICATIONS IN GENERAL.
GENERAL.
GENERAL. WE NEED TO DEAL WITH REDUCED
WE NEED TO DEAL WITH REDUCED
WE NEED TO DEAL WITH REDUCED MEMORY, POWER CON SUMS,
MEMORY, POWER CON SUMS,
MEMORY, POWER CON SUMS, BANDWIDTH BOTH DOWNLOADING
BANDWIDTH BOTH DOWNLOADING
BANDWIDTH BOTH DOWNLOADING MODELS FROM THE CLOUD OR EVEN
MODELS FROM THE CLOUD OR EVEN
MODELS FROM THE CLOUD OR EVEN WE'VE SEEN TO BE ABLE TO
WE'VE SEEN TO BE ABLE TO
WE'VE SEEN TO BE ABLE TO TRANSFER FROM MEMORY TO THE
TRANSFER FROM MEMORY TO THE
TRANSFER FROM MEMORY TO THE PROCESSOR.
PROCESSOR.
PROCESSOR. THIS COULD BE A PROBLEM IF THE
THIS COULD BE A PROBLEM IF THE
THIS COULD BE A PROBLEM IF THE MODEL IS TOO LARGE.
MODEL IS TOO LARGE.
MODEL IS TOO LARGE. PLUS WE HAVE A WIDE VARIETY OF
PLUS WE HAVE A WIDE VARIETY OF
PLUS WE HAVE A WIDE VARIETY OF HARDWARE.
HARDWARE.
HARDWARE. MORE THAN IN THE SERVER AND WE
MORE THAN IN THE SERVER AND WE
MORE THAN IN THE SERVER AND WE NEED TO MAKE SURE THAT THESE
NEED TO MAKE SURE THAT THESE
NEED TO MAKE SURE THAT THESE MODELS RUN EFFICIENTLY ON ALL
MODELS RUN EFFICIENTLY ON ALL
MODELS RUN EFFICIENTLY ON ALL THESE DIFFERENT TYPES OF
THESE DIFFERENT TYPES OF
THESE DIFFERENT TYPES OF HARDWARE.
HARDWARE.
HARDWARE. SO IT FOLLOWS IF WE ARE
SO IT FOLLOWS IF WE ARE
SO IT FOLLOWS IF WE ARE OPTIMIZING THIS MODEL WE CAN
OPTIMIZING THIS MODEL WE CAN
OPTIMIZING THIS MODEL WE CAN HAVE BETTER MODELS AND THEN
HAVE BETTER MODELS AND THEN
HAVE BETTER MODELS AND THEN EVENTUALLY STARTS TRANSLATING
EVENTUALLY STARTS TRANSLATING
EVENTUALLY STARTS TRANSLATING IN NEW PRODUCTS THAT OTHERWISE
IN NEW PRODUCTS THAT OTHERWISE
IN NEW PRODUCTS THAT OTHERWISE COULDN'T EXIST IF WE JUST WERE
COULDN'T EXIST IF WE JUST WERE
COULDN'T EXIST IF WE JUST WERE RUNNING THESE MODELS ON A
RUNNING THESE MODELS ON A
RUNNING THESE MODELS ON A SERVER.
SERVER.
SERVER. AND THESE OPPORTUNITIES ARE
AND THESE OPPORTUNITIES ARE
AND THESE OPPORTUNITIES ARE LARGER THAN JUST -- MACHINE
LARGER THAN JUST -- MACHINE
LARGER THAN JUST -- MACHINE LEARNING IS TRICKLING DOWN INTO
LEARNING IS TRICKLING DOWN INTO
LEARNING IS TRICKLING DOWN INTO MORE ENVIRONMENTS.
MORE ENVIRONMENTS.
MORE ENVIRONMENTS. WE HAVE MACHINE LEARNING MODELS
WE HAVE MACHINE LEARNING MODELS
WE HAVE MACHINE LEARNING MODELS THAT ARE USED TO DETECT
THAT ARE USED TO DETECT
THAT ARE USED TO DETECT FAILURES IN MACHINERY IN
FAILURES IN MACHINERY IN
FAILURES IN MACHINERY IN FACTORIES OR WE USE IT IN
FACTORIES OR WE USE IT IN
FACTORIES OR WE USE IT IN SELF-DRIVING CARS.
SELF-DRIVING CARS.
SELF-DRIVING CARS. WE USE IT IN THE OFFICE TO SCAN
WE USE IT IN THE OFFICE TO SCAN
WE USE IT IN THE OFFICE TO SCAN DOCUMENTS AND TRY TO UNDERSTAND
DOCUMENTS AND TRY TO UNDERSTAND
DOCUMENTS AND TRY TO UNDERSTAND THEM.
THEM.
THEM. AND JUST TO GIVE YOU SOME
AND JUST TO GIVE YOU SOME
AND JUST TO GIVE YOU SOME NUMBERS, THE SIZE OF THE CELL
NUMBERS, THE SIZE OF THE CELL
NUMBERS, THE SIZE OF THE CELL PHONE MARKET IS REALLY A
PHONE MARKET IS REALLY A
PHONE MARKET IS REALLY A FRACTION OF THE POTENTIAL FOR
FRACTION OF THE POTENTIAL FOR
FRACTION OF THE POTENTIAL FOR THE EDGE DEVICES IN GENERAL.
THE EDGE DEVICES IN GENERAL.
THE EDGE DEVICES IN GENERAL. SO BASICALLY THE TWO REASONS IS
SO BASICALLY THE TWO REASONS IS
SO BASICALLY THE TWO REASONS IS WE WANT TO MAKE MACHINE
WE WANT TO MAKE MACHINE
WE WANT TO MAKE MACHINE LEARNING MORE EFFICIENT.
LEARNING MORE EFFICIENT.
LEARNING MORE EFFICIENT. IT IS ALREADY VERY IMPORTANT
IT IS ALREADY VERY IMPORTANT
IT IS ALREADY VERY IMPORTANT FOR THE SERVERS.
FOR THE SERVERS.
FOR THE SERVERS. PRETTY CRUCIAL FOR EMBEDDED
PRETTY CRUCIAL FOR EMBEDDED
PRETTY CRUCIAL FOR EMBEDDED DEVICES.
DEVICES.
DEVICES. SO WE STARTED THIS TOOLKIT
SO WE STARTED THIS TOOLKIT
SO WE STARTED THIS TOOLKIT ABOUT A YEAR AGO.
ABOUT A YEAR AGO.
ABOUT A YEAR AGO. WE INITIALLY LAUNCHED TRAINING
WE INITIALLY LAUNCHED TRAINING
WE INITIALLY LAUNCHED TRAINING WITH THESE HYBRID AND NOW GOING
WITH THESE HYBRID AND NOW GOING
WITH THESE HYBRID AND NOW GOING INTO MORE DETAIL IN THE
INTO MORE DETAIL IN THE
INTO MORE DETAIL IN THE PRESENTATION.
PRESENTATION.
PRESENTATION. EARLIER THIS YEAR WE LAUNCHED
EARLIER THIS YEAR WE LAUNCHED
EARLIER THIS YEAR WE LAUNCHED API FOR NEURAL CONNECTION
API FOR NEURAL CONNECTION
API FOR NEURAL CONNECTION PRUNING AND WE CREATED THIS
PRUNING AND WE CREATED THIS
PRUNING AND WE CREATED THIS OPTIMIZATION FOR TENSORFLOW
OPTIMIZATION FOR TENSORFLOW
OPTIMIZATION FOR TENSORFLOW LIGHT AND WE LAUNCHED THOSE
LIGHT AND WE LAUNCHED THOSE
LIGHT AND WE LAUNCHED THOSE QUANTIZATION FOR TARGETING
QUANTIZATION FOR TARGETING
QUANTIZATION FOR TARGETING SPECICATION.
SPECICATION.
SPECICATION. MORE RECENTLY REDUCED FLOW
MORE RECENTLY REDUCED FLOW
MORE RECENTLY REDUCED FLOW PRECISION AND HOPEFULLY SOON
PRECISION AND HOPEFULLY SOON
PRECISION AND HOPEFULLY SOON WE'RE GOING TO BE LAUNCHING
WE'RE GOING TO BE LAUNCHING
WE'RE GOING TO BE LAUNCHING QUANTIZATION TRAINING API AND
QUANTIZATION TRAINING API AND
QUANTIZATION TRAINING API AND TO APPLY FOR SPARSE COMPUTATION.
TO APPLY FOR SPARSE COMPUTATION.
TO APPLY FOR SPARSE COMPUTATION. NOW WE GO INTO THESE TECHNIQUES
NOW WE GO INTO THESE TECHNIQUES
NOW WE GO INTO THESE TECHNIQUES AND TOOLS IN A LITTLE BIT MORE
AND TOOLS IN A LITTLE BIT MORE
AND TOOLS IN A LITTLE BIT MORE DETAIL.
DETAIL.
DETAIL. LET'S START WITH QUANTIZATION.
LET'S START WITH QUANTIZATION.
LET'S START WITH QUANTIZATION. FIRST I THINK IT'S IMPORTANT IF
FIRST I THINK IT'S IMPORTANT IF
FIRST I THINK IT'S IMPORTANT IF WE HAVE SOME AT LEAST BASIC
WE HAVE SOME AT LEAST BASIC
WE HAVE SOME AT LEAST BASIC UNDERSTANDING OF WHAT IS
UNDERSTANDING OF WHAT IS
UNDERSTANDING OF WHAT IS QUANTIZATION AND WHY WE ARE
QUANTIZATION AND WHY WE ARE
QUANTIZATION AND WHY WE ARE APPROACHING OUR TOOLS THE WAY
APPROACHING OUR TOOLS THE WAY
APPROACHING OUR TOOLS THE WAY WE'RE DOING IT.
WE'RE DOING IT.
WE'RE DOING IT. LET'S START WITH THIS.
LET'S START WITH THIS.
LET'S START WITH THIS. MULTIPLY ITS BASIC OPERATION
MULTIPLY ITS BASIC OPERATION
MULTIPLY ITS BASIC OPERATION FOR MACHINE LEARNING MODELS.
FOR MACHINE LEARNING MODELS.
FOR MACHINE LEARNING MODELS. YOU HAVE TWO MAJOR TENSORFLOW A
YOU HAVE TWO MAJOR TENSORFLOW A
YOU HAVE TWO MAJOR TENSORFLOW A AND B AND YOU DO SOME MULTIPLY
AND B AND YOU DO SOME MULTIPLY
AND B AND YOU DO SOME MULTIPLY INDICATION.
INDICATION.
INDICATION. IT'S A BUNCH OF VALUES THAT
IT'S A BUNCH OF VALUES THAT
IT'S A BUNCH OF VALUES THAT PRODUCE THE THIRD TENSOR.
PRODUCE THE THIRD TENSOR.
PRODUCE THE THIRD TENSOR. THEN JUST A LITTLE REMINDER
THEN JUST A LITTLE REMINDER
THEN JUST A LITTLE REMINDER ABOUT HOW THIS MULTIPLY WORKS.
ABOUT HOW THIS MULTIPLY WORKS.
ABOUT HOW THIS MULTIPLY WORKS. EACH ONE OF THESE RESULTS OF
EACH ONE OF THESE RESULTS OF
EACH ONE OF THESE RESULTS OF THE TENSOR C ARE COMPUTED AS
THE TENSOR C ARE COMPUTED AS
THE TENSOR C ARE COMPUTED AS MULTIPLICATION AND ACCUMULATION.
MULTIPLICATION AND ACCUMULATION.
MULTIPLICATION AND ACCUMULATION. SO IF WE LOOK AT ONE OF THEM
SO IF WE LOOK AT ONE OF THEM
SO IF WE LOOK AT ONE OF THEM THEN WE THINK OUR TRAINING
THEN WE THINK OUR TRAINING
THEN WE THINK OUR TRAINING THESE MODELS TYPICALLY IN
THESE MODELS TYPICALLY IN
THESE MODELS TYPICALLY IN HIGHER PRECISION, FLOW 32, YOU
HIGHER PRECISION, FLOW 32, YOU
HIGHER PRECISION, FLOW 32, YOU FOLLOW THE -- -- THE
FOLLOW THE -- -- THE
FOLLOW THE -- -- THE ACCUMULATION WILL ALSO BE FLOW
ACCUMULATION WILL ALSO BE FLOW
ACCUMULATION WILL ALSO BE FLOW 32.
32.
32. SO THIS IS -- (?) MACHINE
SO THIS IS -- (?) MACHINE
SO THIS IS -- (?) MACHINE LEARNING IS PRETTY GOOD DEALING
LEARNING IS PRETTY GOOD DEALING
LEARNING IS PRETTY GOOD DEALING WITH IT AT THESE LEVELS OF
WITH IT AT THESE LEVELS OF
WITH IT AT THESE LEVELS OF PRECISION SO NO PROBLEM.
PRECISION SO NO PROBLEM.
PRECISION SO NO PROBLEM. WHAT DOES IT HAVE TO DO WITH
WHAT DOES IT HAVE TO DO WITH
WHAT DOES IT HAVE TO DO WITH QUANTIZATION?
QUANTIZATION?
QUANTIZATION? WHAT OUR GOALS FOR OPTIMIZATION.
WHAT OUR GOALS FOR OPTIMIZATION.
WHAT OUR GOALS FOR OPTIMIZATION. WE WANT TO BE ABLE TO ADDRESS
WE WANT TO BE ABLE TO ADDRESS
WE WANT TO BE ABLE TO ADDRESS ALL THESE RESTRICTIONS.
ALL THESE RESTRICTIONS.
ALL THESE RESTRICTIONS. ALSO WE WANT TO BE ABLE TO
ALSO WE WANT TO BE ABLE TO
ALSO WE WANT TO BE ABLE TO DEPLOY AS MUCH HARDWARE AS
DEPLOY AS MUCH HARDWARE AS
DEPLOY AS MUCH HARDWARE AS POSSIBLE.
POSSIBLE.
POSSIBLE. SO COMMON THING WE DO IS LET'S
SO COMMON THING WE DO IS LET'S
SO COMMON THING WE DO IS LET'S REVIEW THE OPTIMIZATION WE'RE
REVIEW THE OPTIMIZATION WE'RE
REVIEW THE OPTIMIZATION WE'RE OPRATING.
OPRATING.
OPRATING. LET'S SAY GO FROM THE 32-BIT
LET'S SAY GO FROM THE 32-BIT
LET'S SAY GO FROM THE 32-BIT FLOWS TO -- (?) AND -- AND
FLOWS TO -- (?) AND -- AND
FLOWS TO -- (?) AND -- AND THESE WILL BE GOOD BECAUSE THEN
THESE WILL BE GOOD BECAUSE THEN
THESE WILL BE GOOD BECAUSE THEN WE ARE GOING FROM 32 BITS.
WE ARE GOING FROM 32 BITS.
WE ARE GOING FROM 32 BITS. SO WE'RE USING MEMORY, THE
SO WE'RE USING MEMORY, THE
SO WE'RE USING MEMORY, THE MODELS ARE FOUR TIMES SMALLER.
MODELS ARE FOUR TIMES SMALLER.
MODELS ARE FOUR TIMES SMALLER. AND THESE OPERATIONS ARE FASTER
AND THESE OPERATIONS ARE FASTER
AND THESE OPERATIONS ARE FASTER TO EXECUTE AND CONSUME LESS
TO EXECUTE AND CONSUME LESS
TO EXECUTE AND CONSUME LESS POWER.
POWER.
POWER. AND THEN BECAUSE THE PARAMETERS
AND THEN BECAUSE THE PARAMETERS
AND THEN BECAUSE THE PARAMETERS AND ALSO THE DYNAMIC VALUES ARE
AND ALSO THE DYNAMIC VALUES ARE
AND ALSO THE DYNAMIC VALUES ARE SMALLER THAN WE REDUCED -- IT
SMALLER THAN WE REDUCED -- IT
SMALLER THAN WE REDUCED -- IT MEANS THERE IS MORE ROOM FOR
MEANS THERE IS MORE ROOM FOR
MEANS THERE IS MORE ROOM FOR THINGS TO FLOW AROUND WHICH CAN
THINGS TO FLOW AROUND WHICH CAN
THINGS TO FLOW AROUND WHICH CAN ALSO TRANSLATE TO FASTER
ALSO TRANSLATE TO FASTER
ALSO TRANSLATE TO FASTER COMPUTERS AND REDUCED POWER.
COMPUTERS AND REDUCED POWER.
COMPUTERS AND REDUCED POWER. OPERATIONS THE END TO BE FAIRLY
OPERATIONS THE END TO BE FAIRLY
OPERATIONS THE END TO BE FAIRLY COMMON DENOMINATOR ACROSS
COMMON DENOMINATOR ACROSS
COMMON DENOMINATOR ACROSS HARDWARE.
HARDWARE.
HARDWARE. CPU, DIFFERENT APU SUPPORT THE
CPU, DIFFERENT APU SUPPORT THE
CPU, DIFFERENT APU SUPPORT THE OPERATIONS.
OPERATIONS.
OPERATIONS. SO WE HAVE -- WE WILL REVIEW
SO WE HAVE -- WE WILL REVIEW
SO WE HAVE -- WE WILL REVIEW THE PRECISION.
THE PRECISION.
THE PRECISION. HOW DO WE CONVERT 32 BITS FLOW.
HOW DO WE CONVERT 32 BITS FLOW.
HOW DO WE CONVERT 32 BITS FLOW. WE DO SOMETHING RIGHT NOW VERY
WE DO SOMETHING RIGHT NOW VERY
WE DO SOMETHING RIGHT NOW VERY SIMPLE, RIGHT?
SIMPLE, RIGHT?
SIMPLE, RIGHT? WE HAVE THIS LINEAR MAPPING
WE HAVE THIS LINEAR MAPPING
WE HAVE THIS LINEAR MAPPING WHERE WE SAY WE TAKE THE VALUES
WHERE WE SAY WE TAKE THE VALUES
WHERE WE SAY WE TAKE THE VALUES FROM A TENSOR AND COMPUTE THE
FROM A TENSOR AND COMPUTE THE
FROM A TENSOR AND COMPUTE THE MINIMUM AND MAXIMUM VALUE AND
MINIMUM AND MAXIMUM VALUE AND
MINIMUM AND MAXIMUM VALUE AND BASED ON THAT WE SPREAD THEM
BASED ON THAT WE SPREAD THEM
BASED ON THAT WE SPREAD THEM EVENLY.
EVENLY.
EVENLY. VERY SIMPLE, RIGHT?
VERY SIMPLE, RIGHT?
VERY SIMPLE, RIGHT? THESE ARE ALL WE NEED TO DO.
THESE ARE ALL WE NEED TO DO.
THESE ARE ALL WE NEED TO DO. WELL, I WISH.
WELL, I WISH.
WELL, I WISH. IT IS NOT THAT SIMPLE.
IT IS NOT THAT SIMPLE.
IT IS NOT THAT SIMPLE. HERE IS AN EXAMPLE.
HERE IS AN EXAMPLE.
HERE IS AN EXAMPLE. WE HAVE THE MULTIPLIER AND NOW
WE HAVE THE MULTIPLIER AND NOW
WE HAVE THE MULTIPLIER AND NOW LET'S SAY WE -- (?) THE
LET'S SAY WE -- (?) THE
LET'S SAY WE -- (?) THE MULTIPLIERS ARE -- AND THEN THE
MULTIPLIERS ARE -- AND THEN THE
MULTIPLIERS ARE -- AND THEN THE MULTIPLICATION THE PRODUCTS NOW
MULTIPLICATION THE PRODUCTS NOW
MULTIPLICATION THE PRODUCTS NOW YOU NEED 16 BITS REPRESENTED
YOU NEED 16 BITS REPRESENTED
YOU NEED 16 BITS REPRESENTED AND THEN YOU NEED TO ACCUMULATE
AND THEN YOU NEED TO ACCUMULATE
AND THEN YOU NEED TO ACCUMULATE AND YOU PROBABLY WANT 32 BITS
AND YOU PROBABLY WANT 32 BITS
AND YOU PROBABLY WANT 32 BITS TO ACCUMULATE, RIGHT?
TO ACCUMULATE, RIGHT?
TO ACCUMULATE, RIGHT? SO THE PROBLEM IS NOW YOU HAVE
SO THE PROBLEM IS NOW YOU HAVE
SO THE PROBLEM IS NOW YOU HAVE TENSOR RECEIVE FULL OF 32-BIT
TENSOR RECEIVE FULL OF 32-BIT
TENSOR RECEIVE FULL OF 32-BIT VALUES AND THAT IS NOT GREAT
VALUES AND THAT IS NOT GREAT
VALUES AND THAT IS NOT GREAT WHEN YOU WANT TO FEED THAT INTO
WHEN YOU WANT TO FEED THAT INTO
WHEN YOU WANT TO FEED THAT INTO ANOTHER -- YOU WANT TO EXECUTE
ANOTHER -- YOU WANT TO EXECUTE
ANOTHER -- YOU WANT TO EXECUTE -- ALREADY -- (?) WHAT YOU DO,
-- ALREADY -- (?) WHAT YOU DO,
-- ALREADY -- (?) WHAT YOU DO, YOU SCALE IT BACK DOWN.
YOU SCALE IT BACK DOWN.
YOU SCALE IT BACK DOWN. YOU QUANTIZE THEM ON THE FLY.
YOU QUANTIZE THEM ON THE FLY.
YOU QUANTIZE THEM ON THE FLY. YOU CAN FEED THEM AND MULTIPLY
YOU CAN FEED THEM AND MULTIPLY
YOU CAN FEED THEM AND MULTIPLY AND ALL GOOD.
AND ALL GOOD.
AND ALL GOOD. BUT WHILE THE IMPLICATIONS OF
BUT WHILE THE IMPLICATIONS OF
BUT WHILE THE IMPLICATIONS OF ALL THESE PROCESS?
ALL THESE PROCESS?
ALL THESE PROCESS? WE NEED TO STUDY THE
WE NEED TO STUDY THE
WE NEED TO STUDY THE PARAMETERS, THE WEIGHTS.
PARAMETERS, THE WEIGHTS.
PARAMETERS, THE WEIGHTS. WE ARE CHANGING ALSO THE
WE ARE CHANGING ALSO THE
WE ARE CHANGING ALSO THE DYNAMIC VALUES, ACTIVATIONS
DYNAMIC VALUES, ACTIVATIONS
DYNAMIC VALUES, ACTIVATIONS BECAUSE WE ARE SCALING THEM AND
BECAUSE WE ARE SCALING THEM AND
BECAUSE WE ARE SCALING THEM AND QUANTIZING THEM ON THE FLY.
QUANTIZING THEM ON THE FLY.
QUANTIZING THEM ON THE FLY. WE'RE CHANGING THE COMPUTATION,
WE'RE CHANGING THE COMPUTATION,
WE'RE CHANGING THE COMPUTATION, SIMPLE, WE HAVE A SCALING
SIMPLE, WE HAVE A SCALING
SIMPLE, WE HAVE A SCALING OPERATION.
OPERATION.
OPERATION. IT CAN BE MORE INVOLVED.
IT CAN BE MORE INVOLVED.
IT CAN BE MORE INVOLVED. SO YOU CAN SAY OKAY THAT
SO YOU CAN SAY OKAY THAT
SO YOU CAN SAY OKAY THAT DOESN'T SEEM THAT HARD.
DOESN'T SEEM THAT HARD.
DOESN'T SEEM THAT HARD. WE JUST HAD A SCALING OPERATION
WE JUST HAD A SCALING OPERATION
WE JUST HAD A SCALING OPERATION AND IT IS JUST EASY, RIGHT?
AND IT IS JUST EASY, RIGHT?
AND IT IS JUST EASY, RIGHT? WELL, SOME MATH IS A LITTLE BIT
WELL, SOME MATH IS A LITTLE BIT
WELL, SOME MATH IS A LITTLE BIT MORE COMPLICATED THAN THAT.
MORE COMPLICATED THAN THAT.
MORE COMPLICATED THAN THAT. THESE EXAMPLE FROM
THESE EXAMPLE FROM
THESE EXAMPLE FROM NORMALIZATION OF AN LSTM AND
NORMALIZATION OF AN LSTM AND
NORMALIZATION OF AN LSTM AND THIS ONE ASIDE FROM LOOKING A
THIS ONE ASIDE FROM LOOKING A
THIS ONE ASIDE FROM LOOKING A BIT MORE COMPLEX IS AN EXAMPLE
BIT MORE COMPLEX IS AN EXAMPLE
BIT MORE COMPLEX IS AN EXAMPLE WHERE YOU APPLY THE RULES OF --
WHERE YOU APPLY THE RULES OF --
WHERE YOU APPLY THE RULES OF -- (?).
(?).
(?). BLACK HOLE WHERE THE SCALES
BLACK HOLE WHERE THE SCALES
BLACK HOLE WHERE THE SCALES CANCEL EACH OTHER AND BASICALLY
CANCEL EACH OTHER AND BASICALLY
CANCEL EACH OTHER AND BASICALLY THINGS DON'T WORK IF YOU JUST
THINGS DON'T WORK IF YOU JUST
THINGS DON'T WORK IF YOU JUST GO NAIVELY ABOUT IT.
GO NAIVELY ABOUT IT.
GO NAIVELY ABOUT IT. AND THEN IT'S MORE COMPLICATED
AND THEN IT'S MORE COMPLICATED
AND THEN IT'S MORE COMPLICATED BECAUSE IN YOUR DECISION ABOUT
BECAUSE IN YOUR DECISION ABOUT
BECAUSE IN YOUR DECISION ABOUT HOW YOU ARE GOING TO REPRESET
HOW YOU ARE GOING TO REPRESET
HOW YOU ARE GOING TO REPRESET THIS COMPUTATION AND INTO YOUR
THIS COMPUTATION AND INTO YOUR
THIS COMPUTATION AND INTO YOUR FORM WE WANT TO BE EFFICIENT.
FORM WE WANT TO BE EFFICIENT.
FORM WE WANT TO BE EFFICIENT. LOWER PERCENTAGE IS GOOD.
LOWER PERCENTAGE IS GOOD.
LOWER PERCENTAGE IS GOOD. WE ALSO WANT TO BE ACCURATE
WE ALSO WANT TO BE ACCURATE
WE ALSO WANT TO BE ACCURATE WHICH MEANS LOWER PRECISION.
WHICH MEANS LOWER PRECISION.
WHICH MEANS LOWER PRECISION. SO IT'S A LOT OF TRADEOFFS THAT
SO IT'S A LOT OF TRADEOFFS THAT
SO IT'S A LOT OF TRADEOFFS THAT YOU HAVE TO DO.
YOU HAVE TO DO.
YOU HAVE TO DO. THEN FURTHER COMPLICATING
THEN FURTHER COMPLICATING
THEN FURTHER COMPLICATING THINGS IS WE HAVE THE HARDWARE.
THINGS IS WE HAVE THE HARDWARE.
THINGS IS WE HAVE THE HARDWARE. THERE IS ALL DIFFERENT TYPES OF
THERE IS ALL DIFFERENT TYPES OF
THERE IS ALL DIFFERENT TYPES OF HARDWARE WITH DIFFERENT CAP
HARDWARE WITH DIFFERENT CAP
HARDWARE WITH DIFFERENT CAP B*ILTS READY FOR OPERATIONS AND
B*ILTS READY FOR OPERATIONS AND
B*ILTS READY FOR OPERATIONS AND EACH HARDWARE HAS DIFFERENCE
EACH HARDWARE HAS DIFFERENCE
EACH HARDWARE HAS DIFFERENCE PREFERENCES.
PREFERENCES.
PREFERENCES. SOME HARDWARE IS BETTER AT
SOME HARDWARE IS BETTER AT
SOME HARDWARE IS BETTER AT SECURING OPERATIONS AND PRODUCE
SECURING OPERATIONS AND PRODUCE
SECURING OPERATIONS AND PRODUCE LOADS OR WITH DIFFERENT WIDTHS
LOADS OR WITH DIFFERENT WIDTHS
LOADS OR WITH DIFFERENT WIDTHS AND RESTRICTIONS.
AND RESTRICTIONS.
AND RESTRICTIONS. WE WANT TO ACCOUNT FOR ALL
WE WANT TO ACCOUNT FOR ALL
WE WANT TO ACCOUNT FOR ALL THOSE THINGS WHEN WE'RE
THOSE THINGS WHEN WE'RE
THOSE THINGS WHEN WE'RE CREATING OUR QUANTIZED PROGRAM
CREATING OUR QUANTIZED PROGRAM
CREATING OUR QUANTIZED PROGRAM OR RECIPE.
OR RECIPE.
OR RECIPE. THEN THERE IS A MACHINE
THEN THERE IS A MACHINE
THEN THERE IS A MACHINE LEARNING.
LEARNING.
LEARNING. IT IS HARD TO INTERPRET.
IT IS HARD TO INTERPRET.
IT IS HARD TO INTERPRET. WE DON'T UNDERSTAND HOW IT
WE DON'T UNDERSTAND HOW IT
WE DON'T UNDERSTAND HOW IT WORKS.
WORKS.
WORKS. NOT TO A LEVEL THAT WE CAN HAVE
NOT TO A LEVEL THAT WE CAN HAVE
NOT TO A LEVEL THAT WE CAN HAVE GOOD PROOFS TO KNOW THAT A
GOOD PROOFS TO KNOW THAT A
GOOD PROOFS TO KNOW THAT A TRANSFORMATION WE'RE DOING TO
TRANSFORMATION WE'RE DOING TO
TRANSFORMATION WE'RE DOING TO THESE MODELS OR PROGRAMS WILL
THESE MODELS OR PROGRAMS WILL
THESE MODELS OR PROGRAMS WILL ACTUALLY WORK OR NOT RESULT IN
ACTUALLY WORK OR NOT RESULT IN
ACTUALLY WORK OR NOT RESULT IN A CATASTROPHIC ERROR.
A CATASTROPHIC ERROR.
A CATASTROPHIC ERROR. YOU DON'T WANT TO QUANTIZE A
YOU DON'T WANT TO QUANTIZE A
YOU DON'T WANT TO QUANTIZE A MODEL AND THEY START GIVING YOU
MODEL AND THEY START GIVING YOU
MODEL AND THEY START GIVING YOU WEIRD RESULTS.
WEIRD RESULTS.
WEIRD RESULTS. SO IT MAKES IT A LITTLE -- MUCH
SO IT MAKES IT A LITTLE -- MUCH
SO IT MAKES IT A LITTLE -- MUCH MORE COMPLICATED TO DEFINE THIS
MORE COMPLICATED TO DEFINE THIS
MORE COMPLICATED TO DEFINE THIS TRANSFORMATION.
TRANSFORMATION.
TRANSFORMATION. AND THEN FINALLY I WILL SAY
AND THEN FINALLY I WILL SAY
AND THEN FINALLY I WILL SAY THIS IS ALL MORE COMPLICATED
THIS IS ALL MORE COMPLICATED
THIS IS ALL MORE COMPLICATED BECAUSE THE MODEL IS NOT ENOUGH.
BECAUSE THE MODEL IS NOT ENOUGH.
BECAUSE THE MODEL IS NOT ENOUGH. THE PROGRAM IS NOT ENOUGH.
THE PROGRAM IS NOT ENOUGH.
THE PROGRAM IS NOT ENOUGH. DEPENDING ON HOW THE
DEPENDING ON HOW THE
DEPENDING ON HOW THE QUANTIZATION IS DEFINED YOU
QUANTIZATION IS DEFINED YOU
QUANTIZATION IS DEFINED YOU MIGHT ALSO NEED SOME EXTRA DATA.
MIGHT ALSO NEED SOME EXTRA DATA.
MIGHT ALSO NEED SOME EXTRA DATA. AN EXAMPLE OF THE MULTIPLIER WE
AN EXAMPLE OF THE MULTIPLIER WE
AN EXAMPLE OF THE MULTIPLIER WE NEED IT TO COMPUTE THE MINIMUM
NEED IT TO COMPUTE THE MINIMUM
NEED IT TO COMPUTE THE MINIMUM AND MAXIMUM VALUES OF THE
AND MAXIMUM VALUES OF THE
AND MAXIMUM VALUES OF THE DYNAMIC ACTIVATIONS, RIGHT?
DYNAMIC ACTIVATIONS, RIGHT?
DYNAMIC ACTIVATIONS, RIGHT? AND THAT ONLY CAN BE DONE IF WE
AND THAT ONLY CAN BE DONE IF WE
AND THAT ONLY CAN BE DONE IF WE RUN INFERENCES THROUGH THE
RUN INFERENCES THROUGH THE
RUN INFERENCES THROUGH THE MODEL, RIGHT?
MODEL, RIGHT?
MODEL, RIGHT? WHICH MEANS YOU NEED TO PROVIDE
WHICH MEANS YOU NEED TO PROVIDE
WHICH MEANS YOU NEED TO PROVIDE SOME REPRESENTATIVE THERE.
SOME REPRESENTATIVE THERE.
SOME REPRESENTATIVE THERE. SO BASICALLY JUST ANOTHER THING
SO BASICALLY JUST ANOTHER THING
SO BASICALLY JUST ANOTHER THING OF WHAT YOU HAVE TO ACCOUNT
OF WHAT YOU HAVE TO ACCOUNT
OF WHAT YOU HAVE TO ACCOUNT WITH QUANTIZING THIS PROGRAM.
WITH QUANTIZING THIS PROGRAM.
WITH QUANTIZING THIS PROGRAM. SO BASICALLY WHEN WE TALK ABOUT
SO BASICALLY WHEN WE TALK ABOUT
SO BASICALLY WHEN WE TALK ABOUT QUANTIZATION WE'RE TALKING
QUANTIZATION WE'RE TALKING
QUANTIZATION WE'RE TALKING ABOUT REWRITING AND
ABOUT REWRITING AND
ABOUT REWRITING AND TRANSFORMING THESE MACHINE
TRANSFORMING THESE MACHINE
TRANSFORMING THESE MACHINE LEARNING PROGRAM TO A
LEARNING PROGRAM TO A
LEARNING PROGRAM TO A REPRESENTATION BASED ON THE
REPRESENTATION BASED ON THE
REPRESENTATION BASED ON THE OPERATIONS YOU HAVE AVAILABE.
OPERATIONS YOU HAVE AVAILABE.
OPERATIONS YOU HAVE AVAILABE. SO NOW HOW ARE WE ADDRESSING
SO NOW HOW ARE WE ADDRESSING
SO NOW HOW ARE WE ADDRESSING THESE IN OUR TOOLKIT?
THESE IN OUR TOOLKIT?
THESE IN OUR TOOLKIT? WELL, THE FIRST THING THAT WE
WELL, THE FIRST THING THAT WE
WELL, THE FIRST THING THAT WE DECIDED TO DO WAS TRY TO SCOPE
DECIDED TO DO WAS TRY TO SCOPE
DECIDED TO DO WAS TRY TO SCOPE DOWN THE PROBLEM AND SAY WE'RE
DOWN THE PROBLEM AND SAY WE'RE
DOWN THE PROBLEM AND SAY WE'RE GOING TO DEFINE THE
GOING TO DEFINE THE
GOING TO DEFINE THE SPECIFICATIONS FOR COMMON
SPECIFICATIONS FOR COMMON
SPECIFICATIONS FOR COMMON OPERATIONS LIKE IN THIS CASE --
OPERATIONS LIKE IN THIS CASE --
OPERATIONS LIKE IN THIS CASE -- (?) HAVE A WELL DEFINED
(?) HAVE A WELL DEFINED
(?) HAVE A WELL DEFINED QUANTIZATION BEHAVIOR, RIGHT?
QUANTIZATION BEHAVIOR, RIGHT?
QUANTIZATION BEHAVIOR, RIGHT? WE KNOW THAT NOW WITH THIS LOW
WE KNOW THAT NOW WITH THIS LOW
WE KNOW THAT NOW WITH THIS LOW LEVEL INFORMATION THERE IS
LEVEL INFORMATION THERE IS
LEVEL INFORMATION THERE IS RELEVANT QUANTIZATION THEN HOW
RELEVANT QUANTIZATION THEN HOW
RELEVANT QUANTIZATION THEN HOW DOES THE TARGET AND
DOES THE TARGET AND
DOES THE TARGET AND SPECIFICATIONS OR TOOLS CAN
SPECIFICATIONS OR TOOLS CAN
SPECIFICATIONS OR TOOLS CAN TARGET THE SPECIFICATION AND
TARGET THE SPECIFICATION AND
TARGET THE SPECIFICATION AND THEN WE CAN ALL WORK AT THIS
THEN WE CAN ALL WORK AT THIS
THEN WE CAN ALL WORK AT THIS LEVEL AND THEN WE ALSO GET THE
LEVEL AND THEN WE ALSO GET THE
LEVEL AND THEN WE ALSO GET THE BENEFIT FROM THE USER POINT OF
BENEFIT FROM THE USER POINT OF
BENEFIT FROM THE USER POINT OF VIEW YOU CAN QUANTIZE THE MODEL
VIEW YOU CAN QUANTIZE THE MODEL
VIEW YOU CAN QUANTIZE THE MODEL AND THIS MODEL CAN RUN IN
AND THIS MODEL CAN RUN IN
AND THIS MODEL CAN RUN IN DIFFERENT HARDWARE WITHOUT ANY
DIFFERENT HARDWARE WITHOUT ANY
DIFFERENT HARDWARE WITHOUT ANY CHANGE.
CHANGE.
CHANGE. SO RIGHT NOW IN OUR TOOLKIT WE
SO RIGHT NOW IN OUR TOOLKIT WE
SO RIGHT NOW IN OUR TOOLKIT WE SUPPORT THREE DIFFERENT
SUPPORT THREE DIFFERENT
SUPPORT THREE DIFFERENT QUANTIZATION TYPES.
QUANTIZATION TYPES.
QUANTIZATION TYPES. INCLUDING HERE REDUCED LOAD
INCLUDING HERE REDUCED LOAD
INCLUDING HERE REDUCED LOAD QUANTIZATION TYPE JUST A MUCH
QUANTIZATION TYPE JUST A MUCH
QUANTIZATION TYPE JUST A MUCH SIMPLER THING WHERE WE GO FROM
SIMPLER THING WHERE WE GO FROM
SIMPLER THING WHERE WE GO FROM FLOW 16 PARAMETERS AND
FLOW 16 PARAMETERS AND
FLOW 16 PARAMETERS AND COMPUTATIONS.
COMPUTATIONS.
COMPUTATIONS. PRETTY STRAIGHT FORWARD.
PRETTY STRAIGHT FORWARD.
PRETTY STRAIGHT FORWARD. THE NEXT ONE IS OUR HYBRID
THE NEXT ONE IS OUR HYBRID
THE NEXT ONE IS OUR HYBRID QUANTIZATION WHICH MAKES USE OF
QUANTIZATION WHICH MAKES USE OF
QUANTIZATION WHICH MAKES USE OF AV FOR THE BAROMETERS, BIASES
AV FOR THE BAROMETERS, BIASES
AV FOR THE BAROMETERS, BIASES AND ACTIVATIONS LEAVE US 32-BIT
AND ACTIVATIONS LEAVE US 32-BIT
AND ACTIVATIONS LEAVE US 32-BIT FLOWS AND THEN WE TRY TO BE AS
FLOWS AND THEN WE TRY TO BE AS
FLOWS AND THEN WE TRY TO BE AS SMART AS POSSIBLE FOR HOW WE
SMART AS POSSIBLE FOR HOW WE
SMART AS POSSIBLE FOR HOW WE EXECUTE THESE PROGRAMS.
EXECUTE THESE PROGRAMS.
EXECUTE THESE PROGRAMS. SO THE GOAL BEING THAT FOR
SO THE GOAL BEING THAT FOR
SO THE GOAL BEING THAT FOR EXAMPLE, HEAVY OPERATIONS LIKE
EXAMPLE, HEAVY OPERATIONS LIKE
EXAMPLE, HEAVY OPERATIONS LIKE MULTIPLIERS ARE LEFT IN (?) AND
MULTIPLIERS ARE LEFT IN (?) AND
MULTIPLIERS ARE LEFT IN (?) AND WE USE FLOATING POINTS FOR
WE USE FLOATING POINTS FOR
WE USE FLOATING POINTS FOR THINGS LIKE ACTIVATION
THINGS LIKE ACTIVATION
THINGS LIKE ACTIVATION FUNCTIONS.
FUNCTIONS.
FUNCTIONS. ACCURACY, PERFORMANCE.
ACCURACY, PERFORMANCE.
ACCURACY, PERFORMANCE. THEN THE THIRD ONE IS --
THEN THE THIRD ONE IS --
THEN THE THIRD ONE IS -- QUANTIZATION.
QUANTIZATION.
QUANTIZATION. EVERYTHING IS IN -- ALL THE
EVERYTHING IS IN -- ALL THE
EVERYTHING IS IN -- ALL THE PARAMETERS AND OPERATIONS ARE
PARAMETERS AND OPERATIONS ARE
PARAMETERS AND OPERATIONS ARE (?) THE MORE COMPLICTED ONE.
(?) THE MORE COMPLICTED ONE.
(?) THE MORE COMPLICTED ONE. SO THE BENEFITS OF THE REUSE
SO THE BENEFITS OF THE REUSE
SO THE BENEFITS OF THE REUSE FLOATS YOUR MODELS ARE NOW HALF
FLOATS YOUR MODELS ARE NOW HALF
FLOATS YOUR MODELS ARE NOW HALF SIZE.
SIZE.
SIZE. AND THEN DEPENDING ON THE
AND THEN DEPENDING ON THE
AND THEN DEPENDING ON THE SUPPORT YOU MIGHT GET SOME --
SUPPORT YOU MIGHT GET SOME --
SUPPORT YOU MIGHT GET SOME -- (?) AND LOSS IS MINIMAL.
(?) AND LOSS IS MINIMAL.
(?) AND LOSS IS MINIMAL. PRETTY MUCH ALWAYS WORKS.
PRETTY MUCH ALWAYS WORKS.
PRETTY MUCH ALWAYS WORKS. I HAVEN'T SEEN MYSELF A MODEL
I HAVEN'T SEEN MYSELF A MODEL
I HAVEN'T SEEN MYSELF A MODEL THAT DOESN'T WORK IN FLOW 16.
THAT DOESN'T WORK IN FLOW 16.
THAT DOESN'T WORK IN FLOW 16. IT WAS FLOW 32.
IT WAS FLOW 32.
IT WAS FLOW 32. QUANT -- FOR USER SIZE AND
QUANT -- FOR USER SIZE AND
QUANT -- FOR USER SIZE AND DEPENDING THE USER SIZE AND
DEPENDING THE USER SIZE AND
DEPENDING THE USER SIZE AND OPERATIONS YOU'RE USING YOU MAY
OPERATIONS YOU'RE USING YOU MAY
OPERATIONS YOU'RE USING YOU MAY GET DIFFERENT PERFORMANCE
GET DIFFERENT PERFORMANCE
GET DIFFERENT PERFORMANCE IMPROVEMENTS.
IMPROVEMENTS.
IMPROVEMENTS. TENDS TO BE LARGER FOR FULLY
TENDS TO BE LARGER FOR FULLY
TENDS TO BE LARGER FOR FULLY CONNECTED MODELS OR RNNS AND
CONNECTED MODELS OR RNNS AND
CONNECTED MODELS OR RNNS AND THEN THE THIRD ONE IS THE
THEN THE THIRD ONE IS THE
THEN THE THIRD ONE IS THE INTEGER -- IN TERMS OF MEMORY
INTEGER -- IN TERMS OF MEMORY
INTEGER -- IN TERMS OF MEMORY SIZE BUT IS FASTER TO EXECUTE
SIZE BUT IS FASTER TO EXECUTE
SIZE BUT IS FASTER TO EXECUTE AND HAS A GREATER ADVANTAGE IT
AND HAS A GREATER ADVANTAGE IT
AND HAS A GREATER ADVANTAGE IT HAS MORE HARDWARE COVERAGE.
HAS MORE HARDWARE COVERAGE.
HAS MORE HARDWARE COVERAGE. FOR EXAMPLE.
FOR EXAMPLE.
FOR EXAMPLE. TYPICAL MPUs SOME OF THEM ARE
TYPICAL MPUs SOME OF THEM ARE
TYPICAL MPUs SOME OF THEM ARE INTEGER BASED.
INTEGER BASED.
INTEGER BASED. NOW, LET'S TALK ABOUT THE TOOLS
NOW, LET'S TALK ABOUT THE TOOLS
NOW, LET'S TALK ABOUT THE TOOLS TO QUANTIZE THOSE TYPES.
TO QUANTIZE THOSE TYPES.
TO QUANTIZE THOSE TYPES. WE HAVE THOSE TYPES OF TOOLS.
WE HAVE THOSE TYPES OF TOOLS.
WE HAVE THOSE TYPES OF TOOLS. ONE THAT WORKS POST TRAINING,
ONE THAT WORKS POST TRAINING,
ONE THAT WORKS POST TRAINING, REGULAR TRAIN MODEL AND THE
REGULAR TRAIN MODEL AND THE
REGULAR TRAIN MODEL AND THE OTHER ONE THAT IS WORK IN
OTHER ONE THAT IS WORK IN
OTHER ONE THAT IS WORK IN PROGRESS DURING TRAINING.
PROGRESS DURING TRAINING.
PROGRESS DURING TRAINING. SO LET'S TALK ABOUT THE POST
SO LET'S TALK ABOUT THE POST
SO LET'S TALK ABOUT THE POST TRAINING.
TRAINING.
TRAINING. THE PROCESS IS VERY SIMPLE.
THE PROCESS IS VERY SIMPLE.
THE PROCESS IS VERY SIMPLE. YOU BASICALLY ASSUME THE USERS
YOU BASICALLY ASSUME THE USERS
YOU BASICALLY ASSUME THE USERS TRAIN MODEL.
TRAIN MODEL.
TRAIN MODEL. DOESN'T REALLY CARE HOW YOU
DOESN'T REALLY CARE HOW YOU
DOESN'T REALLY CARE HOW YOU TRAIN IT.
TRAIN IT.
TRAIN IT. JUST -- (?) THEN CURRENTLY WE
JUST -- (?) THEN CURRENTLY WE
JUST -- (?) THEN CURRENTLY WE HAVE TENSORFLOW CONVERTER, YOU
HAVE TENSORFLOW CONVERTER, YOU
HAVE TENSORFLOW CONVERTER, YOU CONVERT IT TO TENSORFLOW LIGHT
CONVERT IT TO TENSORFLOW LIGHT
CONVERT IT TO TENSORFLOW LIGHT AND QUANTIZES IT ON THE FLIGHT
AND QUANTIZES IT ON THE FLIGHT
AND QUANTIZES IT ON THE FLIGHT AND YOU HAVE A MODEL THAT YOU
AND YOU HAVE A MODEL THAT YOU
AND YOU HAVE A MODEL THAT YOU CAN EXECUTE ON WHATEVER
CAN EXECUTE ON WHATEVER
CAN EXECUTE ON WHATEVER HARDWARE IS SUPPORTED IN THE
HARDWARE IS SUPPORTED IN THE
HARDWARE IS SUPPORTED IN THE QUANTIZATION TYPE.
QUANTIZATION TYPE.
QUANTIZATION TYPE. SO NOW LET'S LOOK AT THE
SO NOW LET'S LOOK AT THE
SO NOW LET'S LOOK AT THE SPECIFIC QUANTIZATION TYPES.
SPECIFIC QUANTIZATION TYPES.
SPECIFIC QUANTIZATION TYPES. SO THE FIRST ONE WE HAVE A
SO THE FIRST ONE WE HAVE A
SO THE FIRST ONE WE HAVE A COUPLE OF FLIGHTS.
COUPLE OF FLIGHTS.
COUPLE OF FLIGHTS. USER DEFAULT OPTIMIZATIONS AND
USER DEFAULT OPTIMIZATIONS AND
USER DEFAULT OPTIMIZATIONS AND THESE ARE TAKE CARE OF CHANGING
THESE ARE TAKE CARE OF CHANGING
THESE ARE TAKE CARE OF CHANGING ALL THE PARAMETERS AND THE
ALL THE PARAMETERS AND THE
ALL THE PARAMETERS AND THE COMPUTATION AND AGAIN DEPENDING
COMPUTATION AND AGAIN DEPENDING
COMPUTATION AND AGAIN DEPENDING ON THE HARDWARE YOU'RE RUNNING
ON THE HARDWARE YOU'RE RUNNING
ON THE HARDWARE YOU'RE RUNNING THESE MODELS YOU MIGHT GET A
THESE MODELS YOU MIGHT GET A
THESE MODELS YOU MIGHT GET A SPEED-UP RIGHT NOW.
SPEED-UP RIGHT NOW.
SPEED-UP RIGHT NOW. IF YOU USE SUPPORT FLOW 16.
IF YOU USE SUPPORT FLOW 16.
IF YOU USE SUPPORT FLOW 16. YOU MIGHT GET SOME SPEED-UP
YOU MIGHT GET SOME SPEED-UP
YOU MIGHT GET SOME SPEED-UP THERE.
THERE.
THERE. EITHER BECAUSE OF THE
EITHER BECAUSE OF THE
EITHER BECAUSE OF THE COMPUTATION OR BECAUSE OF THE
COMPUTATION OR BECAUSE OF THE
COMPUTATION OR BECAUSE OF THE BANDWIDTH IN YOUR TUBE WILL BE
BANDWIDTH IN YOUR TUBE WILL BE
BANDWIDTH IN YOUR TUBE WILL BE REDUCED.
REDUCED.
REDUCED. LIKE I SAID BENEFITS THE SIZE
LIKE I SAID BENEFITS THE SIZE
LIKE I SAID BENEFITS THE SIZE GOES TO HALF.
GOES TO HALF.
GOES TO HALF. AND YOU KNOW, THE DROP IS VERY
AND YOU KNOW, THE DROP IS VERY
AND YOU KNOW, THE DROP IS VERY MINIMAL.
MINIMAL.
MINIMAL. I WILL SAY WITHIN THE NOISE.
I WILL SAY WITHIN THE NOISE.
I WILL SAY WITHIN THE NOISE. THEN THE NEXT ONE IS -- THIS IS
THEN THE NEXT ONE IS -- THIS IS
THEN THE NEXT ONE IS -- THIS IS VERY EASY.
VERY EASY.
VERY EASY. THIS IS THE DEFAULT FOR
THIS IS THE DEFAULT FOR
THIS IS THE DEFAULT FOR TENSORFLOW LIGHT DEFAULT AND
TENSORFLOW LIGHT DEFAULT AND
TENSORFLOW LIGHT DEFAULT AND THEN IT WILL MAKE SURE THE
THEN IT WILL MAKE SURE THE
THEN IT WILL MAKE SURE THE QUANTIZE PARAMETERS AND THEN
QUANTIZE PARAMETERS AND THEN
QUANTIZE PARAMETERS AND THEN OPERATIONS THAT DOESN'T HAVE
OPERATIONS THAT DOESN'T HAVE
OPERATIONS THAT DOESN'T HAVE DEFINED SPECIFICATIONS FOR
DEFINED SPECIFICATIONS FOR
DEFINED SPECIFICATIONS FOR QUANTIZE FORM WILL BE KEPT IN
QUANTIZE FORM WILL BE KEPT IN
QUANTIZE FORM WILL BE KEPT IN THE PRECISION AND YOU WILL GET
THE PRECISION AND YOU WILL GET
THE PRECISION AND YOU WILL GET SOME SPEED-OUTS AND BE ABLE TO
SOME SPEED-OUTS AND BE ABLE TO
SOME SPEED-OUTS AND BE ABLE TO EXECUTE IN WHATEVER HARDWARE
EXECUTE IN WHATEVER HARDWARE
EXECUTE IN WHATEVER HARDWARE COMPLIES WITH THE SPECIFICATION.
COMPLIES WITH THE SPECIFICATION.
COMPLIES WITH THE SPECIFICATION. SO TYPICALLY THIS ONE WORKS
SO TYPICALLY THIS ONE WORKS
SO TYPICALLY THIS ONE WORKS WELL FOR CPUs.
WELL FOR CPUs.
WELL FOR CPUs. AND AGAIN BENEFITS FOR ITS
AND AGAIN BENEFITS FOR ITS
AND AGAIN BENEFITS FOR ITS COMPRESSION FOR THE MODELS, AND
COMPRESSION FOR THE MODELS, AND
COMPRESSION FOR THE MODELS, AND THEN YOU GET SOME SPEED-OUTS,
THEN YOU GET SOME SPEED-OUTS,
THEN YOU GET SOME SPEED-OUTS, CONVOLUTION BASED MODELS.
CONVOLUTION BASED MODELS.
CONVOLUTION BASED MODELS. I WILL SAY THESE ARE ONE YEAR
I WILL SAY THESE ARE ONE YEAR
I WILL SAY THESE ARE ONE YEAR OLD NUMBERS SO PROBABLY RIGHT
OLD NUMBERS SO PROBABLY RIGHT
OLD NUMBERS SO PROBABLY RIGHT NOW IT IS FASTER.
NOW IT IS FASTER.
NOW IT IS FASTER. AND THE SAME FOR ACCURACY.
AND THE SAME FOR ACCURACY.
AND THE SAME FOR ACCURACY. THAT'S PRETTY GOOD.
THAT'S PRETTY GOOD.
THAT'S PRETTY GOOD. THAT'S WE'RE WORKING ON SOME
THAT'S WE'RE WORKING ON SOME
THAT'S WE'RE WORKING ON SOME CHANGES FOR CONVOLUTION MODELS
CHANGES FOR CONVOLUTION MODELS
CHANGES FOR CONVOLUTION MODELS IT WILL BE MORE ACCURATE.
IT WILL BE MORE ACCURATE.
IT WILL BE MORE ACCURATE. THEN THE THIRD ONE IS THE
THEN THE THIRD ONE IS THE
THEN THE THIRD ONE IS THE INTEGER QUANTIZATION.
INTEGER QUANTIZATION.
INTEGER QUANTIZATION. THIS ONE IS MORE COMPLEX.
THIS ONE IS MORE COMPLEX.
THIS ONE IS MORE COMPLEX. YOU NEED TO PROVIDE DATA.
YOU NEED TO PROVIDE DATA.
YOU NEED TO PROVIDE DATA. I WANT TO HAVE A MODEL BUT I
I WANT TO HAVE A MODEL BUT I
I WANT TO HAVE A MODEL BUT I WANT TO USE QUANTIZATION OR NOW
WANT TO USE QUANTIZATION OR NOW
WANT TO USE QUANTIZATION OR NOW YOU NEED TO PROVIDE SOME DATA
YOU NEED TO PROVIDE SOME DATA
YOU NEED TO PROVIDE SOME DATA AND BY DATA I MEAN LABEL
AND BY DATA I MEAN LABEL
AND BY DATA I MEAN LABEL SAMPLES OF WHAT YOUR NEURAL
SAMPLES OF WHAT YOUR NEURAL
SAMPLES OF WHAT YOUR NEURAL NETWORK WILL TYPICALLY SEE IN
NETWORK WILL TYPICALLY SEE IN
NETWORK WILL TYPICALLY SEE IN REALITY.
REALITY.
REALITY. IN THE PROCESSING MODEL YOU
IN THE PROCESSING MODEL YOU
IN THE PROCESSING MODEL YOU NEED TO SEE THE SOME
NEED TO SEE THE SOME
NEED TO SEE THE SOME PRE-PROCESSING MEASURES.
PRE-PROCESSING MEASURES.
PRE-PROCESSING MEASURES. WE AREN'T TALKING ABOUT A LOT
WE AREN'T TALKING ABOUT A LOT
WE AREN'T TALKING ABOUT A LOT OF DATA FOR THE RESULT THAT
OF DATA FOR THE RESULT THAT
OF DATA FOR THE RESULT THAT I'LL SHOW NEXT.
I'LL SHOW NEXT.
I'LL SHOW NEXT. WE ARE TALKING ABOUT 100
WE ARE TALKING ABOUT 100
WE ARE TALKING ABOUT 100 SAMPLES.
SAMPLES.
SAMPLES. THAT WORKS REALLY WELL.
THAT WORKS REALLY WELL.
THAT WORKS REALLY WELL. IT IS A LITTLE MORE COMPLICATED
IT IS A LITTLE MORE COMPLICATED
IT IS A LITTLE MORE COMPLICATED BUT NOT VERY COMPLICATED.
BUT NOT VERY COMPLICATED.
BUT NOT VERY COMPLICATED. SO THESE ARE SOME RESULTS FROM
SO THESE ARE SOME RESULTS FROM
SO THESE ARE SOME RESULTS FROM DECISION ACROSS DIFFERENT MODES.
DECISION ACROSS DIFFERENT MODES.
DECISION ACROSS DIFFERENT MODES. FOR THE MAJORITY OF MODELS THE
FOR THE MAJORITY OF MODELS THE
FOR THE MAJORITY OF MODELS THE LOSS IS NOT THAT BIG WITH
LOSS IS NOT THAT BIG WITH
LOSS IS NOT THAT BIG WITH RESPECT TO THE FULL PRECISION
RESPECT TO THE FULL PRECISION
RESPECT TO THE FULL PRECISION TRAINING BASELINE.
TRAINING BASELINE.
TRAINING BASELINE. THE ONLY ONE I WILL SAY IS THIS
THE ONLY ONE I WILL SAY IS THIS
THE ONLY ONE I WILL SAY IS THIS MODEL.
MODEL.
MODEL. THAT IS A BIT MORE MEANINGFUL
THAT IS A BIT MORE MEANINGFUL
THAT IS A BIT MORE MEANINGFUL DROP.
DROP.
DROP. BUT AGAIN OF OUR MODELS, WITH
BUT AGAIN OF OUR MODELS, WITH
BUT AGAIN OF OUR MODELS, WITH POST TRAINING QUANTIZATION.
POST TRAINING QUANTIZATION.
POST TRAINING QUANTIZATION. NOW TALK ABOUT -- (?) I AM
NOW TALK ABOUT -- (?) I AM
NOW TALK ABOUT -- (?) I AM SHOWING THE PREVIOUS RESULTS.
SHOWING THE PREVIOUS RESULTS.
SHOWING THE PREVIOUS RESULTS. THE MODELS -- FROM DOING THIS
THE MODELS -- FROM DOING THIS
THE MODELS -- FROM DOING THIS TRAINING QUANTIZATION TRAINING.
TRAINING QUANTIZATION TRAINING.
TRAINING QUANTIZATION TRAINING. BY THAT WE MEAN WE TRY TO
BY THAT WE MEAN WE TRY TO
BY THAT WE MEAN WE TRY TO EMULATE THE QUANTIZATION
EMULATE THE QUANTIZATION
EMULATE THE QUANTIZATION OPERATIONS, THE QUANTIZATION
OPERATIONS, THE QUANTIZATION
OPERATIONS, THE QUANTIZATION LOSSES DURING THE FORWARD PASS
LOSSES DURING THE FORWARD PASS
LOSSES DURING THE FORWARD PASS OF THE NEURAL NETWORK AND WE
OF THE NEURAL NETWORK AND WE
OF THE NEURAL NETWORK AND WE HOPE THE PARAMETERS WILL BE
HOPE THE PARAMETERS WILL BE
HOPE THE PARAMETERS WILL BE ABLE TO ACCOUNT FOR THAT.
ABLE TO ACCOUNT FOR THAT.
ABLE TO ACCOUNT FOR THAT. THE PROCESS FOR DOING THE
THE PROCESS FOR DOING THE
THE PROCESS FOR DOING THE QUANTIZATION WORK TRAINING FOR
QUANTIZATION WORK TRAINING FOR
QUANTIZATION WORK TRAINING FOR USING OUR API IS A LITTLE MORE
USING OUR API IS A LITTLE MORE
USING OUR API IS A LITTLE MORE INVOLVED BUT WE ARE TRYING TO
INVOLVED BUT WE ARE TRYING TO
INVOLVED BUT WE ARE TRYING TO MAKE IT VERY SIMPLE.
MAKE IT VERY SIMPLE.
MAKE IT VERY SIMPLE. SO WE BUILD THIS API IN KAIROS
SO WE BUILD THIS API IN KAIROS
SO WE BUILD THIS API IN KAIROS TO MAKE IT VERY EASY TO USE.
TO MAKE IT VERY EASY TO USE.
TO MAKE IT VERY EASY TO USE. WE ASSUME YOU ALREADY HAVE A
WE ASSUME YOU ALREADY HAVE A
WE ASSUME YOU ALREADY HAVE A KAIROS MODEL AND THEN YOU NEED
KAIROS MODEL AND THEN YOU NEED
KAIROS MODEL AND THEN YOU NEED TO CALL API TO APPLY THE
TO CALL API TO APPLY THE
TO CALL API TO APPLY THE QUANTIZATION.
QUANTIZATION.
QUANTIZATION. AND THIS MIGHT CHANGE A LITTLE
AND THIS MIGHT CHANGE A LITTLE
AND THIS MIGHT CHANGE A LITTLE BIT BUT IT WILL LOOK SOMETHING
BIT BUT IT WILL LOOK SOMETHING
BIT BUT IT WILL LOOK SOMETHING LIKE THIS.
LIKE THIS.
LIKE THIS. SO YOU JUST HAVE A MODEL THAT
SO YOU JUST HAVE A MODEL THAT
SO YOU JUST HAVE A MODEL THAT YOU ALREADY BUILD IN KAIROS AND
YOU ALREADY BUILD IN KAIROS AND
YOU ALREADY BUILD IN KAIROS AND THE ONLY THING YOU NEED TO DO
THE ONLY THING YOU NEED TO DO
THE ONLY THING YOU NEED TO DO IS GO API ON YOUR MODEL AND YOU
IS GO API ON YOUR MODEL AND YOU
IS GO API ON YOUR MODEL AND YOU GET NOW A MODEL THAT IS WRITTEN
GET NOW A MODEL THAT IS WRITTEN
GET NOW A MODEL THAT IS WRITTEN TO HAVE ALL THE QUANTIZATION
TO HAVE ALL THE QUANTIZATION
TO HAVE ALL THE QUANTIZATION AND THEN YOU JUST CALL YOUR FIT
AND THEN YOU JUST CALL YOUR FIT
AND THEN YOU JUST CALL YOUR FIT FUNCTION AND THAT'S IT.
FUNCTION AND THAT'S IT.
FUNCTION AND THAT'S IT. SO THEN YOU JUST TRAIN YOUR
SO THEN YOU JUST TRAIN YOUR
SO THEN YOU JUST TRAIN YOUR MODEL AS USUAL.
MODEL AS USUAL.
MODEL AS USUAL. AND THEN YOU CAN GO THROUGH THE
AND THEN YOU CAN GO THROUGH THE
AND THEN YOU CAN GO THROUGH THE CONVERTER AND IT WILL TAKE THIS
CONVERTER AND IT WILL TAKE THIS
CONVERTER AND IT WILL TAKE THIS MODEL AND QUANTIZATION AND HAVE
MODEL AND QUANTIZATION AND HAVE
MODEL AND QUANTIZATION AND HAVE THE MATERIAL TO QUANTIZE IT AND
THE MATERIAL TO QUANTIZE IT AND
THE MATERIAL TO QUANTIZE IT AND PRODUCE A MODEL JUST LIKE THE
PRODUCE A MODEL JUST LIKE THE
PRODUCE A MODEL JUST LIKE THE POST TRAIN MODEL YOU WILL BE
POST TRAIN MODEL YOU WILL BE
POST TRAIN MODEL YOU WILL BE ABLE TO EXECUTE IN DIFFERENT
ABLE TO EXECUTE IN DIFFERENT
ABLE TO EXECUTE IN DIFFERENT HARDWARE.
HARDWARE.
HARDWARE. THESE ARE SOME NUMBERS FROM
THESE ARE SOME NUMBERS FROM
THESE ARE SOME NUMBERS FROM QUANTIZATION TRAINING,
QUANTIZATION TRAINING,
QUANTIZATION TRAINING, PRELIMINARY NUMBERS.
PRELIMINARY NUMBERS.
PRELIMINARY NUMBERS. IF YOU SEE THE DELTA IS A
IF YOU SEE THE DELTA IS A
IF YOU SEE THE DELTA IS A LITTLE BIT BETTER THAN POST
LITTLE BIT BETTER THAN POST
LITTLE BIT BETTER THAN POST TRAINING QUANTIZATION THERE IS
TRAINING QUANTIZATION THERE IS
TRAINING QUANTIZATION THERE IS NOT A BIG DIFFERENCE.
NOT A BIG DIFFERENCE.
NOT A BIG DIFFERENCE. IT WAS 4% BEFORE, IN THIS CASE
IT WAS 4% BEFORE, IN THIS CASE
IT WAS 4% BEFORE, IN THIS CASE 2.9%.
2.9%.
2.9%. QUANTIZATION TRAINING IS THE
QUANTIZATION TRAINING IS THE
QUANTIZATION TRAINING IS THE USUAL TOOL AND WHY WE'RE
USUAL TOOL AND WHY WE'RE
USUAL TOOL AND WHY WE'RE BUILDING IT.
BUILDING IT.
BUILDING IT. NOW YOU MAY WONDER THOSE WERE A
NOW YOU MAY WONDER THOSE WERE A
NOW YOU MAY WONDER THOSE WERE A LOT OF QUANTIZATION TYPES AND
LOT OF QUANTIZATION TYPES AND
LOT OF QUANTIZATION TYPES AND PROOFS.
PROOFS.
PROOFS. WHICH ONE SHOULD I USE?
WHICH ONE SHOULD I USE?
WHICH ONE SHOULD I USE? MY RECOMMENDATION IS IF YOU ARE
MY RECOMMENDATION IS IF YOU ARE
MY RECOMMENDATION IS IF YOU ARE JUST STARTING START REDUCE
JUST STARTING START REDUCE
JUST STARTING START REDUCE FLOAT THE FIRST ONE TO TRY.
FLOAT THE FIRST ONE TO TRY.
FLOAT THE FIRST ONE TO TRY. VERY EASY TO USE.
VERY EASY TO USE.
VERY EASY TO USE. DOESN'T REQUIRE ANY DATA.
DOESN'T REQUIRE ANY DATA.
DOESN'T REQUIRE ANY DATA. IT WILL PROBABLY BE THE SAME
IT WILL PROBABLY BE THE SAME
IT WILL PROBABLY BE THE SAME AND LATENCY DEPENDING ON THE
AND LATENCY DEPENDING ON THE
AND LATENCY DEPENDING ON THE HARDWARE YOU MIGHT GET SOME
HARDWARE YOU MIGHT GET SOME
HARDWARE YOU MIGHT GET SOME BENEFITS.
BENEFITS.
BENEFITS. REDUCED LATENCY AND THEN
REDUCED LATENCY AND THEN
REDUCED LATENCY AND THEN COMPATIBLE EVERYWHERE YOU CAN
COMPATIBLE EVERYWHERE YOU CAN
COMPATIBLE EVERYWHERE YOU CAN EXECUTE FLOATING POINT
EXECUTE FLOATING POINT
EXECUTE FLOATING POINT OPERATIONS YOU WILL BE ABLE TO
OPERATIONS YOU WILL BE ABLE TO
OPERATIONS YOU WILL BE ABLE TO USE IT.
USE IT.
USE IT. THE NEXT THING TO TRY WOULD BE
THE NEXT THING TO TRY WOULD BE
THE NEXT THING TO TRY WOULD BE THE HYBRID QUANTIZATION.
THE HYBRID QUANTIZATION.
THE HYBRID QUANTIZATION. AGAIN THERE IS NO DATA
AGAIN THERE IS NO DATA
AGAIN THERE IS NO DATA REQUIREMENTS.
REQUIREMENTS.
REQUIREMENTS. THE -- FLOW 16 IN SOME CASES
THE -- FLOW 16 IN SOME CASES
THE -- FLOW 16 IN SOME CASES BUT IT IS STILL GOOD.
BUT IT IS STILL GOOD.
BUT IT IS STILL GOOD. IT WILL BE FASTER THAN THE
IT WILL BE FASTER THAN THE
IT WILL BE FASTER THAN THE REDUCED FLOAT AND BASICALLY
REDUCED FLOAT AND BASICALLY
REDUCED FLOAT AND BASICALLY COMPATIBLE EVERYWHEE THAT YOU
COMPATIBLE EVERYWHEE THAT YOU
COMPATIBLE EVERYWHEE THAT YOU HAVE SUPPORT FOR FLOWS AND
HAVE SUPPORT FOR FLOWS AND
HAVE SUPPORT FOR FLOWS AND INTEGER OPERATIONS.
INTEGER OPERATIONS.
INTEGER OPERATIONS. AND THE THIRD ONE TO TRY IS THE
AND THE THIRD ONE TO TRY IS THE
AND THE THIRD ONE TO TRY IS THE INTEGER QUANTIZATION WITH POST
INTEGER QUANTIZATION WITH POST
INTEGER QUANTIZATION WITH POST TRAINING TOOL.
TRAINING TOOL.
TRAINING TOOL. SO THIS ONE IS A BIT MORE
SO THIS ONE IS A BIT MORE
SO THIS ONE IS A BIT MORE COMPLICATED BECAUSE YOU NEED TO
COMPLICATED BECAUSE YOU NEED TO
COMPLICATED BECAUSE YOU NEED TO PROVIDE A LITTLE BIT OF DATA.
PROVIDE A LITTLE BIT OF DATA.
PROVIDE A LITTLE BIT OF DATA. THE -- WILL BE WORKS THROUGH
THE -- WILL BE WORKS THROUGH
THE -- WILL BE WORKS THROUGH THE SAME AS HYBRID BUT THE
THE SAME AS HYBRID BUT THE
THE SAME AS HYBRID BUT THE LATENCY WILL BE THE FASTEST.
LATENCY WILL BE THE FASTEST.
LATENCY WILL BE THE FASTEST. IT WILL ALSO GIVE YOU MORE
IT WILL ALSO GIVE YOU MORE
IT WILL ALSO GIVE YOU MORE HARDWARE COVERAGE.
HARDWARE COVERAGE.
HARDWARE COVERAGE. AND THEN THE LAST THING TO TRY
AND THEN THE LAST THING TO TRY
AND THEN THE LAST THING TO TRY WILL BE THE INTEGER
WILL BE THE INTEGER
WILL BE THE INTEGER QUANTIZATION WITH QUANTIZATION
QUANTIZATION WITH QUANTIZATION
QUANTIZATION WITH QUANTIZATION TRAINING AND BASICALLY THIS IS
TRAINING AND BASICALLY THIS IS
TRAINING AND BASICALLY THIS IS --
--
-- THIS WILL BE THE -- A LITTLE
THIS WILL BE THE -- A LITTLE
THIS WILL BE THE -- A LITTLE BIT MORE INVOLVED BECAUSE NOW
BIT MORE INVOLVED BECAUSE NOW
BIT MORE INVOLVED BECAUSE NOW YOU ARE DOING TRAINING AND
YOU ARE DOING TRAINING AND
YOU ARE DOING TRAINING AND SUPPOSED TO HAVE A TRAINING
SUPPOSED TO HAVE A TRAINING
SUPPOSED TO HAVE A TRAINING SETUP.
SETUP.
SETUP. IT WILL BE BETTER THAN DOING
IT WILL BE BETTER THAN DOING
IT WILL BE BETTER THAN DOING THE POST TRAINING VERSION AND
THE POST TRAINING VERSION AND
THE POST TRAINING VERSION AND AGAIN YOU GET THE BENEFITS OF
AGAIN YOU GET THE BENEFITS OF
AGAIN YOU GET THE BENEFITS OF BEING THE FASTEST ONE AND THE
BEING THE FASTEST ONE AND THE
BEING THE FASTEST ONE AND THE ONE WITH MORE HARDWARE COVERAGE.
ONE WITH MORE HARDWARE COVERAGE.
ONE WITH MORE HARDWARE COVERAGE. SO THAT WAS QUANTIZATION.
SO THAT WAS QUANTIZATION.
SO THAT WAS QUANTIZATION. AGAIN, ALL THESE TOOLS WE'RE
AGAIN, ALL THESE TOOLS WE'RE
AGAIN, ALL THESE TOOLS WE'RE TRYING TO MAKE VERY EASY TO USE.
TRYING TO MAKE VERY EASY TO USE.
TRYING TO MAKE VERY EASY TO USE. IT WILL BE GREAT IF YOU TRY
IT WILL BE GREAT IF YOU TRY
IT WILL BE GREAT IF YOU TRY THEM OUT AND GIVE US SOME
THEM OUT AND GIVE US SOME
THEM OUT AND GIVE US SOME FEEDBACK.
FEEDBACK.
FEEDBACK. CONNECTION PRUNING.
CONNECTION PRUNING.
CONNECTION PRUNING. SO WHAT IS NEURAL CONNECTION
SO WHAT IS NEURAL CONNECTION
SO WHAT IS NEURAL CONNECTION PRUNING?
PRUNING?
PRUNING? THE WAY WE HAVE IMPLEMENTED IT
THE WAY WE HAVE IMPLEMENTED IT
THE WAY WE HAVE IMPLEMENTED IT SO FAR IT MEANS THE TRAINING
SO FAR IT MEANS THE TRAINING
SO FAR IT MEANS THE TRAINING TEHNIQUE THAT DURING THE
TEHNIQUE THAT DURING THE
TEHNIQUE THAT DURING THE TRAINING PROCESS YOU WILL START
TRAINING PROCESS YOU WILL START
TRAINING PROCESS YOU WILL START REMOVING DROPPING CONNECTIONS
REMOVING DROPPING CONNECTIONS
REMOVING DROPPING CONNECTIONS FROM THE NEURAL NETWORK.
FROM THE NEURAL NETWORK.
FROM THE NEURAL NETWORK. AND THEN THESE CONNECTIONS WILL
AND THEN THESE CONNECTIONS WILL
AND THEN THESE CONNECTIONS WILL -- THE DROP CONNECTIONS IN THE
-- THE DROP CONNECTIONS IN THE
-- THE DROP CONNECTIONS IN THE TENSORS OF YOUR TRAINING AND
TENSORS OF YOUR TRAINING AND
TENSORS OF YOUR TRAINING AND YOU END UP WITH SPARSE TENSORS.
YOU END UP WITH SPARSE TENSORS.
YOU END UP WITH SPARSE TENSORS. THAT'S GREAT BECAUSE YOU CAN
THAT'S GREAT BECAUSE YOU CAN
THAT'S GREAT BECAUSE YOU CAN COMPRESS THEM AND POTENTIALLY
COMPRESS THEM AND POTENTIALLY
COMPRESS THEM AND POTENTIALLY EXECUTE THEM FASTER.
EXECUTE THEM FASTER.
EXECUTE THEM FASTER. SO THESE ARE AN EXAMPLE.
SO THESE ARE AN EXAMPLE.
SO THESE ARE AN EXAMPLE. THIS IS SAY A TENSOR HOW WE
THIS IS SAY A TENSOR HOW WE
THIS IS SAY A TENSOR HOW WE START, RANDOM INITIALIZED.
START, RANDOM INITIALIZED.
START, RANDOM INITIALIZED. THE DARK VALUES MEAN VALUES ARE
THE DARK VALUES MEAN VALUES ARE
THE DARK VALUES MEAN VALUES ARE NONE TO 0 AND WHITE MEANS THAT
NONE TO 0 AND WHITE MEANS THAT
NONE TO 0 AND WHITE MEANS THAT VALUES ARE HERE.
VALUES ARE HERE.
VALUES ARE HERE. AS THE TRAINING PROGRESSES YOU
AS THE TRAINING PROGRESSES YOU
AS THE TRAINING PROGRESSES YOU START BECOMING SPARSEER AND
START BECOMING SPARSEER AND
START BECOMING SPARSEER AND SPARSEER.
SPARSEER.
SPARSEER. IF YOU SEE THE TENSORS WE'RE
IF YOU SEE THE TENSORS WE'RE
IF YOU SEE THE TENSORS WE'RE REMOVING MOST OF THE PARAMETERS
REMOVING MOST OF THE PARAMETERS
REMOVING MOST OF THE PARAMETERS THERE.
THERE.
THERE. THE PRESENT API IS SIMILAR TO
THE PRESENT API IS SIMILAR TO
THE PRESENT API IS SIMILAR TO THE WORD TRAINING API TRYING TO
THE WORD TRAINING API TRYING TO
THE WORD TRAINING API TRYING TO BRING CONSISTENCY TO OUR API.
BRING CONSISTENCY TO OUR API.
BRING CONSISTENCY TO OUR API. YOU HAVE A MODEL THAT IS
YOU HAVE A MODEL THAT IS
YOU HAVE A MODEL THAT IS TRAINABLE IN KAIROS AND YOU ARE
TRAINABLE IN KAIROS AND YOU ARE
TRAINABLE IN KAIROS AND YOU ARE GOING TO CALL API TO APPLY THE
GOING TO CALL API TO APPLY THE
GOING TO CALL API TO APPLY THE PRUNING.
PRUNING.
PRUNING. THESE AGAIN WE'RE TRYING TO
THESE AGAIN WE'RE TRYING TO
THESE AGAIN WE'RE TRYING TO MAKE THIS AS SIMPLE AS POSSIBLE.
MAKE THIS AS SIMPLE AS POSSIBLE.
MAKE THIS AS SIMPLE AS POSSIBLE. THE ONLY THING YOU NEED TO
THE ONLY THING YOU NEED TO
THE ONLY THING YOU NEED TO DEFINE IS THE PRUNING SCHEDULE.
DEFINE IS THE PRUNING SCHEDULE.
DEFINE IS THE PRUNING SCHEDULE. BASICALLY WHEN YOU WANT TO
BASICALLY WHEN YOU WANT TO
BASICALLY WHEN YOU WANT TO START DROPPING THESE
START DROPPING THESE
START DROPPING THESE CONNECTIONS UNTIL WHEN AND HOW
CONNECTIONS UNTIL WHEN AND HOW
CONNECTIONS UNTIL WHEN AND HOW FAST AND HOW AGGRESSIVE YOU
FAST AND HOW AGGRESSIVE YOU
FAST AND HOW AGGRESSIVE YOU WANT THE PRUNING TO BE.
WANT THE PRUNING TO BE.
WANT THE PRUNING TO BE. AND THEN YOU JUST GO OVER PRUNE
AND THEN YOU JUST GO OVER PRUNE
AND THEN YOU JUST GO OVER PRUNE FUNCTION WHICH AGAIN WILL
FUNCTION WHICH AGAIN WILL
FUNCTION WHICH AGAIN WILL MODIFY YOUR GRAPH TO ADD ALL
MODIFY YOUR GRAPH TO ADD ALL
MODIFY YOUR GRAPH TO ADD ALL THE PRUNING OPERATIONS
THE PRUNING OPERATIONS
THE PRUNING OPERATIONS INTERNALLY AND YOU JUST CALL
INTERNALLY AND YOU JUST CALL
INTERNALLY AND YOU JUST CALL YOUR FIT FUNCTION AND TRAINING
YOUR FIT FUNCTION AND TRAINING
YOUR FIT FUNCTION AND TRAINING AS USUAL.
AS USUAL.
AS USUAL. SO BASICALLY YOU TRAIN AS USUAL
SO BASICALLY YOU TRAIN AS USUAL
SO BASICALLY YOU TRAIN AS USUAL AND THEN ONCE YOU TRAIN YOU
AND THEN ONCE YOU TRAIN YOU
AND THEN ONCE YOU TRAIN YOU HAVE TWO OPTIONS NOW.
HAVE TWO OPTIONS NOW.
HAVE TWO OPTIONS NOW. SOON YOU WILL HAVE TWO OPTIONS.
SOON YOU WILL HAVE TWO OPTIONS.
SOON YOU WILL HAVE TWO OPTIONS. YOU CAN JUST TAKE THIS SAVE
YOU CAN JUST TAKE THIS SAVE
YOU CAN JUST TAKE THIS SAVE MODEL AND JUST COMPRESS IT AND
MODEL AND JUST COMPRESS IT AND
MODEL AND JUST COMPRESS IT AND THE MODEL WILL BE SMALLER.
THE MODEL WILL BE SMALLER.
THE MODEL WILL BE SMALLER. AND SOON YOU WILL BE ABLE TO
AND SOON YOU WILL BE ABLE TO
AND SOON YOU WILL BE ABLE TO CONVERT IT TO TENSORFLOW LIGHT
CONVERT IT TO TENSORFLOW LIGHT
CONVERT IT TO TENSORFLOW LIGHT AND YOU WILL GET ALSO REDUCTION
AND YOU WILL GET ALSO REDUCTION
AND YOU WILL GET ALSO REDUCTION IN SIZE AND POTENTIALLY --
IN SIZE AND POTENTIALLY --
IN SIZE AND POTENTIALLY -- DEPENDING ON WHAT CONFIGURATION
DEPENDING ON WHAT CONFIGURATION
DEPENDING ON WHAT CONFIGURATION YOU'RE USING AND THE HARDWARE
YOU'RE USING AND THE HARDWARE
YOU'RE USING AND THE HARDWARE YOU'RE TARGETING.
YOU'RE TARGETING.
YOU'RE TARGETING. SO THIS SHOULD BE DONE PRETTY
SO THIS SHOULD BE DONE PRETTY
SO THIS SHOULD BE DONE PRETTY SOON.
SOON.
SOON. NOW WHAT ARE THE BENEFITS OF
NOW WHAT ARE THE BENEFITS OF
NOW WHAT ARE THE BENEFITS OF PRUNING?
PRUNING?
PRUNING? WE'VE TRIED IT IN A LOT OF
WE'VE TRIED IT IN A LOT OF
WE'VE TRIED IT IN A LOT OF TASKS, REALLY A LOT OF TASKS
TASKS, REALLY A LOT OF TASKS
TASKS, REALLY A LOT OF TASKS AND IT WORKS PRETTY WELL.
AND IT WORKS PRETTY WELL.
AND IT WORKS PRETTY WELL. A LOT OF THINGS ARE HYPER
A LOT OF THINGS ARE HYPER
A LOT OF THINGS ARE HYPER PARAMETER TUNING AND CAREFUL
PARAMETER TUNING AND CAREFUL
PARAMETER TUNING AND CAREFUL FOR STARTING YOUR MODELS AND
FOR STARTING YOUR MODELS AND
FOR STARTING YOUR MODELS AND THINGS LIKE THAT.
THINGS LIKE THAT.
THINGS LIKE THAT. IT HAS WORKED WELL WITHOUT A
IT HAS WORKED WELL WITHOUT A
IT HAS WORKED WELL WITHOUT A LOT OF BABYSITTING.
LOT OF BABYSITTING.
LOT OF BABYSITTING. IT HAS POTENTIAL FOR --
IT HAS POTENTIAL FOR --
IT HAS POTENTIAL FOR -- DEPENDING ON HARDWARE SUPPORT
DEPENDING ON HARDWARE SUPPORT
DEPENDING ON HARDWARE SUPPORT AND WE ALSO HAVE PRETTY GOOD
AND WE ALSO HAVE PRETTY GOOD
AND WE ALSO HAVE PRETTY GOOD RESULTS.
RESULTS.
RESULTS. WE CAN MAKE A LOT OF THE
WE CAN MAKE A LOT OF THE
WE CAN MAKE A LOT OF THE PARAMETERS BASICALLY GO AWAY.
PARAMETERS BASICALLY GO AWAY.
PARAMETERS BASICALLY GO AWAY. WE SEE 50% WITH NEGLIGIBLE
WE SEE 50% WITH NEGLIGIBLE
WE SEE 50% WITH NEGLIGIBLE ACCURACY LOSS AND THE OTHER
ACCURACY LOSS AND THE OTHER
ACCURACY LOSS AND THE OTHER GREAT THING IT WORKS WELL WITH
GREAT THING IT WORKS WELL WITH
GREAT THING IT WORKS WELL WITH QUANTIZATION.
QUANTIZATION.
QUANTIZATION. A TYPICAL SETUP WE TRY.
A TYPICAL SETUP WE TRY.
A TYPICAL SETUP WE TRY. TRAINING PRUNING AND POST
TRAINING PRUNING AND POST
TRAINING PRUNING AND POST TRAINING QUANTIZATION AND YOU
TRAINING QUANTIZATION AND YOU
TRAINING QUANTIZATION AND YOU GET THE COMPOUND BENEFIT OF
GET THE COMPOUND BENEFIT OF
GET THE COMPOUND BENEFIT OF BOTH TECHNIQUES.
BOTH TECHNIQUES.
BOTH TECHNIQUES. THESE ARE SOME -- WE SHOW WE
THESE ARE SOME -- WE SHOW WE
THESE ARE SOME -- WE SHOW WE HAD WHEN WE LAUNCHED THIS.
HAD WHEN WE LAUNCHED THIS.
HAD WHEN WE LAUNCHED THIS. YOU SEE WE CAN GET ALL THE WAY
YOU SEE WE CAN GET ALL THE WAY
YOU SEE WE CAN GET ALL THE WAY ALMOST TO 90% SPARSETY WITH
ALMOST TO 90% SPARSETY WITH
ALMOST TO 90% SPARSETY WITH RELATIVELY SMALL ACCURACY
RELATIVELY SMALL ACCURACY
RELATIVELY SMALL ACCURACY LOSSES.
LOSSES.
LOSSES. WE CAN TAKE IT TO ALMOST 90%
WE CAN TAKE IT TO ALMOST 90%
WE CAN TAKE IT TO ALMOST 90% PRUNING AND ALSO SMALLER
PRUNING AND ALSO SMALLER
PRUNING AND ALSO SMALLER ACCURACY LOSSES.
ACCURACY LOSSES.
ACCURACY LOSSES. WE HAVE DONE THESE FOR -- (?)
WE HAVE DONE THESE FOR -- (?)
WE HAVE DONE THESE FOR -- (?) WE HAD RECENTLY THE GOOGLE
WE HAD RECENTLY THE GOOGLE
WE HAD RECENTLY THE GOOGLE PIXEL EVENT WHERE THE MODELS
PIXEL EVENT WHERE THE MODELS
PIXEL EVENT WHERE THE MODELS ARE USED PRUNING AND
ARE USED PRUNING AND
ARE USED PRUNING AND QUANTIZATION AND WE WERE ABLE
QUANTIZATION AND WE WERE ABLE
QUANTIZATION AND WE WERE ABLE TO HAVE A MODEL WITH SERVERS
TO HAVE A MODEL WITH SERVERS
TO HAVE A MODEL WITH SERVERS HIGH QUALITY RUNNING ON OUR
HIGH QUALITY RUNNING ON OUR
HIGH QUALITY RUNNING ON OUR PHONE.
PHONE.
PHONE. WHICH IS REALLY GOOD.
WHICH IS REALLY GOOD.
WHICH IS REALLY GOOD. SO NOW FINALLY REAL QUICK.
SO NOW FINALLY REAL QUICK.
SO NOW FINALLY REAL QUICK. LIKE I MENTIONED WE'RE WORKING
LIKE I MENTIONED WE'RE WORKING
LIKE I MENTIONED WE'RE WORKING ON QUANTIZATION TRAINING API
ON QUANTIZATION TRAINING API
ON QUANTIZATION TRAINING API AND IT SHOULD BE READY SOON AND
AND IT SHOULD BE READY SOON AND
AND IT SHOULD BE READY SOON AND QUANTIZING OUR -- (?).
QUANTIZING OUR -- (?).
QUANTIZING OUR -- (?). THEN I INCLUDE WE'RE MAKING
THEN I INCLUDE WE'RE MAKING
THEN I INCLUDE WE'RE MAKING IMPROVE MGTS TO THE HYBRID
IMPROVE MGTS TO THE HYBRID
IMPROVE MGTS TO THE HYBRID QUANTIZATION TO BE MORE
QUANTIZATION TO BE MORE
QUANTIZATION TO BE MORE ACCURATE FOR -- (?)
ACCURATE FOR -- (?)
ACCURATE FOR -- (?) LONGER TERM I DON'T KNOW IF YOU
LONGER TERM I DON'T KNOW IF YOU
LONGER TERM I DON'T KNOW IF YOU HAVE HEARD ABOUT --
HAVE HEARD ABOUT --
HAVE HEARD ABOUT -- STATE-OF-THE-ART INFRASTRUCTURE.
STATE-OF-THE-ART INFRASTRUCTURE.
STATE-OF-THE-ART INFRASTRUCTURE. THIS IS PARTICULARLY
THIS IS PARTICULARLY
THIS IS PARTICULARLY INTERESTING TO US BECAUSE IT IS
INTERESTING TO US BECAUSE IT IS
INTERESTING TO US BECAUSE IT IS A BETTER WAY FOR US TO WRITE
A BETTER WAY FOR US TO WRITE
A BETTER WAY FOR US TO WRITE THESE TRANSFORMATIONS AND AT
THESE TRANSFORMATIONS AND AT
THESE TRANSFORMATIONS AND AT THE BEGINNING OF THE TALK WE'RE
THE BEGINNING OF THE TALK WE'RE
THE BEGINNING OF THE TALK WE'RE TAKING THE MODERN WAY OF
TAKING THE MODERN WAY OF
TAKING THE MODERN WAY OF TRANSFORMING ONE PROGRAM INTO
TRANSFORMING ONE PROGRAM INTO
TRANSFORMING ONE PROGRAM INTO ANOTHER.
ANOTHER.
ANOTHER. SOME OF THE THINGS WE WANT TO
SOME OF THE THINGS WE WANT TO
SOME OF THE THINGS WE WANT TO ENABLE IS BETTER TARGETED
ENABLE IS BETTER TARGETED
ENABLE IS BETTER TARGETED HARDWARE.
HARDWARE.
HARDWARE. WHERE SPECIFICATIONS ARE GREAT.
WHERE SPECIFICATIONS ARE GREAT.
WHERE SPECIFICATIONS ARE GREAT. USERS CAN TARGET AND EXECUTE
USERS CAN TARGET AND EXECUTE
USERS CAN TARGET AND EXECUTE DIFFERENT HARDWARE.
DIFFERENT HARDWARE.
DIFFERENT HARDWARE. SOME USERS JUST WANT TO STAY IN
SOME USERS JUST WANT TO STAY IN
SOME USERS JUST WANT TO STAY IN ONE HARDWARE AND GET THE BEST
ONE HARDWARE AND GET THE BEST
ONE HARDWARE AND GET THE BEST OUT OF IT.
OUT OF IT.
OUT OF IT. WE'RE HOPING WITH THE NEW
WE'RE HOPING WITH THE NEW
WE'RE HOPING WITH THE NEW INFRASTRUCTURE WE'RE BUILDING
INFRASTRUCTURE WE'RE BUILDING
INFRASTRUCTURE WE'RE BUILDING IT SHOULD BE POSSIBLE.
IT SHOULD BE POSSIBLE.
IT SHOULD BE POSSIBLE. FINALLY I REALLY JUST WANT TO
FINALLY I REALLY JUST WANT TO
FINALLY I REALLY JUST WANT TO ENCOURAGE YOU TO TRY IT OUT AND
ENCOURAGE YOU TO TRY IT OUT AND
ENCOURAGE YOU TO TRY IT OUT AND GIVE US FEEDBACK WHAT
GIVE US FEEDBACK WHAT
GIVE US FEEDBACK WHAT TECHNIQUES YOU WOULD LIKE TO
TECHNIQUES YOU WOULD LIKE TO
TECHNIQUES YOU WOULD LIKE TO SEE.
SEE.
SEE. RESEARCHES, THERE IS TECHNIQUES
RESEARCHES, THERE IS TECHNIQUES
RESEARCHES, THERE IS TECHNIQUES POPPING UP-TO-DATE ALL OVER THE
POPPING UP-TO-DATE ALL OVER THE
POPPING UP-TO-DATE ALL OVER THE PLACE AND A LOT OF THE WORK WE
PLACE AND A LOT OF THE WORK WE
PLACE AND A LOT OF THE WORK WE HAVE TO GO THROUGH IS WHAT IS
HAVE TO GO THROUGH IS WHAT IS
HAVE TO GO THROUGH IS WHAT IS USEFUL AND NOT AND WHAT IS
USEFUL AND NOT AND WHAT IS
USEFUL AND NOT AND WHAT IS GENERAL AND WHAT IS VERY
GENERAL AND WHAT IS VERY
GENERAL AND WHAT IS VERY SPECIFIC.
SPECIFIC.
SPECIFIC. WE WOULD LOVE TO HEAR YOUR
WE WOULD LOVE TO HEAR YOUR
WE WOULD LOVE TO HEAR YOUR FEEDBACK ABOUT THAT AND ALSO
FEEDBACK ABOUT THAT AND ALSO
FEEDBACK ABOUT THAT AND ALSO ABOUT THE TOOLS THAT WE ALREADY
ABOUT THE TOOLS THAT WE ALREADY
ABOUT THE TOOLS THAT WE ALREADY HAVE IN THE TOOLKIT.
HAVE IN THE TOOLKIT.
HAVE IN THE TOOLKIT. WE'RE TRYING TO MAKE THEM
WE'RE TRYING TO MAKE THEM
WE'RE TRYING TO MAKE THEM EASIER FOR PEOPLE TO USE.
EASIER FOR PEOPLE TO USE.
EASIER FOR PEOPLE TO USE. WE KNOW WE STILL HAVE A LONG
WE KNOW WE STILL HAVE A LONG
WE KNOW WE STILL HAVE A LONG WAY TO GO BUT ANYTHING FEEDBACK
WAY TO GO BUT ANYTHING FEEDBACK
WAY TO GO BUT ANYTHING FEEDBACK YOU CAN PROVIDE WOULD BE REALLY
YOU CAN PROVIDE WOULD BE REALLY
YOU CAN PROVIDE WOULD BE REALLY APPRECIATED AND I THINK THERE
APPRECIATED AND I THINK THERE
APPRECIATED AND I THINK THERE IS A LITTLE BIT OF TIME FOR
IS A LITTLE BIT OF TIME FOR
IS A LITTLE BIT OF TIME FOR QUESTIONS IF ANY OF YOU HAVE
QUESTIONS IF ANY OF YOU HAVE
QUESTIONS IF ANY OF YOU HAVE QUESTIONS.
QUESTIONS.
QUESTIONS. [APPLAUSE]
[APPLAUSE]
[APPLAUSE] THANKS.
>> HI, THANK YOU FOR THE
>> HI, THANK YOU FOR THE
>> HI, THANK YOU FOR THE PRESENTATION.
PRESENTATION.
PRESENTATION. I HAVE A QUESTION REGARDING THE
I HAVE A QUESTION REGARDING THE
I HAVE A QUESTION REGARDING THE TRAINING WITH INTEGER
TRAINING WITH INTEGER
TRAINING WITH INTEGER QUANTIZATIONS.
QUANTIZATIONS.
QUANTIZATIONS. IN THE PIPELINE IS THAT GOING
IN THE PIPELINE IS THAT GOING
IN THE PIPELINE IS THAT GOING TO BE TRUE QUANTIZATION DURING
TO BE TRUE QUANTIZATION DURING
TO BE TRUE QUANTIZATION DURING TRAINING?
TRAINING?
TRAINING? >> NO, RIGHT NOW THE WAY -- BY
>> NO, RIGHT NOW THE WAY -- BY
>> NO, RIGHT NOW THE WAY -- BY TRUE YOU EXPECT ALL APPEAR
TRUE YOU EXPECT ALL APPEAR
TRUE YOU EXPECT ALL APPEAR RATIONS HAPPEN IN THE INTEGER.
RATIONS HAPPEN IN THE INTEGER.
RATIONS HAPPEN IN THE INTEGER. NOT RIGHT NOW.
NOT RIGHT NOW.
NOT RIGHT NOW. SOMETHING I WANT ENABLED
SOMETHING I WANT ENABLED
SOMETHING I WANT ENABLED BECAUSE I WANT TO MAKE TRAINING
BECAUSE I WANT TO MAKE TRAINING
BECAUSE I WANT TO MAKE TRAINING AVAILABLE BUT RIGHT NOW THE WAY
AVAILABLE BUT RIGHT NOW THE WAY
AVAILABLE BUT RIGHT NOW THE WAY THAT WE'RE TARGETING IS I DON'T
THAT WE'RE TARGETING IS I DON'T
THAT WE'RE TARGETING IS I DON'T KNOW IF YOU'RE FAMILIAR WITH
KNOW IF YOU'RE FAMILIAR WITH
KNOW IF YOU'RE FAMILIAR WITH TENSORFLOW APIs, WE HAVE THE
TENSORFLOW APIs, WE HAVE THE
TENSORFLOW APIs, WE HAVE THE LOW-LEVEL API QUANTIZATION THAT
LOW-LEVEL API QUANTIZATION THAT
LOW-LEVEL API QUANTIZATION THAT EMULATES THESE LOSSES.
EMULATES THESE LOSSES.
EMULATES THESE LOSSES. AND THAT ONE IS STILL BASICALLY
AND THAT ONE IS STILL BASICALLY
AND THAT ONE IS STILL BASICALLY WHAT WE DO THERE IS WE QUANTIZE
WHAT WE DO THERE IS WE QUANTIZE
WHAT WE DO THERE IS WE QUANTIZE PARAMETERS AND QUANTIZE THEM
PARAMETERS AND QUANTIZE THEM
PARAMETERS AND QUANTIZE THEM AND DO THE FLOW OPERATION.
AND DO THE FLOW OPERATION.
AND DO THE FLOW OPERATION. THAT'S WHAT WE'RE USING RIGHT
THAT'S WHAT WE'RE USING RIGHT
THAT'S WHAT WE'RE USING RIGHT NOW.
NOW.
NOW. LONGER TERM WE WANT TO USE IT
LONGER TERM WE WANT TO USE IT
LONGER TERM WE WANT TO USE IT THROUGH INTEGER FORWARD PACKETS.
THROUGH INTEGER FORWARD PACKETS.
THROUGH INTEGER FORWARD PACKETS. >> THANK YOU.
>> THANK YOU.
>> THANK YOU. >> ONE QUESTION.
>> ONE QUESTION.
>> ONE QUESTION. SO AFTER YOU DO THE
SO AFTER YOU DO THE
SO AFTER YOU DO THE QUANTIZATION, IS THERE A WAY
QUANTIZATION, IS THERE A WAY
QUANTIZATION, IS THERE A WAY YOU CAN ALSO VISUALIZE OF THE
YOU CAN ALSO VISUALIZE OF THE
YOU CAN ALSO VISUALIZE OF THE FINISHED QUANT ADVERTISED MODEL.
FINISHED QUANT ADVERTISED MODEL.
FINISHED QUANT ADVERTISED MODEL. IS THERE A WAY YOU CAN ALSO --
IS THERE A WAY YOU CAN ALSO --
IS THERE A WAY YOU CAN ALSO -- THE OTHER QUESTION WAS WHAT
THE OTHER QUESTION WAS WHAT
THE OTHER QUESTION WAS WHAT SORT OF TOOLS WILL YOU PROVIDE
SORT OF TOOLS WILL YOU PROVIDE
SORT OF TOOLS WILL YOU PROVIDE AS FAR AS SORT OF DO LIKE MODEL
AS FAR AS SORT OF DO LIKE MODEL
AS FAR AS SORT OF DO LIKE MODEL CORRECTNESS AT LEAST EVALUATE
CORRECTNESS AT LEAST EVALUATE
CORRECTNESS AT LEAST EVALUATE WHETHER THE QUANTIZED MODEL IS
WHETHER THE QUANTIZED MODEL IS
WHETHER THE QUANTIZED MODEL IS FUNCTIONALLY CORRECT IN A SENSE?
FUNCTIONALLY CORRECT IN A SENSE?
FUNCTIONALLY CORRECT IN A SENSE? >> VISUALIZATION, WE HAVE
>> VISUALIZATION, WE HAVE
>> VISUALIZATION, WE HAVE ADDITIONAL -- YOU CAN SEE THE
ADDITIONAL -- YOU CAN SEE THE
ADDITIONAL -- YOU CAN SEE THE QUANTIZED MODEL.
QUANTIZED MODEL.
QUANTIZED MODEL. I DON'T KNOW IF IT WILL GIVE
I DON'T KNOW IF IT WILL GIVE
I DON'T KNOW IF IT WILL GIVE YOU A LOT OF INFORMATION
YOU A LOT OF INFORMATION
YOU A LOT OF INFORMATION DEPENDING WHAT YOU ARE LOOKING
DEPENDING WHAT YOU ARE LOOKING
DEPENDING WHAT YOU ARE LOOKING FOR.
FOR.
FOR. WE ALSO WANT TO MAKE OUR
WE ALSO WANT TO MAKE OUR
WE ALSO WANT TO MAKE OUR TOOLING A BIT BETTER PERHAPS
TOOLING A BIT BETTER PERHAPS
TOOLING A BIT BETTER PERHAPS FOR WHATEVER REASON YOU WANT TO
FOR WHATEVER REASON YOU WANT TO
FOR WHATEVER REASON YOU WANT TO GET RESEARCH AND START LOOKING
GET RESEARCH AND START LOOKING
GET RESEARCH AND START LOOKING AT THE ACTIVATIONS AND HOW THEY
AT THE ACTIVATIONS AND HOW THEY
AT THE ACTIVATIONS AND HOW THEY CHANGE.
CHANGE.
CHANGE. >> INSERTED OPS AND SO FORTH.
>> INSERTED OPS AND SO FORTH.
>> INSERTED OPS AND SO FORTH. >> WITH THE VISUALIZER YOU CAN
>> WITH THE VISUALIZER YOU CAN
>> WITH THE VISUALIZER YOU CAN SEE HOW THE GRAPH CHANGES.
SEE HOW THE GRAPH CHANGES.
SEE HOW THE GRAPH CHANGES. CORRECTNESS IS REALLY TRICKY
CORRECTNESS IS REALLY TRICKY
CORRECTNESS IS REALLY TRICKY BECAUSE IN MY EXPERIENCE THE
BECAUSE IN MY EXPERIENCE THE
BECAUSE IN MY EXPERIENCE THE ONLY THING THAT REALLY WORKS IS
ONLY THING THAT REALLY WORKS IS
ONLY THING THAT REALLY WORKS IS TO REALLY EVALUATE UNDER REAL
TO REALLY EVALUATE UNDER REAL
TO REALLY EVALUATE UNDER REAL DATA THAT YOU CARE TO RUN YOUR
DATA THAT YOU CARE TO RUN YOUR
DATA THAT YOU CARE TO RUN YOUR MODEL ON.
MODEL ON.
MODEL ON. WE TRIED TO DO THINGS LIKE --
WE TRIED TO DO THINGS LIKE --
WE TRIED TO DO THINGS LIKE -- VERSUS THE FULL PRECISION
VERSUS THE FULL PRECISION
VERSUS THE FULL PRECISION VERSES THE QUANTIZED ONE.
VERSES THE QUANTIZED ONE.
VERSES THE QUANTIZED ONE. SOME CATASTROPHIC NUMERICAL
SOME CATASTROPHIC NUMERICAL
SOME CATASTROPHIC NUMERICAL ERRORS BUT OTHERWISE JUST A
ERRORS BUT OTHERWISE JUST A
ERRORS BUT OTHERWISE JUST A GUESS PARTICULARLY DEPENDING ON
GUESS PARTICULARLY DEPENDING ON
GUESS PARTICULARLY DEPENDING ON THE OUTPUT LAYERS.
THE OUTPUT LAYERS.
THE OUTPUT LAYERS. THEY ARE EASIER TO QUANTIZE.
THEY ARE EASIER TO QUANTIZE.
THEY ARE EASIER TO QUANTIZE. REGRESSIONS ARE HARDER BECAUSE
REGRESSIONS ARE HARDER BECAUSE
REGRESSIONS ARE HARDER BECAUSE YOU NOW CARE ABOUT THE ACTUAL
YOU NOW CARE ABOUT THE ACTUAL
YOU NOW CARE ABOUT THE ACTUAL VALUES.
VALUES.
VALUES. IT'S AN OPEN --
IT'S AN OPEN --
IT'S AN OPEN -- >> THANK YOU.
>> THANK YOU.
>> THANK YOU. >> I HAVE A QUESTION ABOUT THE
>> I HAVE A QUESTION ABOUT THE
>> I HAVE A QUESTION ABOUT THE RESULTS FROM THE GNT TRAINING
RESULTS FROM THE GNT TRAINING
RESULTS FROM THE GNT TRAINING WITH INDUCED SPARSETY.
WITH INDUCED SPARSETY.
WITH INDUCED SPARSETY. I WAS WONDERING IF YOU HAD
I WAS WONDERING IF YOU HAD
I WAS WONDERING IF YOU HAD INSIGHTS ON WHY THE TRAINING
INSIGHTS ON WHY THE TRAINING
INSIGHTS ON WHY THE TRAINING WITH 80% SPARSETY BETTER THAN
WITH 80% SPARSETY BETTER THAN
WITH 80% SPARSETY BETTER THAN THE ORIGINAL VERSION.
THE ORIGINAL VERSION.
THE ORIGINAL VERSION. >> IN THIS CASE IT IS SOUND
>> IN THIS CASE IT IS SOUND
>> IN THIS CASE IT IS SOUND REGULARIZATION HAPPENS.
REGULARIZATION HAPPENS.
REGULARIZATION HAPPENS. I'VE SEEN THE SAME ARE SOME
I'VE SEEN THE SAME ARE SOME
I'VE SEEN THE SAME ARE SOME QUANTIZED MODELS.
QUANTIZED MODELS.
QUANTIZED MODELS. I NEVER SAT DOWN TO UNDERSTAND
I NEVER SAT DOWN TO UNDERSTAND
I NEVER SAT DOWN TO UNDERSTAND WHY THE REASONS ARE.
WHY THE REASONS ARE.
WHY THE REASONS ARE. SOMETIMES IT IS JUST WITHIN THE
SOMETIMES IT IS JUST WITHIN THE
SOMETIMES IT IS JUST WITHIN THE NOISE.
NOISE.
NOISE. IT ALL DEPENDS ON THE
IT ALL DEPENDS ON THE
IT ALL DEPENDS ON THE EVALUATION SET, RIGHT?
EVALUATION SET, RIGHT?
EVALUATION SET, RIGHT? IF IT'S REALLY NOT THAT BIG OR
IF IT'S REALLY NOT THAT BIG OR
IF IT'S REALLY NOT THAT BIG OR NOT THAT MEANINGFUL THESE JOBS
NOT THAT MEANINGFUL THESE JOBS
NOT THAT MEANINGFUL THESE JOBS ARE AT ALL POSSIBLE.
ARE AT ALL POSSIBLE.
ARE AT ALL POSSIBLE. I'VE SEEN SOME MODELS WHERE IT
I'VE SEEN SOME MODELS WHERE IT
I'VE SEEN SOME MODELS WHERE IT LOOKS GREAT AFTER YOU QUANTIZE
LOOKS GREAT AFTER YOU QUANTIZE
LOOKS GREAT AFTER YOU QUANTIZE IT AND YOU THROW A NEW DATASET
IT AND YOU THROW A NEW DATASET
IT AND YOU THROW A NEW DATASET AND NOISE HERE AND THEN YOU
AND NOISE HERE AND THEN YOU
AND NOISE HERE AND THEN YOU CLEARLY SEE THE DIFFERENCE
CLEARLY SEE THE DIFFERENCE
CLEARLY SEE THE DIFFERENCE BETWEEN ONE AND THE OTHER.
BETWEEN ONE AND THE OTHER.
BETWEEN ONE AND THE OTHER. IT IS A LOT OF IT CAN BE JUST
IT IS A LOT OF IT CAN BE JUST
IT IS A LOT OF IT CAN BE JUST NOISE.
NOISE.
NOISE. >> YOU MENTIONED EXPLAINABILITY
>> YOU MENTIONED EXPLAINABILITY
>> YOU MENTIONED EXPLAINABILITY AND TECHNIQUE COULD BE LIKE --
AND TECHNIQUE COULD BE LIKE --
AND TECHNIQUE COULD BE LIKE -- DO YOU HAVE ANY INSIGHTS ON HOW
DO YOU HAVE ANY INSIGHTS ON HOW
DO YOU HAVE ANY INSIGHTS ON HOW THESE TECHNIQUES EFFECT THE
THESE TECHNIQUES EFFECT THE
THESE TECHNIQUES EFFECT THE ABILITY TO CALCULATE THE
ABILITY TO CALCULATE THE
ABILITY TO CALCULATE THE GRADIENTS TO CALCULATE THE --
GRADIENTS TO CALCULATE THE --
GRADIENTS TO CALCULATE THE -- >> TRYING TO UNDERSTAND THE
>> TRYING TO UNDERSTAND THE
>> TRYING TO UNDERSTAND THE NETWORKS AND TRYING TO
NETWORKS AND TRYING TO
NETWORKS AND TRYING TO UNDERSTAND THE NEURAL NETWORKS.
UNDERSTAND THE NEURAL NETWORKS.
UNDERSTAND THE NEURAL NETWORKS. SO FAR I HAVEN'T GOT ANY LUCK
SO FAR I HAVEN'T GOT ANY LUCK
SO FAR I HAVEN'T GOT ANY LUCK TRYING TO GET THE PEOPLE ON
TRYING TO GET THE PEOPLE ON
TRYING TO GET THE PEOPLE ON THAT SIDE EXCITED ABOUT IT.
THAT SIDE EXCITED ABOUT IT.
THAT SIDE EXCITED ABOUT IT. I REALLY DON'T HAVE ANY
I REALLY DON'T HAVE ANY
I REALLY DON'T HAVE ANY MEANINGFUL THING TO SAY BECAUSE
MEANINGFUL THING TO SAY BECAUSE
MEANINGFUL THING TO SAY BECAUSE I HAVEN'T RUN MANY EXPERIMENTS
I HAVEN'T RUN MANY EXPERIMENTS
I HAVEN'T RUN MANY EXPERIMENTS AROUND IT.
AROUND IT.
AROUND IT. >> SO WHAT IS THE BEST WAY TO
>> SO WHAT IS THE BEST WAY TO
>> SO WHAT IS THE BEST WAY TO HANDLE FRAGMENTATION OF
HANDLE FRAGMENTATION OF
HANDLE FRAGMENTATION OF HARDWARE AND MORE OFTEN THERE
HARDWARE AND MORE OFTEN THERE
HARDWARE AND MORE OFTEN THERE AE MOVING PARTS AND HAVE --
AE MOVING PARTS AND HAVE --
AE MOVING PARTS AND HAVE -- WHAT ARE THE BEST WAYS THERE?
WHAT ARE THE BEST WAYS THERE?
WHAT ARE THE BEST WAYS THERE? >> ONE WAY WE TRIED TO DO IT
>> ONE WAY WE TRIED TO DO IT
>> ONE WAY WE TRIED TO DO IT WAS SPECIFICATIONS.
WAS SPECIFICATIONS.
WAS SPECIFICATIONS. AND YOU KNOW LIKE I DON'T KNOW
AND YOU KNOW LIKE I DON'T KNOW
AND YOU KNOW LIKE I DON'T KNOW IF IT MAKES OUR HARDWARE
IF IT MAKES OUR HARDWARE
IF IT MAKES OUR HARDWARE PARTNERS HAPPY.
PARTNERS HAPPY.
PARTNERS HAPPY. WE WOULD LIKE TO TARGET THEIR
WE WOULD LIKE TO TARGET THEIR
WE WOULD LIKE TO TARGET THEIR HARDWARE IN THE MOST PRECISE
HARDWARE IN THE MOST PRECISE
HARDWARE IN THE MOST PRECISE AND EFFICIENT WAY BUT IT IS ONE
AND EFFICIENT WAY BUT IT IS ONE
AND EFFICIENT WAY BUT IT IS ONE WAY WE TRIED TO ADDRESS IT.
WAY WE TRIED TO ADDRESS IT.
WAY WE TRIED TO ADDRESS IT. YOU KNOW, WITH OUR KNOWLEDGE OF
YOU KNOW, WITH OUR KNOWLEDGE OF
YOU KNOW, WITH OUR KNOWLEDGE OF SOME HARDWARE IS THERE AND WHAT
SOME HARDWARE IS THERE AND WHAT
SOME HARDWARE IS THERE AND WHAT IS SUPPORTED WE TRIED TO CREATE
IS SUPPORTED WE TRIED TO CREATE
IS SUPPORTED WE TRIED TO CREATE SPECIFICATIONS FOR EVERYBODY.
SPECIFICATIONS FOR EVERYBODY.
SPECIFICATIONS FOR EVERYBODY. LONGER TERM AND I DON'T WANT TO
LONGER TERM AND I DON'T WANT TO
LONGER TERM AND I DON'T WANT TO SAY TOO MUCH BECAUSE I REALLY
SAY TOO MUCH BECAUSE I REALLY
SAY TOO MUCH BECAUSE I REALLY HAVEN'T -- I DON'T HAVE A VERY
HAVEN'T -- I DON'T HAVE A VERY
HAVEN'T -- I DON'T HAVE A VERY CONCRETE PLAN TO SHARE.
CONCRETE PLAN TO SHARE.
CONCRETE PLAN TO SHARE. PART OF WHAT WE'RE BUILDING
PART OF WHAT WE'RE BUILDING
PART OF WHAT WE'RE BUILDING WITH THE INFRASTRUCTURE WE WANT
WITH THE INFRASTRUCTURE WE WANT
WITH THE INFRASTRUCTURE WE WANT TO BE ABLE TO BETTER TARGET THE
TO BE ABLE TO BETTER TARGET THE
TO BE ABLE TO BETTER TARGET THE HARDWARE AND BETTER PARTNER
HARDWARE AND BETTER PARTNER
HARDWARE AND BETTER PARTNER WITH HARDWARE VENDORS TO
WITH HARDWARE VENDORS TO
WITH HARDWARE VENDORS TO UNDERSTAND THEIR CAPABILITIES
UNDERSTAND THEIR CAPABILITIES
UNDERSTAND THEIR CAPABILITIES AND CREATE THE TRANSFORMATIONS
AND CREATE THE TRANSFORMATIONS
AND CREATE THE TRANSFORMATIONS THAT TARGET THAT HARDWARE.
THAT TARGET THAT HARDWARE.
THAT TARGET THAT HARDWARE. WE ARE GOING TO MAKE BETTER.
WE ARE GOING TO MAKE BETTER.
WE ARE GOING TO MAKE BETTER. >> YOU GO WITH THE LESS COMMON
>> YOU GO WITH THE LESS COMMON
>> YOU GO WITH THE LESS COMMON DENOMINATOR.
DENOMINATOR.
DENOMINATOR. (?)
(?)
(?) >> WE HAVE ALL THE DIFFERENT
>> WE HAVE ALL THE DIFFERENT
>> WE HAVE ALL THE DIFFERENT ONES LIKE WE HAVE THREE TYPES.
ONES LIKE WE HAVE THREE TYPES.
ONES LIKE WE HAVE THREE TYPES. TWO HOPEFULLY WE'LL BE ABLE TO
TWO HOPEFULLY WE'LL BE ABLE TO
TWO HOPEFULLY WE'LL BE ABLE TO MIX AND MATCH THOSE DIFFERENT
MIX AND MATCH THOSE DIFFERENT
MIX AND MATCH THOSE DIFFERENT TYPES.
TYPES.
TYPES. AT THE END OF THE DAY IT'S A
AT THE END OF THE DAY IT'S A
AT THE END OF THE DAY IT'S A VERY ARBITRARY AND IT'S THIS
VERY ARBITRARY AND IT'S THIS
VERY ARBITRARY AND IT'S THIS ONE IS HYBRID AND WE SHOULD BE
ONE IS HYBRID AND WE SHOULD BE
ONE IS HYBRID AND WE SHOULD BE ABLE TO TAKE ADVANTAGE OF
ABLE TO TAKE ADVANTAGE OF
ABLE TO TAKE ADVANTAGE OF MIXING AND MATCHING UP TO GET
MIXING AND MATCHING UP TO GET
MIXING AND MATCHING UP TO GET SOMETHING BETTER.
SOMETHING BETTER.
SOMETHING BETTER. >> THANK YOU.
>> THANK YOU.
>> THANK YOU. >> I HAVE A QUESTION ABOUT
>> I HAVE A QUESTION ABOUT
>> I HAVE A QUESTION ABOUT PRUNING.
PRUNING.
PRUNING. AS A GENERAL RULE IN LAYERS
AS A GENERAL RULE IN LAYERS
AS A GENERAL RULE IN LAYERS THINGS OPERATIONS ARE CONVERTED
THINGS OPERATIONS ARE CONVERTED
THINGS OPERATIONS ARE CONVERTED TO MATRIX MULTIPLY BECAUSE OF
TO MATRIX MULTIPLY BECAUSE OF
TO MATRIX MULTIPLY BECAUSE OF THEIR EFFICIENCY.
THEIR EFFICIENCY.
THEIR EFFICIENCY. WITH PRUNING YOUR PASSING AN
WITH PRUNING YOUR PASSING AN
WITH PRUNING YOUR PASSING AN INDIVIDUAL MULTIPLE OPERATIONS
INDIVIDUAL MULTIPLE OPERATIONS
INDIVIDUAL MULTIPLE OPERATIONS ONE-BY-ONE.
ONE-BY-ONE.
ONE-BY-ONE. THERE MUST BE SOME CROSSOVER
THERE MUST BE SOME CROSSOVER
THERE MUST BE SOME CROSSOVER POINT WHICH YOU NEED TO PRUNE
POINT WHICH YOU NEED TO PRUNE
POINT WHICH YOU NEED TO PRUNE BY 10, 15, 20% BEFORE YOU ARE
BY 10, 15, 20% BEFORE YOU ARE
BY 10, 15, 20% BEFORE YOU ARE CROSSING OVER AND ACTUALLY GET
CROSSING OVER AND ACTUALLY GET
CROSSING OVER AND ACTUALLY GET AN IMPROVEMENT.
AN IMPROVEMENT.
AN IMPROVEMENT. THOUGHTS ON WHERE THAT IS?
THOUGHTS ON WHERE THAT IS?
THOUGHTS ON WHERE THAT IS? >> I DON'T KNOW IF THIS IS
>> I DON'T KNOW IF THIS IS
>> I DON'T KNOW IF THIS IS EXACTLY WHAT YOU'RE ASKING BUT
EXACTLY WHAT YOU'RE ASKING BUT
EXACTLY WHAT YOU'RE ASKING BUT OUR PRUNING API SUPPORTS DEFINE
OUR PRUNING API SUPPORTS DEFINE
OUR PRUNING API SUPPORTS DEFINE WHAT THE PRUNING STRUCTURE IS.
WHAT THE PRUNING STRUCTURE IS.
WHAT THE PRUNING STRUCTURE IS. FOR EXAMPLE, WE KNOW THAT FOR
FOR EXAMPLE, WE KNOW THAT FOR
FOR EXAMPLE, WE KNOW THAT FOR CPU, THE INSTRUCTIONS WE HAVE
CPU, THE INSTRUCTIONS WE HAVE
CPU, THE INSTRUCTIONS WE HAVE CAN ACCOMMODATE 16 VALUE.
CAN ACCOMMODATE 16 VALUE.
CAN ACCOMMODATE 16 VALUE. WE WANT TO WORK ON CPU WE
WE WANT TO WORK ON CPU WE
WE WANT TO WORK ON CPU WE EXPECT TO SET THE SETTINGS TO
EXPECT TO SET THE SETTINGS TO
EXPECT TO SET THE SETTINGS TO SAY I WANT TO PRUNE IN BLOCKS
SAY I WANT TO PRUNE IN BLOCKS
SAY I WANT TO PRUNE IN BLOCKS OF 1 BY 16 AND HOW WE CAN GET
OF 1 BY 16 AND HOW WE CAN GET
OF 1 BY 16 AND HOW WE CAN GET THE SPEED OF CPU FOR EXAMPLE.
THE SPEED OF CPU FOR EXAMPLE.
THE SPEED OF CPU FOR EXAMPLE. UNFORTUNATELY RIGHT NOW BROADLY
UNFORTUNATELY RIGHT NOW BROADLY
UNFORTUNATELY RIGHT NOW BROADLY WILL BE HARDWARE DEPENDENT.
WILL BE HARDWARE DEPENDENT.
WILL BE HARDWARE DEPENDENT. BUT THAT'S ONE THING THAT YOU
BUT THAT'S ONE THING THAT YOU
BUT THAT'S ONE THING THAT YOU CAN DO RIGHT NOW.
CAN DO RIGHT NOW.
CAN DO RIGHT NOW. >> IS THERE SOME SORT OF
>> IS THERE SOME SORT OF
>> IS THERE SOME SORT OF MINIMUM?
MINIMUM?
MINIMUM? >> WE'RE OUT OF TIME.
>> WE'RE OUT OF TIME.
>> WE'RE OUT OF TIME. I'M SORRY.
I'M SORRY.
I'M SORRY. I'M SURE YOU ALL CAN TALK.
I'M SURE YOU ALL CAN TALK.
I'M SURE YOU ALL CAN TALK. THANK YOU ALL SO MUCH FOR
THANK YOU ALL SO MUCH FOR
THANK YOU ALL SO MUCH FOR COMING AND --
COMING AND --
COMING AND -- [APPLAUSE]
AND HOW IT WORKS.
AND HOW IT WORKS.
AND HOW IT WORKS. YOU SAW SIMPLE EXAMPLE OF
YOU SAW SIMPLE EXAMPLE OF
YOU SAW SIMPLE EXAMPLE OF MATCHING NUMBERS TO EACH OTHER
MATCHING NUMBERS TO EACH OTHER
MATCHING NUMBERS TO EACH OTHER AND HOW USING PYTHON CODE A
AND HOW USING PYTHON CODE A
AND HOW USING PYTHON CODE A COMPUTER COULD LEARN THROUGH
COMPUTER COULD LEARN THROUGH
COMPUTER COULD LEARN THROUGH TRIAL AND ERROR WHAT THE
TRIAL AND ERROR WHAT THE
TRIAL AND ERROR WHAT THE RELATIONSHIP BETWEEN THE NUMBERS
RELATIONSHIP BETWEEN THE NUMBERS
RELATIONSHIP BETWEEN THE NUMBERS WAS.
WAS.
WAS. IN THIS EPISODE YOU'RE GOING TO
IN THIS EPISODE YOU'RE GOING TO
IN THIS EPISODE YOU'RE GOING TO TAKE IT A LITTLE FURTHER BY
TAKE IT A LITTLE FURTHER BY
TAKE IT A LITTLE FURTHER BY TEACHING YOUR COMPUTER HOW TO
TEACHING YOUR COMPUTER HOW TO
TEACHING YOUR COMPUTER HOW TO SEE AND RECOGNIZE DIFFERENT
SEE AND RECOGNIZE DIFFERENT
SEE AND RECOGNIZE DIFFERENT OBJECTS.
OBJECTS.
OBJECTS. FOR EXAMPLE, LOOK AT THOSE
FOR EXAMPLE, LOOK AT THOSE
FOR EXAMPLE, LOOK AT THOSE PICTURES.
PICTURES.
PICTURES. HOW MANY SHOES DO YOU SEE?
HOW MANY SHOES DO YOU SEE?
HOW MANY SHOES DO YOU SEE? YOU MIGHT SAY TWO; RIGHT?
YOU MIGHT SAY TWO; RIGHT?
YOU MIGHT SAY TWO; RIGHT? BUT HOW DO YOU KNOW THEY ARE
BUT HOW DO YOU KNOW THEY ARE
BUT HOW DO YOU KNOW THEY ARE SHOES?
SHOES?
SHOES? IMAGINE IF SOMEBODY HAD NEVER
IMAGINE IF SOMEBODY HAD NEVER
IMAGINE IF SOMEBODY HAD NEVER SEEN SHOES BEFORE, HOW WOULD YOU
SEEN SHOES BEFORE, HOW WOULD YOU
SEEN SHOES BEFORE, HOW WOULD YOU TELL THEM THAT DESPITE THE GREAT
TELL THEM THAT DESPITE THE GREAT
TELL THEM THAT DESPITE THE GREAT DIFFERENCE BETWEEN THE HIGH HEEL
DIFFERENCE BETWEEN THE HIGH HEEL
DIFFERENCE BETWEEN THE HIGH HEEL AND THE SPORTS SHOE THEY'RE
AND THE SPORTS SHOE THEY'RE
AND THE SPORTS SHOE THEY'RE STILL BOTH SHOES?
STILL BOTH SHOES?
STILL BOTH SHOES? WHICH IF YOU REMEMBER WAS THE
WHICH IF YOU REMEMBER WAS THE
WHICH IF YOU REMEMBER WAS THE SIZE OF THE OUR IMAGE.
SIZE OF THE OUR IMAGE.
SIZE OF THE OUR IMAGE. OUR LAST LAYER IS 10 WHICH IF
OUR LAST LAYER IS 10 WHICH IF
OUR LAST LAYER IS 10 WHICH IF YOU REMEMBER IS THE NUMBER OF
YOU REMEMBER IS THE NUMBER OF
YOU REMEMBER IS THE NUMBER OF DIFFERENT 2 OF THEM CLOSING
DIFFERENT 2 OF THEM CLOSING
DIFFERENT 2 OF THEM CLOSING DEFINITE 2 OF THEMITEMS OF CLOS
DEFINITE 2 OF THEMITEMS OF CLOS
DEFINITE 2 OF THEMITEMS OF CLOS DEFINITE ITEMS OF FILTER WHICH
DEFINITE ITEMS OF FILTER WHICH
DEFINITE ITEMS OF FILTER WHICH AND OUTPUTS ONE OF THE VALUES.
AND OUTPUTS ONE OF THE VALUES.
AND OUTPUTS ONE OF THE VALUES. SO WHAT ABOUT THIS NUMBER, 128?
SO WHAT ABOUT THIS NUMBER, 128?
SO WHAT ABOUT THIS NUMBER, 128? WHAT DOES THAT DO?
WHAT DOES THAT DO?
WHAT DOES THAT DO? WELL, THINK OF IT LIKE THIS.
WELL, THINK OF IT LIKE THIS.
WELL, THINK OF IT LIKE THIS. WE'RE GOING TO HAVE 128
WE'RE GOING TO HAVE 128
WE'RE GOING TO HAVE 128 FUNCTIONS, EACH ONE OF WHICH HAS
FUNCTIONS, EACH ONE OF WHICH HAS
FUNCTIONS, EACH ONE OF WHICH HAS PARAMETERS INSIDE OF IT.
PARAMETERS INSIDE OF IT.
PARAMETERS INSIDE OF IT. LET'S CALL THESE F0 THROUGH
LET'S CALL THESE F0 THROUGH
LET'S CALL THESE F0 THROUGH F127.
F127.
F127. WHAT WE WANT IS THAT WHEN THE
WHAT WE WANT IS THAT WHEN THE
WHAT WE WANT IS THAT WHEN THE PIXELS OF THE SHOES GET FED INTO
PIXELS OF THE SHOES GET FED INTO
PIXELS OF THE SHOES GET FED INTO THEM, 1 BY 1, THAT THE
THEM, 1 BY 1, THAT THE
THEM, 1 BY 1, THAT THE COMBINATION OF ALL OF THESE
COMBINATION OF ALL OF THESE
COMBINATION OF ALL OF THESE FUNCTIONS WILL OUTPUT THE
FUNCTIONS WILL OUTPUT THE
FUNCTIONS WILL OUTPUT THE CORRECT VALUE IN THIS CASE 9.
CORRECT VALUE IN THIS CASE 9.
CORRECT VALUE IN THIS CASE 9. IN ORDER TO DO THAT THE COMPUTER
IN ORDER TO DO THAT THE COMPUTER
IN ORDER TO DO THAT THE COMPUTER WILL NEED TO FIGURE OUT THE
WILL NEED TO FIGURE OUT THE
WILL NEED TO FIGURE OUT THE PARAMETERS INSIDE THESE
PARAMETERS INSIDE THESE
PARAMETERS INSIDE THESE FUNCTIONS TO GET THESE RESULTS.
FUNCTIONS TO GET THESE RESULTS.
FUNCTIONS TO GET THESE RESULTS. IT WILL EXTEND IT TO ALL THE
IT WILL EXTEND IT TO ALL THE
IT WILL EXTEND IT TO ALL THE OTHER ITEMS OF COLORS IN THE --
OTHER ITEMS OF COLORS IN THE --
OTHER ITEMS OF COLORS IN THE -- >> WHAT SOFTMAX DOES IT SETS AT
>> WHAT SOFTMAX DOES IT SETS AT
>> WHAT SOFTMAX DOES IT SETS AT 1 AND THE REST IS ZERO SO ALL WE
1 AND THE REST IS ZERO SO ALL WE
1 AND THE REST IS ZERO SO ALL WE HAVE TO DO IS FIND THE 1.
HAVE TO DO IS FIND THE 1.
HAVE TO DO IS FIND THE 1. TRAINING IS VERY SEARCH.
TRAINING IS VERY SEARCH.
TRAINING IS VERY SEARCH. WE FIT THE IMAGES TO THE LABELS.
WE FIT THE IMAGES TO THE LABELS.
WE FIT THE IMAGES TO THE LABELS. THIS TIME WE'LL TRY IT FOR JUST
THIS TIME WE'LL TRY IT FOR JUST
THIS TIME WE'LL TRY IT FOR JUST 5 ePOX.
5 ePOX.
5 ePOX. REMEMBER EARLIER WE HAD 10,000
REMEMBER EARLIER WE HAD 10,000
REMEMBER EARLIER WE HAD 10,000 IMAGES AND LABELS THAT WE DIDN'T
IMAGES AND LABELS THAT WE DIDN'T
IMAGES AND LABELS THAT WE DIDN'T TRAIN WITH.
TRAIN WITH.
TRAIN WITH. THESE ARE IMAGES THAT THE MODEL
THESE ARE IMAGES THAT THE MODEL
THESE ARE IMAGES THAT THE MODEL HASN'T PREVIOUSLY SEEN SO WE CAN
HASN'T PREVIOUSLY SEEN SO WE CAN
HASN'T PREVIOUSLY SEEN SO WE CAN USE THEM TO TEST HOW WELL OUR
USE THEM TO TEST HOW WELL OUR
USE THEM TO TEST HOW WELL OUR MODEL PERFORMS.
MODEL PERFORMS.
MODEL PERFORMS. WE CAN DO THAT TEST BY PASSING
WE CAN DO THAT TEST BY PASSING
WE CAN DO THAT TEST BY PASSING THEM TO THE EVALUATE METHOD LIKE
THEM TO THE EVALUATE METHOD LIKE
THEM TO THE EVALUATE METHOD LIKE THIS.
THIS.
THIS. AND THEN FINALLY WE CAN GET
AND THEN FINALLY WE CAN GET
AND THEN FINALLY WE CAN GET PRICKLETIONS BACK FROM --
PRICKLETIONS BACK FROM --
PRICKLETIONS BACK FROM -- PREDICTIONS FOR MODEL PREDICT
PREDICTIONS FOR MODEL PREDICT
PREDICTIONS FOR MODEL PREDICT LIKE THIS, AND THAT'S IT TAKES
LIKE THIS, AND THAT'S IT TAKES
LIKE THIS, AND THAT'S IT TAKES TO TEACH A COMPUTER TO SEE AND
TO TEACH A COMPUTER TO SEE AND
TO TEACH A COMPUTER TO SEE AND RECOGNIZE IMAGES.
RECOGNIZE IMAGES.
RECOGNIZE IMAGES. YOU CAN TRY THIS OUT IN THE LINK
YOU CAN TRY THIS OUT IN THE LINK
YOU CAN TRY THIS OUT IN THE LINK THAT I DESCRIBED BEFORE.
THAT I DESCRIBED BEFORE.
THAT I DESCRIBED BEFORE. HAVING GO THROUGH THIS I ♪
HAVING GO THROUGH THIS I ♪
HAVING GO THROUGH THIS I ♪ ♪
♪
♪ ♪
♪
♪ ♪
♪
♪ >> THROUGH HIS TECHNICAL
>> THROUGH HIS TECHNICAL
>> THROUGH HIS TECHNICAL DIFFICULTIES -- THROUGH THOSE
DIFFICULTIES -- THROUGH THOSE
DIFFICULTIES -- THROUGH THOSE TECHNICAL DIFFICULTIES.
TECHNICAL DIFFICULTIES.
TECHNICAL DIFFICULTIES. IT'S A PLEASURE TO BRING UP MY
IT'S A PLEASURE TO BRING UP MY
IT'S A PLEASURE TO BRING UP MY TENSORFLOW FLOW TEAM AT GOOGLE.
TENSORFLOW FLOW TEAM AT GOOGLE.
TENSORFLOW FLOW TEAM AT GOOGLE. THEY WILL TALK ABOUT
THEY WILL TALK ABOUT
THEY WILL TALK ABOUT TENSORFLOW.JS.
TENSORFLOW.JS.
TENSORFLOW.JS. >> THANK YOU.
>> THANK YOU.
>> THANK YOU. HELLO, EVERYONE.
HELLO, EVERYONE.
HELLO, EVERYONE. THANK YOU FOR YOUR PATIENCE.
THANK YOU FOR YOUR PATIENCE.
THANK YOU FOR YOUR PATIENCE. WE ARE SUPER EXCITED TO BE HERE
WE ARE SUPER EXCITED TO BE HERE
WE ARE SUPER EXCITED TO BE HERE AND TALKING TO YOU ABOUT
AND TALKING TO YOU ABOUT
AND TALKING TO YOU ABOUT TENSORFLOW JS AND TO BRING --
TENSORFLOW JS AND TO BRING --
TENSORFLOW JS AND TO BRING -- AND HOW TO BRING HERE'S MY
AND HOW TO BRING HERE'S MY
AND HOW TO BRING HERE'S MY COLLEAGUE WHO'S A SOFTWARE
COLLEAGUE WHO'S A SOFTWARE
COLLEAGUE WHO'S A SOFTWARE ENGINEERING ON THE TENSORFLOW
ENGINEERING ON THE TENSORFLOW
ENGINEERING ON THE TENSORFLOW PROGRAM.
PROGRAM.
PROGRAM. HERE'S AN OUTLINE OF TENSORFLOW
HERE'S AN OUTLINE OF TENSORFLOW
HERE'S AN OUTLINE OF TENSORFLOW JS.
JS.
JS. AND WE WILL DO A CODE SE LEVELS-
AND WE WILL DO A CODE SE LEVELS-
AND WE WILL DO A CODE SE LEVELS- WALK-THROUGH OF THE DEVELOPMENT
WALK-THROUGH OF THE DEVELOPMENT
WALK-THROUGH OF THE DEVELOPMENT BUT TO USE ONE OF THESE
BUT TO USE ONE OF THESE
BUT TO USE ONE OF THESE PRETRAINED MODELS AS WELL AS,
PRETRAINED MODELS AS WELL AS,
PRETRAINED MODELS AS WELL AS, YOU KNOW, AND WE'D LOVE TO SHOW
YOU KNOW, AND WE'D LOVE TO SHOW
YOU KNOW, AND WE'D LOVE TO SHOW YOU SOME EXAMPLES AND FINALLY
YOU SOME EXAMPLES AND FINALLY
YOU SOME EXAMPLES AND FINALLY WE'LL POINT TO SOME RESOURCES
WE'LL POINT TO SOME RESOURCES
WE'LL POINT TO SOME RESOURCES THAT YOU CAN START EXPLORING.
THAT YOU CAN START EXPLORING.
THAT YOU CAN START EXPLORING. ALL RIGHT. SO WE HAVE A LOT OF
ALL RIGHT. SO WE HAVE A LOT OF
ALL RIGHT. SO WE HAVE A LOT OF EXCITING CONTENT TO COVER SO
EXCITING CONTENT TO COVER SO
EXCITING CONTENT TO COVER SO LET'S GET STARTED.
LET'S GET STARTED.
LET'S GET STARTED. YOU MAY HAVE HEARD AT THE -- AT
YOU MAY HAVE HEARD AT THE -- AT
YOU MAY HAVE HEARD AT THE -- AT THE TENSORFLOW JS KEYNOTE AN
THE TENSORFLOW JS KEYNOTE AN
THE TENSORFLOW JS KEYNOTE AN OVERVIEW OF THE TECHNOLOGY IN
OVERVIEW OF THE TECHNOLOGY IN
OVERVIEW OF THE TECHNOLOGY IN THE MORNING WE'RE GOING TO BUILD
THE MORNING WE'RE GOING TO BUILD
THE MORNING WE'RE GOING TO BUILD ON THAT HERE BUT FIRST A QUICK
ON THAT HERE BUT FIRST A QUICK
ON THAT HERE BUT FIRST A QUICK RECAP OF THE BASICS OF
RECAP OF THE BASICS OF
RECAP OF THE BASICS OF TENSORFLOW JS.
TENSORFLOW JS.
TENSORFLOW JS. SO IN A NUTSHELL, TENSORFLOW.JS
SO IN A NUTSHELL, TENSORFLOW.JS
SO IN A NUTSHELL, TENSORFLOW.JS IS MACHINE LEARNING FOR JAVA
IS MACHINE LEARNING FOR JAVA
IS MACHINE LEARNING FOR JAVA SCRIPT.
SCRIPT.
SCRIPT. IT'S BUILT FOR JAVA SCRIPT
IT'S BUILT FOR JAVA SCRIPT
IT'S BUILT FOR JAVA SCRIPT BUILDERS TO CREATE AND RUN
BUILDERS TO CREATE AND RUN
BUILDERS TO CREATE AND RUN MISSION MACHINE LEARNING MODELS
MISSION MACHINE LEARNING MODELS
MISSION MACHINE LEARNING MODELS WITH JAVASCRIPT TRAINING YOU CAN
WITH JAVASCRIPT TRAINING YOU CAN
WITH JAVASCRIPT TRAINING YOU CAN HAVE BROWSER PLATFORMS AND --
HAVE BROWSER PLATFORMS AND --
HAVE BROWSER PLATFORMS AND -- [INAUDIBLE]
[INAUDIBLE]
[INAUDIBLE] >> ML OPERATIONS ARE GPU
>> ML OPERATIONS ARE GPU
>> ML OPERATIONS ARE GPU ACCELERATED AND THE LIBRARY IS
ACCELERATED AND THE LIBRARY IS
ACCELERATED AND THE LIBRARY IS FULLY OPEN SOURCE AND ANYONE IS
FULLY OPEN SOURCE AND ANYONE IS
FULLY OPEN SOURCE AND ANYONE IS WELCOME TO CONTRIBUTE.
WELCOME TO CONTRIBUTE.
WELCOME TO CONTRIBUTE. OKAY.
OKAY.
OKAY. SO TENSORFLOW JS HAS YOU CAN USE
SO TENSORFLOW JS HAS YOU CAN USE
SO TENSORFLOW JS HAS YOU CAN USE OFF THE SHELF MODELS THAT THE
OFF THE SHELF MODELS THAT THE
OFF THE SHELF MODELS THAT THE LIBRARY PROVIDES AND WE'LL SEE
LIBRARY PROVIDES AND WE'LL SEE
LIBRARY PROVIDES AND WE'LL SEE THOSE IN A BIT.
THOSE IN A BIT.
THOSE IN A BIT. WE CAN ALSO USE YOUR PYTHON
WE CAN ALSO USE YOUR PYTHON
WE CAN ALSO USE YOUR PYTHON MODELS WITH OUR WITHOUT PLATFORM
MODELS WITH OUR WITHOUT PLATFORM
MODELS WITH OUR WITHOUT PLATFORM CONVERSION WITH THE PLATFORM
CONVERSION WITH THE PLATFORM
CONVERSION WITH THE PLATFORM YOU'RE USING OR YOU CAN RETRAIN
YOU'RE USING OR YOU CAN RETRAIN
YOU'RE USING OR YOU CAN RETRAIN EXISTING MODEL FOR TRAINING AND
EXISTING MODEL FOR TRAINING AND
EXISTING MODEL FOR TRAINING AND CUSTOMIZING IT TO YOUR DATASETS.
CUSTOMIZING IT TO YOUR DATASETS.
CUSTOMIZING IT TO YOUR DATASETS. THAT'S THE SECOND, YOU KNOW,
THAT'S THE SECOND, YOU KNOW,
THAT'S THE SECOND, YOU KNOW, STARTING POINTED -- AND THE
STARTING POINTED -- AND THE
STARTING POINTED -- AND THE THIRD STARTING MODEL YOU CAN
THIRD STARTING MODEL YOU CAN
THIRD STARTING MODEL YOU CAN BUILD YOUR ENTIRE MODEL WITH A
BUILD YOUR ENTIRE MODEL WITH A
BUILD YOUR ENTIRE MODEL WITH A KE DEFINITE KERAS LASER PLANNING
KE DEFINITE KERAS LASER PLANNING
KE DEFINITE KERAS LASER PLANNING KERAS WITH MANY A PLATFORMS LIKE
KERAS WITH MANY A PLATFORMS LIKE
KERAS WITH MANY A PLATFORMS LIKE WHICH APP WE HAVE JUST ADDED
WHICH APP WE HAVE JUST ADDED
WHICH APP WE HAVE JUST ADDED FIRST CLASS INTEGRATE WITH
FIRST CLASS INTEGRATE WITH
FIRST CLASS INTEGRATE WITH TENSORFLOW JS ON SERVER JS RUNS
TENSORFLOW JS ON SERVER JS RUNS
TENSORFLOW JS ON SERVER JS RUNS ON NOTE AND IN ADDITION IT CAN
ON NOTE AND IN ADDITION IT CAN
ON NOTE AND IN ADDITION IT CAN RUN ON DESKTOP APPLICATIONS
RUN ON DESKTOP APPLICATIONS
RUN ON DESKTOP APPLICATIONS USING THE ELECTRON FRAMEWORK.
USING THE ELECTRON FRAMEWORK.
USING THE ELECTRON FRAMEWORK. SO WHY TENSORFLOW JS?
SO WHY TENSORFLOW JS?
SO WHY TENSORFLOW JS? WHY RUN ML IN THE BROWSER WELL,
WHY RUN ML IN THE BROWSER WELL,
WHY RUN ML IN THE BROWSER WELL, WE WILL BE THERE ARE COMPELLING
WE WILL BE THERE ARE COMPELLING
WE WILL BE THERE ARE COMPELLING REASONS TO RUN ML AUSTIN BROWSER
REASONS TO RUN ML AUSTIN BROWSER
REASONS TO RUN ML AUSTIN BROWSER CLIENT ESPECIALLY FOR MODEL
CLIENT ESPECIALLY FOR MODEL
CLIENT ESPECIALLY FOR MODEL INFERENCING.
INFERENCING.
INFERENCING. LET'S LOOK AT SOME OF THESE
LET'S LOOK AT SOME OF THESE
LET'S LOOK AT SOME OF THESE MODELS.
MODELS.
MODELS. FIRSTLY THERE ARE NO DRIVERS AND
FIRSTLY THERE ARE NO DRIVERS AND
FIRSTLY THERE ARE NO DRIVERS AND NOTHING TO INSTALL.
NOTHING TO INSTALL.
NOTHING TO INSTALL. INCLUDE THE TENSORFLOW JS
INCLUDE THE TENSORFLOW JS
INCLUDE THE TENSORFLOW JS LIBRARY BY SCRIPT SOURCING IT
LIBRARY BY SCRIPT SOURCING IT
LIBRARY BY SCRIPT SOURCING IT INTO YOUR HTML PAGE OR BY
INTO YOUR HTML PAGE OR BY
INTO YOUR HTML PAGE OR BY BUNDLING INTO YOUR PACKAGE
BUNDLING INTO YOUR PACKAGE
BUNDLING INTO YOUR PACKAGE MANAGER IN YOUR CLIENT APP AND
MANAGER IN YOUR CLIENT APP AND
MANAGER IN YOUR CLIENT APP AND YOU'RE GOOD TO GO, AND THAT'S
YOU'RE GOOD TO GO, AND THAT'S
YOU'RE GOOD TO GO, AND THAT'S IT.
IT.
IT. THE -- THE SECOND ADVANTAGE IS
THE -- THE SECOND ADVANTAGE IS
THE -- THE SECOND ADVANTAGE IS YOU CAN UTILIZE A 1R5IVARIETY O
YOU CAN UTILIZE A 1R5IVARIETY O
YOU CAN UTILIZE A 1R5IVARIETY O VARIETY OF INPUTS SUCH AS
VARIETY OF INPUTS SUCH AS
VARIETY OF INPUTS SUCH AS CAMERA, MICROPHONE, GPS THROUGH
CAMERA, MICROPHONE, GPS THROUGH
CAMERA, MICROPHONE, GPS THROUGH STANDARD API HTML WEB API AND A
STANDARD API HTML WEB API AND A
STANDARD API HTML WEB API AND A SIMPLIFIED DATA API AND WE'RE
SIMPLIFIED DATA API AND WE'RE
SIMPLIFIED DATA API AND WE'RE GOING TO SEE SOME EXAMPLES
GOING TO SEE SOME EXAMPLES
GOING TO SEE SOME EXAMPLES TODAY.
TODAY.
TODAY. WHICH IS A GREAT CHOICE FOR
WHICH IS A GREAT CHOICE FOR
WHICH IS A GREAT CHOICE FOR PRIVACY SENSITIVE APPLICATIONS.
PRIVACY SENSITIVE APPLICATIONS.
PRIVACY SENSITIVE APPLICATIONS. AVOIDS ROUND TRIP LATENCY TO THE
AVOIDS ROUND TRIP LATENCY TO THE
AVOIDS ROUND TRIP LATENCY TO THE SERVER.
SERVER.
SERVER. IT'S ACCELERATED THESE FACTORS
IT'S ACCELERATED THESE FACTORS
IT'S ACCELERATED THESE FACTORS IN COMBINATION, YOU
IN COMBINATION, YOU
IN COMBINATION, YOU KNOW -- COMBINE TO MAKE FOR A
KNOW -- COMBINE TO MAKE FOR A
KNOW -- COMBINE TO MAKE FOR A MORE FLUID AND INTERACTIVE USER
MORE FLUID AND INTERACTIVE USER
MORE FLUID AND INTERACTIVE USER EXPERIENCE.
EXPERIENCE.
EXPERIENCE. ALSO RUNNING ML ON THE CLIENT
ALSO RUNNING ML ON THE CLIENT
ALSO RUNNING ML ON THE CLIENT HELPS REDUCE SERVICE SIDE COSTS
HELPS REDUCE SERVICE SIDE COSTS
HELPS REDUCE SERVICE SIDE COSTS AND SERVICE YOUR INFRASTRUCTURE,
AND SERVICE YOUR INFRASTRUCTURE,
AND SERVICE YOUR INFRASTRUCTURE, FOR EXAMPLE, NO ONLINE ML
FOR EXAMPLE, NO ONLINE ML
FOR EXAMPLE, NO ONLINE ML SERVING THAT NEEDS TO SCALE TO
SERVING THAT NEEDS TO SCALE TO
SERVING THAT NEEDS TO SCALE TO INCREASING TRAFFIC AND SO FORTH
INCREASING TRAFFIC AND SO FORTH
INCREASING TRAFFIC AND SO FORTH IS NEEDED BECAUSE YOU'RE
IS NEEDED BECAUSE YOU'RE
IS NEEDED BECAUSE YOU'RE OFFLOADING ALL YOUR COMPUTING TO
OFFLOADING ALL YOUR COMPUTING TO
OFFLOADING ALL YOUR COMPUTING TO THE CLIENT YOU HOST AN ML MODEL
THE CLIENT YOU HOST AN ML MODEL
THE CLIENT YOU HOST AN ML MODEL FROM A STATIC FILE LOCATION AND
FROM A STATIC FILE LOCATION AND
FROM A STATIC FILE LOCATION AND THAT'S IT.
THAT'S IT.
THAT'S IT. ON THE SERVER THERE ARE ALSO
ON THE SERVER THERE ARE ALSO
ON THE SERVER THERE ARE ALSO BENEFITS TO INTEGRATING
BENEFITS TO INTEGRATING
BENEFITS TO INTEGRATING TENSORFLOW JS INTO YOUR NORD JS
TENSORFLOW JS INTO YOUR NORD JS
TENSORFLOW JS INTO YOUR NORD JS ENVIRONMENT.
ENVIRONMENT.
ENVIRONMENT. IF YOU ARE USING A NODE JS
IF YOU ARE USING A NODE JS
IF YOU ARE USING A NODE JS SERVICE STACK IT ALLOWS YOU TO
SERVICE STACK IT ALLOWS YOU TO
SERVICE STACK IT ALLOWS YOU TO GO IN A STACK INSTEAD OF A
GO IN A STACK INSTEAD OF A
GO IN A STACK INSTEAD OF A PYTHON-BASED STACK.
PYTHON-BASED STACK.
PYTHON-BASED STACK. YOU CAN USE YOUR CORE INTENSIVE
YOU CAN USE YOUR CORE INTENSIVE
YOU CAN USE YOUR CORE INTENSIVE MODELS TO BRING THEM INTO JS NOT
MODELS TO BRING THEM INTO JS NOT
MODELS TO BRING THEM INTO JS NOT JUST THE PREBUILT MODELS YOUR
JUST THE PREBUILT MODELS YOUR
JUST THE PREBUILT MODELS YOUR CUSTOM MODELS BUILT WITH PYTHON
CUSTOM MODELS BUILT WITH PYTHON
CUSTOM MODELS BUILT WITH PYTHON TENSORFLOW CAN BE CONVERTED AND
TENSORFLOW CAN BE CONVERTED AND
TENSORFLOW CAN BE CONVERTED AND IN AN UPCOMING RELEASE YOU DON'T
IN AN UPCOMING RELEASE YOU DON'T
IN AN UPCOMING RELEASE YOU DON'T NEED THE CONVERSION PROCESS YOU
NEED THE CONVERSION PROCESS YOU
NEED THE CONVERSION PROCESS YOU CAN JUST USE THEM DIRECTLY IN
CAN JUST USE THEM DIRECTLY IN
CAN JUST USE THEM DIRECTLY IN NODE, AND FINALLY YOU CAN DO ALL
NODE, AND FINALLY YOU CAN DO ALL
NODE, AND FINALLY YOU CAN DO ALL OF THIS WITHOUT SACRIFICING
OF THIS WITHOUT SACRIFICING
OF THIS WITHOUT SACRIFICING PERFORMANCE.
PERFORMANCE.
PERFORMANCE. YOU GET CPU AND GPU ACCELERATION
YOU GET CPU AND GPU ACCELERATION
YOU GET CPU AND GPU ACCELERATION WITH THE UNDERLYING TENSORFLOW C
WITH THE UNDERLYING TENSORFLOW C
WITH THE UNDERLYING TENSORFLOW C LIBRARY BECAUSE THAT'S WHAT NODE
LIBRARY BECAUSE THAT'S WHAT NODE
LIBRARY BECAUSE THAT'S WHAT NODE USES AND WE'RE ALSO WORKING ON
USES AND WE'RE ALSO WORKING ON
USES AND WE'RE ALSO WORKING ON GPU ACCELERATION VIA OPEN GL SO
GPU ACCELERATION VIA OPEN GL SO
GPU ACCELERATION VIA OPEN GL SO THAT REMOVES THE NEED DEPENDING
THAT REMOVES THE NEED DEPENDING
THAT REMOVES THE NEED DEPENDING ON CORD DRIVERS AS WELL SO
ON CORD DRIVERS AS WELL SO
ON CORD DRIVERS AS WELL SO EFFECTIVELY, YOU GET PERFORMANCE
EFFECTIVELY, YOU GET PERFORMANCE
EFFECTIVELY, YOU GET PERFORMANCE THAT'S SIMILAR TO THE PYTHON
THAT'S SIMILAR TO THE PYTHON
THAT'S SIMILAR TO THE PYTHON LIBRARY.
LIBRARY.
LIBRARY. SO -- SO THESE ATTRIBUTES OF
SO -- SO THESE ATTRIBUTES OF
SO -- SO THESE ATTRIBUTES OF THE LIBRARY ENABLE A VARIETY OF
THE LIBRARY ENABLE A VARIETY OF
THE LIBRARY ENABLE A VARIETY OF USE CASES ACROSS THE CLIENT
USE CASES ACROSS THE CLIENT
USE CASES ACROSS THE CLIENT SERVER SPECTRUM SO LET'S TAKE A
SERVER SPECTRUM SO LET'S TAKE A
SERVER SPECTRUM SO LET'S TAKE A LOOK AT SOME OF THOSE ON THE
LOOK AT SOME OF THOSE ON THE
LOOK AT SOME OF THOSE ON THE CLIENT SIDE IT ENABLES YOU TO
CLIENT SIDE IT ENABLES YOU TO
CLIENT SIDE IT ENABLES YOU TO BUILD FEATURES THAT NEED HIGHER
BUILD FEATURES THAT NEED HIGHER
BUILD FEATURES THAT NEED HIGHER END ACTIVITY AUGMENTING HIGHER
END ACTIVITY AUGMENTING HIGHER
END ACTIVITY AUGMENTING HIGHER INTERACTIONS, SPEECH RECOGNITION
INTERACTIONS, SPEECH RECOGNITION
INTERACTIONS, SPEECH RECOGNITION AND ACCESSIBILITY AND SO FORTH
AND ACCESSIBILITY AND SO FORTH
AND ACCESSIBILITY AND SO FORTH ON THE SERVER SIDE OF THE
ON THE SERVER SIDE OF THE
ON THE SERVER SIDE OF THE SPECTRUM IT LETS YOU, YOU KNOW,
SPECTRUM IT LETS YOU, YOU KNOW,
SPECTRUM IT LETS YOU, YOU KNOW, HAVE YOUR MORE TRADITIONAL ML
HAVE YOUR MORE TRADITIONAL ML
HAVE YOUR MORE TRADITIONAL ML PIPELINES THAT SOLVE ENTERPRISE
PIPELINES THAT SOLVE ENTERPRISE
PIPELINES THAT SOLVE ENTERPRISE LIKE USE CASES AND IN THE MIDDLE
LIKE USE CASES AND IN THE MIDDLE
LIKE USE CASES AND IN THE MIDDLE THAT CAN LIVE EITHER ON THE
THAT CAN LIVE EITHER ON THE
THAT CAN LIVE EITHER ON THE SILVER ON THE CLIENT
SILVER ON THE CLIENT
SILVER ON THE CLIENT APPLICATIONS THAT THAT DO
APPLICATIONS THAT THAT DO
APPLICATIONS THAT THAT DO SENTIMENT ANALYSIS,
SENTIMENT ANALYSIS,
SENTIMENT ANALYSIS, CONVERSATIONAL AI, ML ASSISTED
CONVERSATIONAL AI, ML ASSISTED
CONVERSATIONAL AI, ML ASSISTED CONTENT ASSISTANCE AND SO FORTH
CONTENT ASSISTANCE AND SO FORTH
CONTENT ASSISTANCE AND SO FORTH SO YOU GET THE FLEXIBILITY OF
SO YOU GET THE FLEXIBILITY OF
SO YOU GET THE FLEXIBILITY OF CHOOSING WHERE YOU WANT YOUR ML
CHOOSING WHERE YOU WANT YOUR ML
CHOOSING WHERE YOU WANT YOUR ML TO RUN ON THE CLIENT ON THE
TO RUN ON THE CLIENT ON THE
TO RUN ON THE CLIENT ON THE SILVER EITHER/OR.
SILVER EITHER/OR.
SILVER EITHER/OR. SO WHAT ABOUT YOUR USE CASE --
SO WHAT ABOUT YOUR USE CASE --
SO WHAT ABOUT YOUR USE CASE -- WHATEVER YOUR USE CASE IS
WHATEVER YOUR USE CASE IS
WHATEVER YOUR USE CASE IS TENSORFLOW JS IS READY TO BE
TENSORFLOW JS IS READY TO BE
TENSORFLOW JS IS READY TO BE LEVERAGED.
LEVERAGED.
LEVERAGED. SO WITH THAT INTRO I'D LIKE TO
SO WITH THAT INTRO I'D LIKE TO
SO WITH THAT INTRO I'D LIKE TO DELVE DEEPER INTO THE READY TO
DELVE DEEPER INTO THE READY TO
DELVE DEEPER INTO THE READY TO USE MODELS AVAILABLE IN
USE MODELS AVAILABLE IN
USE MODELS AVAILABLE IN TENSORFLOW JS.
TENSORFLOW JS.
TENSORFLOW JS. OUR COLLECTION OF MODELS HAVE
OUR COLLECTION OF MODELS HAVE
OUR COLLECTION OF MODELS HAVE GROWN AND IS GROWING TO ADDRESS
GROWN AND IS GROWING TO ADDRESS
GROWN AND IS GROWING TO ADDRESS THE USE CASES THAT WE JUST
THE USE CASES THAT WE JUST
THE USE CASES THAT WE JUST MENTIONED NAMELY IMAGE
MENTIONED NAMELY IMAGE
MENTIONED NAMELY IMAGE CLASSIFICATION FOR CLASSIFYING
CLASSIFICATION FOR CLASSIFYING
CLASSIFICATION FOR CLASSIFYING WHOLE IMAGES, DETECTING
WHOLE IMAGES, DETECTING
WHOLE IMAGES, DETECTING SEGMENTING OBJECTS AND OBJECT
SEGMENTING OBJECTS AND OBJECT
SEGMENTING OBJECTS AND OBJECT BOUNDARIES DETECTING THE HUMAN
BOUNDARIES DETECTING THE HUMAN
BOUNDARIES DETECTING THE HUMAN BODY AND ESTIMATING POSE
BODY AND ESTIMATING POSE
BODY AND ESTIMATING POSE RECOGNIZING SPEECH COMMANDS AND
RECOGNIZING SPEECH COMMANDS AND
RECOGNIZING SPEECH COMMANDS AND COMMON WORDS FROM AUDIO DATA AND
COMMON WORDS FROM AUDIO DATA AND
COMMON WORDS FROM AUDIO DATA AND TEXT MODELS FOR TEXT
TEXT MODELS FOR TEXT
TEXT MODELS FOR TEXT CLASSIFICATION DEDUCTION.
CLASSIFICATION DEDUCTION.
CLASSIFICATION DEDUCTION. YOU CAN EXPLORE ALL OF THESE
YOU CAN EXPLORE ALL OF THESE
YOU CAN EXPLORE ALL OF THESE MODELS TODAY ON GITHUB.
MODELS TODAY ON GITHUB.
MODELS TODAY ON GITHUB. YOU CAN USE THEM BY INSTALLING
YOU CAN USE THEM BY INSTALLING
YOU CAN USE THEM BY INSTALLING THEM WITH MPM OR BY DIRECTLY
THEM WITH MPM OR BY DIRECTLY
THEM WITH MPM OR BY DIRECTLY INCLUDING FROM OUR HOLSTERED
INCLUDING FROM OUR HOLSTERED
INCLUDING FROM OUR HOLSTERED SCRIPTS.
SCRIPTS.
SCRIPTS. LET'S DIVE INTO SOME OF THESE
LET'S DIVE INTO SOME OF THESE
LET'S DIVE INTO SOME OF THESE MODELS AND GET MORE DETAILS OR
MODELS AND GET MORE DETAILS OR
MODELS AND GET MORE DETAILS OR THESE.
THESE.
THESE. SO THIS IS THE POST NET MODEL IT
SO THIS IS THE POST NET MODEL IT
SO THIS IS THE POST NET MODEL IT DETECTS 17 LANDMARK POINTS ON
DETECTS 17 LANDMARK POINTS ON
DETECTS 17 LANDMARK POINTS ON THE HUMAN BODY.
THE HUMAN BODY.
THE HUMAN BODY. IT SUPPORTS BOTH SINGLE PERSON
IT SUPPORTS BOTH SINGLE PERSON
IT SUPPORTS BOTH SINGLE PERSON AND MULTIPERSON DETECTION, YOU
AND MULTIPERSON DETECTION, YOU
AND MULTIPERSON DETECTION, YOU KNOW, WITH AN IMAGE.
KNOW, WITH AN IMAGE.
KNOW, WITH AN IMAGE. NOW, THERE ARE MULTIPLE VERSIONS
NOW, THERE ARE MULTIPLE VERSIONS
NOW, THERE ARE MULTIPLE VERSIONS OF THIS MODEL, VERSIONS THAT ARE
OF THIS MODEL, VERSIONS THAT ARE
OF THIS MODEL, VERSIONS THAT ARE BACKED BY MOBILE NET AND THOSE
BACKED BY MOBILE NET AND THOSE
BACKED BY MOBILE NET AND THOSE BACKED BY RES.NET THAT PROVIDE
BACKED BY RES.NET THAT PROVIDE
BACKED BY RES.NET THAT PROVIDE OPTIONS FOR BALANCING ACCURACY
OPTIONS FOR BALANCING ACCURACY
OPTIONS FOR BALANCING ACCURACY VERSUS MODEL SIZE VERSUS LATENCY
VERSUS MODEL SIZE VERSUS LATENCY
VERSUS MODEL SIZE VERSUS LATENCY DEPENDING ON YOUR NEEDS AND IT
DEPENDING ON YOUR NEEDS AND IT
DEPENDING ON YOUR NEEDS AND IT ENABLES USE CASES LIKE GESTURE
ENABLES USE CASES LIKE GESTURE
ENABLES USE CASES LIKE GESTURE BASE OR TRACTION AUGMENTED
BASE OR TRACTION AUGMENTED
BASE OR TRACTION AUGMENTED REALITY, ANIMATION AND SO ON,
REALITY, ANIMATION AND SO ON,
REALITY, ANIMATION AND SO ON, THINGS THAT, YOU KNOW -- THAT
THINGS THAT, YOU KNOW -- THAT
THINGS THAT, YOU KNOW -- THAT ARE RUNNING ML ON THE CLIENT BUT
ARE RUNNING ML ON THE CLIENT BUT
ARE RUNNING ML ON THE CLIENT BUT YOU CAN EXPLORE A DEMO OF THIS
YOU CAN EXPLORE A DEMO OF THIS
YOU CAN EXPLORE A DEMO OF THIS PARTICULAR MODEL ON BOTH THE
PARTICULAR MODEL ON BOTH THE
PARTICULAR MODEL ON BOTH THE EXPO HALL.
EXPO HALL.
EXPO HALL. ANOTHER HUMAN MODEL IS THE BODY
ANOTHER HUMAN MODEL IS THE BODY
ANOTHER HUMAN MODEL IS THE BODY PICK MODEL WHICH ENABLES PERSON
PICK MODEL WHICH ENABLES PERSON
PICK MODEL WHICH ENABLES PERSON SEGMENTATION AGAIN BOTH SINGLE
SEGMENTATION AGAIN BOTH SINGLE
SEGMENTATION AGAIN BOTH SINGLE AND MULTIPLE PERSONS IMAGE.
AND MULTIPLE PERSONS IMAGE.
AND MULTIPLE PERSONS IMAGE. IT IDENTIFIES BODY PARTS SUCH AS
IT IDENTIFIES BODY PARTS SUCH AS
IT IDENTIFIES BODY PARTS SUCH AS LEFT ARM, RIGHT ARM, TORSO, LEFT
LEFT ARM, RIGHT ARM, TORSO, LEFT
LEFT ARM, RIGHT ARM, TORSO, LEFT RIGHT LEDS AND SO FORTH IT ALSO
RIGHT LEDS AND SO FORTH IT ALSO
RIGHT LEDS AND SO FORTH IT ALSO PROVIDES CONVENIENCE API TO
PROVIDES CONVENIENCE API TO
PROVIDES CONVENIENCE API TO SEGMENT AND ASK EACH BODY PART
SEGMENT AND ASK EACH BODY PART
SEGMENT AND ASK EACH BODY PART IN A DIFFERENT COLOR WHICH IS
IN A DIFFERENT COLOR WHICH IS
IN A DIFFERENT COLOR WHICH IS WHAT YOU'RE SEEING IN THIS
WHAT YOU'RE SEEING IN THIS
WHAT YOU'RE SEEING IN THIS PARTICULAR GIF AND IN TERMS OF
PARTICULAR GIF AND IN TERMS OF
PARTICULAR GIF AND IN TERMS OF USE CASES YOU CAN BE USED FOR
USE CASES YOU CAN BE USED FOR
USE CASES YOU CAN BE USED FOR THINGS LIKE BLURRING FACES IN AN
THINGS LIKE BLURRING FACES IN AN
THINGS LIKE BLURRING FACES IN AN IMAGE OR BLURRED THE BACKGROUND
IMAGE OR BLURRED THE BACKGROUND
IMAGE OR BLURRED THE BACKGROUND TO PROTECT PRIVACY.
TO PROTECT PRIVACY.
TO PROTECT PRIVACY. THE SSID DETECTION MODEL ENABLES
THE SSID DETECTION MODEL ENABLES
THE SSID DETECTION MODEL ENABLES DETECTION OF MULTIPLE OBJECTS IN
DETECTION OF MULTIPLE OBJECTS IN
DETECTION OF MULTIPLE OBJECTS IN AN IMAGE.
AN IMAGE.
AN IMAGE. IT DETECTS ABOUT 90 CLASSES TO
IT DETECTS ABOUT 90 CLASSES TO
IT DETECTS ABOUT 90 CLASSES TO FIND IN THE DATASET AND THE NICE
FIND IN THE DATASET AND THE NICE
FIND IN THE DATASET AND THE NICE THING IS THAT IT TAKES THIS
THING IS THAT IT TAKES THIS
THING IS THAT IT TAKES THIS INPUT -- ANY BROWSER-BASED --
INPUT -- ANY BROWSER-BASED --
INPUT -- ANY BROWSER-BASED -- LIKE AN IMAGE OR VIDEO OR A
LIKE AN IMAGE OR VIDEO OR A
LIKE AN IMAGE OR VIDEO OR A CANVAS ELEMENT AND RETURNS A
CANVAS ELEMENT AND RETURNS A
CANVAS ELEMENT AND RETURNS A SCENARIO WITH THE DETECTOR CLASS
SCENARIO WITH THE DETECTOR CLASS
SCENARIO WITH THE DETECTOR CLASS AND THE ADVANCED LEVEL.
AND THE ADVANCED LEVEL.
AND THE ADVANCED LEVEL. IN THIS SAMPLE IMAGE YOU CAN SEE
IN THIS SAMPLE IMAGE YOU CAN SEE
IN THIS SAMPLE IMAGE YOU CAN SEE IT'S DETECTED WITH A HIGH
IT'S DETECTED WITH A HIGH
IT'S DETECTED WITH A HIGH CONFIDENCE SCORE AND YOU GET A
CONFIDENCE SCORE AND YOU GET A
CONFIDENCE SCORE AND YOU GET A BOUNDING BOX.
BOUNDING BOX.
BOUNDING BOX. THERE'S ALSO A DEEP LAB MODEL
THERE'S ALSO A DEEP LAB MODEL
THERE'S ALSO A DEEP LAB MODEL WHICH HAS SEMANICS SEGMENTATION
WHICH HAS SEMANICS SEGMENTATION
WHICH HAS SEMANICS SEGMENTATION THAT WILL BE COMING OUT SOON.
THAT WILL BE COMING OUT SOON.
THAT WILL BE COMING OUT SOON. WHAT I WOULD LIKE TO SHOW YOU
WHAT I WOULD LIKE TO SHOW YOU
WHAT I WOULD LIKE TO SHOW YOU HERE IS HOW EASY IT IS TO USE A
HERE IS HOW EASY IT IS TO USE A
HERE IS HOW EASY IT IS TO USE A MODEL LIKE COCOA SSD IN JAVA
MODEL LIKE COCOA SSD IN JAVA
MODEL LIKE COCOA SSD IN JAVA APPS AND YOU CAN MPM INSTALL AND
APPS AND YOU CAN MPM INSTALL AND
APPS AND YOU CAN MPM INSTALL AND USE A WEBPACK TO BUNDLE INTO A
USE A WEBPACK TO BUNDLE INTO A
USE A WEBPACK TO BUNDLE INTO A CLIENT FILE AND THEN YOU LOAD
CLIENT FILE AND THEN YOU LOAD
CLIENT FILE AND THEN YOU LOAD THE MODEL THAT'S THE DOT LOAD
THE MODEL THAT'S THE DOT LOAD
THE MODEL THAT'S THE DOT LOAD METHOD AND YOU -- AND YOU CALL
METHOD AND YOU -- AND YOU CALL
METHOD AND YOU -- AND YOU CALL THE MODEL METHOD ON -- ON THE
THE MODEL METHOD ON -- ON THE
THE MODEL METHOD ON -- ON THE IMAGE ELEMENT THAT YOU ARE
IMAGE ELEMENT THAT YOU ARE
IMAGE ELEMENT THAT YOU ARE TRYING TO ANALYZE AND THAT'S IT.
TRYING TO ANALYZE AND THAT'S IT.
TRYING TO ANALYZE AND THAT'S IT. WHAT YOU GET BACK IS A -- IS AN
WHAT YOU GET BACK IS A -- IS AN
WHAT YOU GET BACK IS A -- IS AN AREA OF OBJECTS THAT THAT THE
AREA OF OBJECTS THAT THAT THE
AREA OF OBJECTS THAT THAT THE BOUNDING BOXES AND THE CLASSES
BOUNDING BOXES AND THE CLASSES
BOUNDING BOXES AND THE CLASSES THAT ARE DETECTED SO REALLY 4 OR
THAT ARE DETECTED SO REALLY 4 OR
THAT ARE DETECTED SO REALLY 4 OR 5 LINES OF CODE TO END UP
5 LINES OF CODE TO END UP
5 LINES OF CODE TO END UP LEVERAGING A POWERFUL ML MODEL
LEVERAGING A POWERFUL ML MODEL
LEVERAGING A POWERFUL ML MODEL IN THE BROWSER.
IN THE BROWSER.
IN THE BROWSER. TEXT MODELS ARE ANOTHER USEFUL
TEXT MODELS ARE ANOTHER USEFUL
TEXT MODELS ARE ANOTHER USEFUL SET.
SET.
SET. MORE USER GENERATED SENSING D A-
MORE USER GENERATED SENSING D A-
MORE USER GENERATED SENSING D A- CODER MODEL AND LET'S USE THIS A
CODER MODEL AND LET'S USE THIS A
CODER MODEL AND LET'S USE THIS A LITTLE CLOSER.
LITTLE CLOSER.
LITTLE CLOSER. I WANT TO SHOW YOU A LIVE
I WANT TO SHOW YOU A LIVE
I WANT TO SHOW YOU A LIVE EXAMPLE OF THIS MODEL IN A WEB
EXAMPLE OF THIS MODEL IN A WEB
EXAMPLE OF THIS MODEL IN A WEB APP GIVE ME A SECOND.
APP GIVE ME A SECOND.
APP GIVE ME A SECOND. OKAY.
OKAY.
OKAY. THERE YOU GO.
THERE YOU GO.
THERE YOU GO. SO HERE IS A SUPER SIMPLE WEB
SO HERE IS A SUPER SIMPLE WEB
SO HERE IS A SUPER SIMPLE WEB APP THAT SIMPLY LOADS THE MODEL
APP THAT SIMPLY LOADS THE MODEL
APP THAT SIMPLY LOADS THE MODEL AND, YOU KNOW, PASSES A FEW
AND, YOU KNOW, PASSES A FEW
AND, YOU KNOW, PASSES A FEW SENTENCES TO IT AND WHAT THE
SENTENCES TO IT AND WHAT THE
SENTENCES TO IT AND WHAT THE MODEL DOES IS CLASSIFIED ON A
MODEL DOES IS CLASSIFIED ON A
MODEL DOES IS CLASSIFIED ON A FEW DIMENSIONS LIKE -- DOES IT
FEW DIMENSIONS LIKE -- DOES IT
FEW DIMENSIONS LIKE -- DOES IT SIGNIFY AN INSULT.
SIGNIFY AN INSULT.
SIGNIFY AN INSULT. IS THERE A TOXICITY OR -- SO
IS THERE A TOXICITY OR -- SO
IS THERE A TOXICITY OR -- SO LET'S TRY AN EXAMPLE HERE.
LET'S TRY AN EXAMPLE HERE.
LET'S TRY AN EXAMPLE HERE. SO I'M GOING TO SAY SOMETHING
SO I'M GOING TO SAY SOMETHING
SO I'M GOING TO SAY SOMETHING TOXIC.
TOXIC.
TOXIC. TYPING TYPING OKAY.
TYPING TYPING OKAY.
TYPING TYPING OKAY. SO SOMETHING THAT I WOULD THINK
SO SOMETHING THAT I WOULD THINK
SO SOMETHING THAT I WOULD THINK IS TOXIC SO THAT RETURNS AS
IS TOXIC SO THAT RETURNS AS
IS TOXIC SO THAT RETURNS AS TOXIC AS WELL AS REDUNDANCE AND
TOXIC AS WELL AS REDUNDANCE AND
TOXIC AS WELL AS REDUNDANCE AND INSULT HOWEVER THIS MODEL I WANT
INSULT HOWEVER THIS MODEL I WANT
INSULT HOWEVER THIS MODEL I WANT TO SHOW YOU THIS IS
TO SHOW YOU THIS IS
TO SHOW YOU THIS IS CONTEXT-BASED AND NOT KEYWORD
CONTEXT-BASED AND NOT KEYWORD
CONTEXT-BASED AND NOT KEYWORD BASED SO, YOU KNOW, YOU TYPE THE
BASED SO, YOU KNOW, YOU TYPE THE
BASED SO, YOU KNOW, YOU TYPE THE SAME WORD IN A -- IN A TOTALLY
SAME WORD IN A -- IN A TOTALLY
SAME WORD IN A -- IN A TOTALLY NONTOXIC CONCEPT -- CONTEXT
NONTOXIC CONCEPT -- CONTEXT
NONTOXIC CONCEPT -- CONTEXT YOU'RE GOING TO SEE A DIFFERENT
YOU'RE GOING TO SEE A DIFFERENT
YOU'RE GOING TO SEE A DIFFERENT ANSWER.
ANSWER.
ANSWER. AND THAT'S DETECTED AS, YOU
AND THAT'S DETECTED AS, YOU
AND THAT'S DETECTED AS, YOU KNOW, NOT AN INSULT SO THIS SORT
KNOW, NOT AN INSULT SO THIS SORT
KNOW, NOT AN INSULT SO THIS SORT OF MODEL CAN BE USED IN -- ON
OF MODEL CAN BE USED IN -- ON
OF MODEL CAN BE USED IN -- ON THE CLIENT SIDE IN PRODUCT
THE CLIENT SIDE IN PRODUCT
THE CLIENT SIDE IN PRODUCT REVIEWS OR IN CHAT TYPE OF
REVIEWS OR IN CHAT TYPE OF
REVIEWS OR IN CHAT TYPE OF HERE'S AN EXAMPLE I WANT TO SHOW
HERE'S AN EXAMPLE I WANT TO SHOW
HERE'S AN EXAMPLE I WANT TO SHOW YOU FROM A DEVELOPER WHO
YOU FROM A DEVELOPER WHO
YOU FROM A DEVELOPER WHO INTEGRATED TENSORFLOW JS INTO A
INTEGRATED TENSORFLOW JS INTO A
INTEGRATED TENSORFLOW JS INTO A CHAT APP AND, AGAIN, HERE IT'S
CHAT APP AND, AGAIN, HERE IT'S
CHAT APP AND, AGAIN, HERE IT'S ABLE TO DETECT THAT, YOU KNOW,
ABLE TO DETECT THAT, YOU KNOW,
ABLE TO DETECT THAT, YOU KNOW, TOXIC COMMENTS AND TWITTERED
TOXIC COMMENTS AND TWITTERED
TOXIC COMMENTS AND TWITTERED THEM RIGHT BEFORE SENDING.
THEM RIGHT BEFORE SENDING.
THEM RIGHT BEFORE SENDING. ALL RIGHT.
ALL RIGHT.
ALL RIGHT. SLIDE.
SLIDE.
SLIDE. ANOTHER INTERESTING MODEL IS
ANOTHER INTERESTING MODEL IS
ANOTHER INTERESTING MODEL IS FACE MESH IT TRACKING FACIAL
FACE MESH IT TRACKING FACIAL
FACE MESH IT TRACKING FACIAL FEATURES IT TRACKS 400 POINTS ON
FEATURES IT TRACKS 400 POINTS ON
FEATURES IT TRACKS 400 POINTS ON A APPEARANCE FACE SO WE BELIEVE
A APPEARANCE FACE SO WE BELIEVE
A APPEARANCE FACE SO WE BELIEVE THIS MODEL HAS GREAT POTENTIAL
THIS MODEL HAS GREAT POTENTIAL
THIS MODEL HAS GREAT POTENTIAL FOR REAL WORLD APPLICATIONS, FOR
FOR REAL WORLD APPLICATIONS, FOR
FOR REAL WORLD APPLICATIONS, FOR EXAMPLE, DETECTING FACIAL JES
EXAMPLE, DETECTING FACIAL JES
EXAMPLE, DETECTING FACIAL JES TOURS, EMOTION, RESEARCHING
TOURS, EMOTION, RESEARCHING
TOURS, EMOTION, RESEARCHING GREATER ACCESSIBILITY FEATURES
GREATER ACCESSIBILITY FEATURES
GREATER ACCESSIBILITY FEATURES AND SO FORTH SO NOW I WOULD LIKE
AND SO FORTH SO NOW I WOULD LIKE
AND SO FORTH SO NOW I WOULD LIKE TO SHOW YOU A COUPLE OF COOL
TO SHOW YOU A COUPLE OF COOL
TO SHOW YOU A COUPLE OF COOL DEMOS BOTH USING FACE MEASURE.
DEMOS BOTH USING FACE MEASURE.
DEMOS BOTH USING FACE MEASURE. FIRST UP -- AND YOU MIGHT HAVE
FIRST UP -- AND YOU MIGHT HAVE
FIRST UP -- AND YOU MIGHT HAVE SEEN THIS IN THE KEYNOTE SESSION
SEEN THIS IN THE KEYNOTE SESSION
SEEN THIS IN THE KEYNOTE SESSION IF YOU ATTENDED HERE'S AN
IF YOU ATTENDED HERE'S AN
IF YOU ATTENDED HERE'S AN APPLICATION BUILT BY ONE OF OUR
APPLICATION BUILT BY ONE OF OUR
APPLICATION BUILT BY ONE OF OUR PARTNER TEAMS AT GOOGLE.
PARTNER TEAMS AT GOOGLE.
PARTNER TEAMS AT GOOGLE. THIS APP IS CALLED LIP SYNC
THIS APP IS CALLED LIP SYNC
THIS APP IS CALLED LIP SYNC SINK.
SINK.
SINK. THIS IS A GAME THAT TRACKS HOW
THIS IS A GAME THAT TRACKS HOW
THIS IS A GAME THAT TRACKS HOW WELL YOU'RE LIP SYNC SINKING TOO
WELL YOU'RE LIP SYNC SINKING TOO
WELL YOU'RE LIP SYNC SINKING TOO SONG WHILE IN THE BROWSER.
SONG WHILE IN THE BROWSER.
SONG WHILE IN THE BROWSER. NOTICE IN PARTICULAR HOW THE
NOTICE IN PARTICULAR HOW THE
NOTICE IN PARTICULAR HOW THE DISPLAY CREATES WHEN THE LIP
DISPLAY CREATES WHEN THE LIP
DISPLAY CREATES WHEN THE LIP SYNC SINK DOESN'T MATCH AND HOW
SYNC SINK DOESN'T MATCH AND HOW
SYNC SINK DOESN'T MATCH AND HOW THE SCORE MATCHES HOW WELL THE
THE SCORE MATCHES HOW WELL THE
THE SCORE MATCHES HOW WELL THE PERSON IS LIP SYNC SIN.
PERSON IS LIP SYNC SIN.
PERSON IS LIP SYNC SIN. THERE YOU GO.
THERE YOU GO.
THERE YOU GO. I'M GOING TO USE MY MIC -- MOVE
I'M GOING TO USE MY MIC -- MOVE
I'M GOING TO USE MY MIC -- MOVE MY MIC HERE.
MY MIC HERE.
MY MIC HERE. OKAY.
OKAY.
OKAY. THERE'S NO AUDIO HERE.
THERE'S NO AUDIO HERE.
THERE'S NO AUDIO HERE. WE DON'T HAVE AUDIO SO YOU MIGHT
WE DON'T HAVE AUDIO SO YOU MIGHT
WE DON'T HAVE AUDIO SO YOU MIGHT IMAGINE IF YOU RECALL SO WHEN
IMAGINE IF YOU RECALL SO WHEN
IMAGINE IF YOU RECALL SO WHEN THE LIP SYNCING WHEN THE PERSON
THE LIP SYNCING WHEN THE PERSON
THE LIP SYNCING WHEN THE PERSON IS NOT LIP SYNC CORRECTLY
IS NOT LIP SYNC CORRECTLY
IS NOT LIP SYNC CORRECTLY THERE'S FEEDBACK GIVEN BACK TO
THERE'S FEEDBACK GIVEN BACK TO
THERE'S FEEDBACK GIVEN BACK TO THE USER, AND THIS YOU CAN SEE
THE USER, AND THIS YOU CAN SEE
THE USER, AND THIS YOU CAN SEE IS -- IS A TYPE OF EXAMPLE THAT,
IS -- IS A TYPE OF EXAMPLE THAT,
IS -- IS A TYPE OF EXAMPLE THAT, YOU KNOW, ONE CAN BUILD ENTIRELY
YOU KNOW, ONE CAN BUILD ENTIRELY
YOU KNOW, ONE CAN BUILD ENTIRELY ON THE CLIENT USING THIS AND LET
ON THE CLIENT USING THIS AND LET
ON THE CLIENT USING THIS AND LET YOUR IMAGINATION, YOU KNOW, GO
YOUR IMAGINATION, YOU KNOW, GO
YOUR IMAGINATION, YOU KNOW, GO FROM THERE.
FROM THERE.
FROM THERE. SO AGAIN THIS IS AVAILABLE AT
SO AGAIN THIS IS AVAILABLE AT
SO AGAIN THIS IS AVAILABLE AT THE TENSORFLOW JS BOOTH FEEL
THE TENSORFLOW JS BOOTH FEEL
THE TENSORFLOW JS BOOTH FEEL FREE TO TRY IT OUT AND SEE HOW
FREE TO TRY IT OUT AND SEE HOW
FREE TO TRY IT OUT AND SEE HOW YOU DO.
YOU DO.
YOU DO. THE NEXT APPLICATION WE WOULD
THE NEXT APPLICATION WE WOULD
THE NEXT APPLICATION WE WOULD LIKE TO SHOW USE A TRY ON APP
LIKE TO SHOW USE A TRY ON APP
LIKE TO SHOW USE A TRY ON APP WHICH IS A FACE.
WHICH IS A FACE.
WHICH IS A FACE. THIS IS WHAT THEY HAVE PUT ON
THIS IS WHAT THEY HAVE PUT ON
THIS IS WHAT THEY HAVE PUT ON THE PLATFORM I WOULD LIKE
THE PLATFORM I WOULD LIKE
THE PLATFORM I WOULD LIKE BRANDON TO COME ON STAGE AND
BRANDON TO COME ON STAGE AND
BRANDON TO COME ON STAGE AND SHOW YOU HOW THEY USE TENSORFLOW
SHOW YOU HOW THEY USE TENSORFLOW
SHOW YOU HOW THEY USE TENSORFLOW JS TO BUILD THIS APP.
JS TO BUILD THIS APP.
JS TO BUILD THIS APP. [APPLAUSE]
[APPLAUSE]
[APPLAUSE] >> THANKS, SO HI I'M BRENDON I'M
>> THANKS, SO HI I'M BRENDON I'M
>> THANKS, SO HI I'M BRENDON I'M RESEARCH SCIENTISTS AT MODDAFACE
RESEARCH SCIENTISTS AT MODDAFACE
RESEARCH SCIENTISTS AT MODDAFACE AND FIRST LET ME BRIEFLY
AND FIRST LET ME BRIEFLY
AND FIRST LET ME BRIEFLY INTRODUCE MODDAFACE.
INTRODUCE MODDAFACE.
INTRODUCE MODDAFACE. MODDAFACE IS A AUGMENTED REALITY
MODDAFACE IS A AUGMENTED REALITY
MODDAFACE IS A AUGMENTED REALITY FOR A BEAUTY COMPANY.
FOR A BEAUTY COMPANY.
FOR A BEAUTY COMPANY. IT WAS FOUNDED IN 2007 IN
IT WAS FOUNDED IN 2007 IN
IT WAS FOUNDED IN 2007 IN TORONTO, CANADA, AND ACQUIRED
TORONTO, CANADA, AND ACQUIRED
TORONTO, CANADA, AND ACQUIRED JUST LAST YEAR 100% BY L'OREAL.
JUST LAST YEAR 100% BY L'OREAL.
JUST LAST YEAR 100% BY L'OREAL. SO TODAY MODDAFACE COLLABORATES
SO TODAY MODDAFACE COLLABORATES
SO TODAY MODDAFACE COLLABORATES BEAUTY BRANDS L'OREAL PARIS AND
BEAUTY BRANDS L'OREAL PARIS AND
BEAUTY BRANDS L'OREAL PARIS AND MAYBE LINE.
MAYBE LINE.
MAYBE LINE. YOU CAN FIND OUR TECHNOLOGY ON
YOU CAN FIND OUR TECHNOLOGY ON
YOU CAN FIND OUR TECHNOLOGY ON SUCH ONLINE RETAIL GIANTS SUCH
SUCH ONLINE RETAIL GIANTS SUCH
SUCH ONLINE RETAIL GIANTS SUCH AS MACY, SEPHORA AND AMAZON.
AS MACY, SEPHORA AND AMAZON.
AS MACY, SEPHORA AND AMAZON. NOW I'M GOING TO TALK ABOUT HOW
NOW I'M GOING TO TALK ABOUT HOW
NOW I'M GOING TO TALK ABOUT HOW WE MAKE USE OF TENSORFLOW JS IN
WE MAKE USE OF TENSORFLOW JS IN
WE MAKE USE OF TENSORFLOW JS IN OUR VIRTUAL TRY-ON APPLICATIONS.
OUR VIRTUAL TRY-ON APPLICATIONS.
OUR VIRTUAL TRY-ON APPLICATIONS. SO IN ORDER TO INTRODUCE WHY WE
SO IN ORDER TO INTRODUCE WHY WE
SO IN ORDER TO INTRODUCE WHY WE NEED A FRAMEWORK LIKE TENSORFLOW
NEED A FRAMEWORK LIKE TENSORFLOW
NEED A FRAMEWORK LIKE TENSORFLOW JS I'M GOING TO USE WE CHAT MINI
JS I'M GOING TO USE WE CHAT MINI
JS I'M GOING TO USE WE CHAT MINI PROGRAM FOR MAKEUP VIRTUAL TRION
PROGRAM FOR MAKEUP VIRTUAL TRION
PROGRAM FOR MAKEUP VIRTUAL TRION TO SHOWCASE THE KIND OF
TO SHOWCASE THE KIND OF
TO SHOWCASE THE KIND OF CHALLENGES YOU'RE DEPLOYING
CHALLENGES YOU'RE DEPLOYING
CHALLENGES YOU'RE DEPLOYING VIRTUAL TRI-ON SYSTEMS.
VIRTUAL TRI-ON SYSTEMS.
VIRTUAL TRI-ON SYSTEMS. SECOND OF ALL WE WANTED TO PLAY
SECOND OF ALL WE WANTED TO PLAY
SECOND OF ALL WE WANTED TO PLAY OUR APPLICATIONS ON THE CLIENT
OUR APPLICATIONS ON THE CLIENT
OUR APPLICATIONS ON THE CLIENT WE WANTED -- USER USER PRIVACY
WE WANTED -- USER USER PRIVACY
WE WANTED -- USER USER PRIVACY TO AVOID THE LATENCY FROM DOING
TO AVOID THE LATENCY FROM DOING
TO AVOID THE LATENCY FROM DOING A ROUND TRIP TO BACK-END SERVER
A ROUND TRIP TO BACK-END SERVER
A ROUND TRIP TO BACK-END SERVER EVERY FRAME AND ALSO WE HAVE A
EVERY FRAME AND ALSO WE HAVE A
EVERY FRAME AND ALSO WE HAVE A SOFTWARE REQUIREMENT OF ABOUT 10
SOFTWARE REQUIREMENT OF ABOUT 10
SOFTWARE REQUIREMENT OF ABOUT 10 FRAME PER SECOND FRAME RATE IN
FRAME PER SECOND FRAME RATE IN
FRAME PER SECOND FRAME RATE IN ORDER FOR OUR OBLIGATIONS TO
ORDER FOR OUR OBLIGATIONS TO
ORDER FOR OUR OBLIGATIONS TO FEEL INTERACTIVE.
FEEL INTERACTIVE.
FEEL INTERACTIVE. SO THIS IS PARTICULARLY
SO THIS IS PARTICULARLY
SO THIS IS PARTICULARLY DIFFICULT ON THE WE CHAT
DIFFICULT ON THE WE CHAT
DIFFICULT ON THE WE CHAT PLATFORM BECAUSE YOU HAVE -- YOU
PLATFORM BECAUSE YOU HAVE -- YOU
PLATFORM BECAUSE YOU HAVE -- YOU RUN YOUR -- YOU RUN YOUR MINI
RUN YOUR -- YOU RUN YOUR MINI
RUN YOUR -- YOU RUN YOUR MINI PROGRAM IN WE CHAT JAVA SCRIPT
PROGRAM IN WE CHAT JAVA SCRIPT
PROGRAM IN WE CHAT JAVA SCRIPT ENVIRONMENT AND YOU INHERIT THE
ENVIRONMENT AND YOU INHERIT THE
ENVIRONMENT AND YOU INHERIT THE LIMITATIONS OF JAVA SCRIPT SO WE
LIMITATIONS OF JAVA SCRIPT SO WE
LIMITATIONS OF JAVA SCRIPT SO WE NEEDED TO DEVELOP A LIGHTWEIGHT
NEEDED TO DEVELOP A LIGHTWEIGHT
NEEDED TO DEVELOP A LIGHTWEIGHT SPARTAN MODEL FOR FACE TRACKING
SPARTAN MODEL FOR FACE TRACKING
SPARTAN MODEL FOR FACE TRACKING AND ALSO TO FIND A FRAMEWORK
AND ALSO TO FIND A FRAMEWORK
AND ALSO TO FIND A FRAMEWORK THAT RUNS QUICKLY IN JAVA SCRIPT
THAT RUNS QUICKLY IN JAVA SCRIPT
THAT RUNS QUICKLY IN JAVA SCRIPT AND IDEALLY MAKES SENSE OF WEB
AND IDEALLY MAKES SENSE OF WEB
AND IDEALLY MAKES SENSE OF WEB GL TO RUN -- TO RUN OPERATORS
GL TO RUN -- TO RUN OPERATORS
GL TO RUN -- TO RUN OPERATORS USING THE GPU HARBOR
USING THE GPU HARBOR
USING THE GPU HARBOR ACCELERATION.
ACCELERATION.
ACCELERATION. SO SECOND -- THE SECOND
SO SECOND -- THE SECOND
SO SECOND -- THE SECOND CHALLENGE WE RAN INTO THE WE
CHALLENGE WE RAN INTO THE WE
CHALLENGE WE RAN INTO THE WE CHAT -- WE CHAT MINI PROGRAMS
CHAT -- WE CHAT MINI PROGRAMS
CHAT -- WE CHAT MINI PROGRAMS HAVE A MINI -- IT HAS TO BE
HAVE A MINI -- IT HAS TO BE
HAVE A MINI -- IT HAS TO BE SMALL AND ALLOWS US TO DEVELOP A
SMALL AND ALLOWS US TO DEVELOP A
SMALL AND ALLOWS US TO DEVELOP A SMALL MODEL AND SO WE CAN LOAD
SMALL MODEL AND SO WE CAN LOAD
SMALL MODEL AND SO WE CAN LOAD IT AS QUICKLY AS POSSIBLE AS
IT AS QUICKLY AS POSSIBLE AS
IT AS QUICKLY AS POSSIBLE AS WELL.
WELL.
WELL. AND THIRDLY, WE -- BECAUSE OUR
AND THIRDLY, WE -- BECAUSE OUR
AND THIRDLY, WE -- BECAUSE OUR MODELS HAVE SOME CUSTOM
MODELS HAVE SOME CUSTOM
MODELS HAVE SOME CUSTOM OPERATORS WE NEEDED A FRAMEWORK
OPERATORS WE NEEDED A FRAMEWORK
OPERATORS WE NEEDED A FRAMEWORK THAT'S ON HIS TENSIBLE WITH
THAT'S ON HIS TENSIBLE WITH
THAT'S ON HIS TENSIBLE WITH CUSTOM OPERATORS AND WE ALSO
CUSTOM OPERATORS AND WE ALSO
CUSTOM OPERATORS AND WE ALSO NEEDED A FRAMEWORK THAT IS GOING
NEEDED A FRAMEWORK THAT IS GOING
NEEDED A FRAMEWORK THAT IS GOING TO SUPPORT ALL THE DIFFERENT
TO SUPPORT ALL THE DIFFERENT
TO SUPPORT ALL THE DIFFERENT MOBILE PHONES THAT ARE SUPPORTED
MOBILE PHONES THAT ARE SUPPORTED
MOBILE PHONES THAT ARE SUPPORTED BY -- MOBILE PHONE MODELS THAT
BY -- MOBILE PHONE MODELS THAT
BY -- MOBILE PHONE MODELS THAT ARE SUPPORTED BY WE CHAT ITSELF.
ARE SUPPORTED BY WE CHAT ITSELF.
ARE SUPPORTED BY WE CHAT ITSELF. SO NOW I'M GOING TO TELL YOU HOW
SO NOW I'M GOING TO TELL YOU HOW
SO NOW I'M GOING TO TELL YOU HOW TENSORFLOW JS FIT THE BILL AND
TENSORFLOW JS FIT THE BILL AND
TENSORFLOW JS FIT THE BILL AND WAS ABLE TO OVERCOME SOME OF
WAS ABLE TO OVERCOME SOME OF
WAS ABLE TO OVERCOME SOME OF THESE CHALLENGES THAT WE RAN
THESE CHALLENGES THAT WE RAN
THESE CHALLENGES THAT WE RAN INTO IN DEPLOYING OUR MAKEUP TRY
INTO IN DEPLOYING OUR MAKEUP TRY
INTO IN DEPLOYING OUR MAKEUP TRY ON.
ON.
ON. FIRST EXPO JS RUNS ON THE CLIENT
FIRST EXPO JS RUNS ON THE CLIENT
FIRST EXPO JS RUNS ON THE CLIENT SIDE BECAUSE IT MAKES SENSE OF
SIDE BECAUSE IT MAKES SENSE OF
SIDE BECAUSE IT MAKES SENSE OF WEB GL BACK-END IT'S ABLE TO
WEB GL BACK-END IT'S ABLE TO
WEB GL BACK-END IT'S ABLE TO HARNESS THE MOBILE PHONE GPU
HARNESS THE MOBILE PHONE GPU
HARNESS THE MOBILE PHONE GPU HARBOR ACCELERATION WHICH GIVES
HARBOR ACCELERATION WHICH GIVES
HARBOR ACCELERATION WHICH GIVES IT LIKE AN ORDER OF MAKING
IT LIKE AN ORDER OF MAKING
IT LIKE AN ORDER OF MAKING NEWSPAPER SPEEDUP OVER BROWSER
NEWSPAPER SPEEDUP OVER BROWSER
NEWSPAPER SPEEDUP OVER BROWSER SUCH AS WEB ASSEMBLY WHICH ARE
SUCH AS WEB ASSEMBLY WHICH ARE
SUCH AS WEB ASSEMBLY WHICH ARE LIMITED RIGHT NOW BY LACK OF
LIMITED RIGHT NOW BY LACK OF
LIMITED RIGHT NOW BY LACK OF MULTITHREADING.
MULTITHREADING.
MULTITHREADING. SO IT REALLY ALLOWS US TO FRAME
SO IT REALLY ALLOWS US TO FRAME
SO IT REALLY ALLOWS US TO FRAME REQUIREMENT AND SECOND OF ALL
REQUIREMENT AND SECOND OF ALL
REQUIREMENT AND SECOND OF ALL THE TENSORFLOW JS LIBRARY IS
THE TENSORFLOW JS LIBRARY IS
THE TENSORFLOW JS LIBRARY IS SMALL 700 KILOBYTES AND COMBINED
SMALL 700 KILOBYTES AND COMBINED
SMALL 700 KILOBYTES AND COMBINED WITH OUR 400 KILOBITE MODEL
WITH OUR 400 KILOBITE MODEL
WITH OUR 400 KILOBITE MODEL SIZES WE WERE ABLE TO FIT
SIZES WE WERE ABLE TO FIT
SIZES WE WERE ABLE TO FIT EVERYTHING WITHIN THE WE CHAT
EVERYTHING WITHIN THE WE CHAT
EVERYTHING WITHIN THE WE CHAT MINI PROGRAM FILE SIZE LIMIT.
MINI PROGRAM FILE SIZE LIMIT.
MINI PROGRAM FILE SIZE LIMIT. AND THIRDLY, TENSORFLOW JS BOTH
AND THIRDLY, TENSORFLOW JS BOTH
AND THIRDLY, TENSORFLOW JS BOTH HAS WIDESPREAD SUPPORT FOR
HAS WIDESPREAD SUPPORT FOR
HAS WIDESPREAD SUPPORT FOR BUILT-IN DEEP LEARNING OPERATORS
BUILT-IN DEEP LEARNING OPERATORS
BUILT-IN DEEP LEARNING OPERATORS AND ALSO ALLOWED TO US EXTEND IT
AND ALSO ALLOWED TO US EXTEND IT
AND ALSO ALLOWED TO US EXTEND IT WITH OUR CUSTOM OPERATORS SO WE
WITH OUR CUSTOM OPERATORS SO WE
WITH OUR CUSTOM OPERATORS SO WE NEED FOR OUR FACE TRACK AND
NEED FOR OUR FACE TRACK AND
NEED FOR OUR FACE TRACK AND FINALLY TENSORFLOW JS JS IS
FINALLY TENSORFLOW JS JS IS
FINALLY TENSORFLOW JS JS IS SUPPORTING A WIDE VARIETY OF
SUPPORTING A WIDE VARIETY OF
SUPPORTING A WIDE VARIETY OF MICROPHONE MODELS AND HAS
MICROPHONE MODELS AND HAS
MICROPHONE MODELS AND HAS CONTINUOUS SUPPORT FROM THE
CONTINUOUS SUPPORT FROM THE
CONTINUOUS SUPPORT FROM THE TENSORFLOW JS AND WE CHOSE THAT
TENSORFLOW JS AND WE CHOSE THAT
TENSORFLOW JS AND WE CHOSE THAT AS THE FRAMEWORK TO DEPLOY ON
AS THE FRAMEWORK TO DEPLOY ON
AS THE FRAMEWORK TO DEPLOY ON THE WE CHAT PLUG-IN.
THE WE CHAT PLUG-IN.
THE WE CHAT PLUG-IN. SO NOW I'D LIKE TO SHARE WITH
SO NOW I'D LIKE TO SHARE WITH
SO NOW I'D LIKE TO SHARE WITH YOU SOME OF OUR RESULTS.
YOU SOME OF OUR RESULTS.
YOU SOME OF OUR RESULTS. SO WITH THE FAITH OF TENSORFLOW
SO WITH THE FAITH OF TENSORFLOW
SO WITH THE FAITH OF TENSORFLOW JS WE WERE ABLE TO SUCCESSFULLY
JS WE WERE ABLE TO SUCCESSFULLY
JS WE WERE ABLE TO SUCCESSFULLY DEPLOY OUR -- DEPLOY TO WE CHAT
DEPLOY OUR -- DEPLOY TO WE CHAT
DEPLOY OUR -- DEPLOY TO WE CHAT OUR EASY TO USE REAL TIME SYSTEM
OUR EASY TO USE REAL TIME SYSTEM
OUR EASY TO USE REAL TIME SYSTEM FOR REALISTIC AR VIRTUAL TRY-ON
FOR REALISTIC AR VIRTUAL TRY-ON
FOR REALISTIC AR VIRTUAL TRY-ON AND OUR ENTIRE FINAL SOLUTION
AND OUR ENTIRE FINAL SOLUTION
AND OUR ENTIRE FINAL SOLUTION FIT INTO THE 1.8 MEGABITES
FIT INTO THE 1.8 MEGABITES
FIT INTO THE 1.8 MEGABITES INCLUDING THE CODE AND THE
INCLUDING THE CODE AND THE
INCLUDING THE CODE AND THE MODELS AND ON AN iPHONE 8S AND
MODELS AND ON AN iPHONE 8S AND
MODELS AND ON AN iPHONE 8S AND TRACKING IS 25 FRAMES PER
TRACKING IS 25 FRAMES PER
TRACKING IS 25 FRAMES PER SECOND.
SECOND.
SECOND. SO TENSORFLOW JS COUPLED WITH
SO TENSORFLOW JS COUPLED WITH
SO TENSORFLOW JS COUPLED WITH OUR TINY CNN FACE TRACKING MODEL
OUR TINY CNN FACE TRACKING MODEL
OUR TINY CNN FACE TRACKING MODEL ALLOWED US TO DEPLOY TO WEB --
ALLOWED US TO DEPLOY TO WEB --
ALLOWED US TO DEPLOY TO WEB -- OUR FASTEST, SMALLEST MAKEUP
OUR FASTEST, SMALLEST MAKEUP
OUR FASTEST, SMALLEST MAKEUP VIRTUAL TRY-ON APP TO DATE.
VIRTUAL TRY-ON APP TO DATE.
VIRTUAL TRY-ON APP TO DATE. SO NOW I'D LIKE TO BRIEFLY
SO NOW I'D LIKE TO BRIEFLY
SO NOW I'D LIKE TO BRIEFLY MENTION WHAT'S GOING ON.
MENTION WHAT'S GOING ON.
MENTION WHAT'S GOING ON. W'VE ALREADY USED TENSORFLOW JS
W'VE ALREADY USED TENSORFLOW JS
W'VE ALREADY USED TENSORFLOW JS TO CREATE WEB APPLICATION DEMOS
TO CREATE WEB APPLICATION DEMOS
TO CREATE WEB APPLICATION DEMOS FOR OUR MAKEUP -- MAKEUP TRY-ON,
FOR OUR MAKEUP -- MAKEUP TRY-ON,
FOR OUR MAKEUP -- MAKEUP TRY-ON, OUR HAIR COLOR TRY-ON AND NAIL
OUR HAIR COLOR TRY-ON AND NAIL
OUR HAIR COLOR TRY-ON AND NAIL TRY ON AND WE WERE ABLE TO
TRY ON AND WE WERE ABLE TO
TRY ON AND WE WERE ABLE TO ACHIEVE AN ORDER OF MAGNITUDE
ACHIEVE AN ORDER OF MAGNITUDE
ACHIEVE AN ORDER OF MAGNITUDE SPEED UP FOR A GUIDED OPERATOR
SPEED UP FOR A GUIDED OPERATOR
SPEED UP FOR A GUIDED OPERATOR BY TAKING OUR WEB ASSEMBLY
BY TAKING OUR WEB ASSEMBLY
BY TAKING OUR WEB ASSEMBLY IMPLEMENTION AND REIMPLEMENTING
IMPLEMENTION AND REIMPLEMENTING
IMPLEMENTION AND REIMPLEMENTING IT ON TENSORFLOW JS THIS
IT ON TENSORFLOW JS THIS
IT ON TENSORFLOW JS THIS AMOUNTED TO ABOUT A 20
AMOUNTED TO ABOUT A 20
AMOUNTED TO ABOUT A 20 MILLISECOND LATENT SI FOR THE
MILLISECOND LATENT SI FOR THE
MILLISECOND LATENT SI FOR THE WHOLE SYSTEM AND WE HAVE A
WHOLE SYSTEM AND WE HAVE A
WHOLE SYSTEM AND WE HAVE A NUMBER OF OTHER PRODUCTS GOING
NUMBER OF OTHER PRODUCTS GOING
NUMBER OF OTHER PRODUCTS GOING ON AT MODDAFACE, VIRTUAL AGING
ON AT MODDAFACE, VIRTUAL AGING
ON AT MODDAFACE, VIRTUAL AGING SIMULATION AND SKIN ANALYSIS
SIMULATION AND SKIN ANALYSIS
SIMULATION AND SKIN ANALYSIS THAT WE HOPE TO USE TENSORFLOW
THAT WE HOPE TO USE TENSORFLOW
THAT WE HOPE TO USE TENSORFLOW JS TO DEPLOY IN THE NEAR FUTURE.
JS TO DEPLOY IN THE NEAR FUTURE.
JS TO DEPLOY IN THE NEAR FUTURE. SO THANK YOU FOR EVERYONE FOR
SO THANK YOU FOR EVERYONE FOR
SO THANK YOU FOR EVERYONE FOR LISTENING AND I'LL PASS THE
LISTENING AND I'LL PASS THE
LISTENING AND I'LL PASS THE PRESENTATION ON.
PRESENTATION ON.
PRESENTATION ON. [APPLAUSE]
[APPLAUSE]
[APPLAUSE] AND I WANT TO SHOW YOU DETAILS
AND I WANT TO SHOW YOU DETAILS
AND I WANT TO SHOW YOU DETAILS WHICH USE AUGMENTED
WHICH USE AUGMENTED
WHICH USE AUGMENTED >> THROUGH THE CAMERA AND WEB
>> THROUGH THE CAMERA AND WEB
>> THROUGH THE CAMERA AND WEB PAGE THERE ARE FIRST WE NEED A
PAGE THERE ARE FIRST WE NEED A
PAGE THERE ARE FIRST WE NEED A MODEL THAT IS TRAINED TO FIND
MODEL THAT IS TRAINED TO FIND
MODEL THAT IS TRAINED TO FIND THE FACE.
THE FACE.
THE FACE. HEO ON AA
WITH
LAMENT.
LAMENT.
WE WERE SURPRISED HOW IT SCALES.
WE WERE SURPRISED HOW IT SCALES.
WE WERE SURPRISED HOW IT SCALES. 5,000 NOTES.
5,000 NOTES.
5,000 NOTES. >> THIS WAS THE FIRST TIME
>> THIS WAS THE FIRST TIME
>> THIS WAS THE FIRST TIME ANYBODY HAS RUN AN API
ANYBODY HAS RUN AN API
ANYBODY HAS RUN AN API APPLICATION AT THIS SCALE
APPLICATION AT THIS SCALE
APPLICATION AT THIS SCALE INSTEAD OF HAVING THE CLIMATE
INSTEAD OF HAVING THE CLIMATE
INSTEAD OF HAVING THE CLIMATE SCIENTISTS HOW TO WRITE HIGH
SCIENTISTS HOW TO WRITE HIGH
SCIENTISTS HOW TO WRITE HIGH TUNED CODE THEY COULD WRITE?
TUNED CODE THEY COULD WRITE?
TUNED CODE THEY COULD WRITE? -- GET ALL THE HIGH PERFORMANCE
-- GET ALL THE HIGH PERFORMANCE
-- GET ALL THE HIGH PERFORMANCE CODE MOST PEOPLE ARE USED TO
CODE MOST PEOPLE ARE USED TO
CODE MOST PEOPLE ARE USED TO WITHIN TENSORFLOW.
WITHIN TENSORFLOW.
WITHIN TENSORFLOW. >> WE'RE ENTERING THE SPACE
>> WE'RE ENTERING THE SPACE
>> WE'RE ENTERING THE SPACE WHERE IT CAN CONTRADICT TO THE
WHERE IT CAN CONTRADICT TO THE
WHERE IT CAN CONTRADICT TO THE EXTREME WEATHER EVENTS.
EXTREME WEATHER EVENTS.
EXTREME WEATHER EVENTS. >> WHEN YOU COMBINE
>> WHEN YOU COMBINE
>> WHEN YOU COMBINE TRADITIONALLY WITH AI FUSION
TRADITIONALLY WITH AI FUSION
TRADITIONALLY WITH AI FUSION REACTOR RESEARCH UNDERSTANDING
REACTOR RESEARCH UNDERSTANDING
REACTOR RESEARCH UNDERSTANDING DISEASES LIKE ALZHEIMER'S,
DISEASES LIKE ALZHEIMER'S,
DISEASES LIKE ALZHEIMER'S, CANCER, RIGHT, THAT'S LIKE
CANCER, RIGHT, THAT'S LIKE
CANCER, RIGHT, THAT'S LIKE INCREDIBLE.
INCREDIBLE.
INCREDIBLE. >> WE'VE SHOWN WITH A HIGH
>> WE'VE SHOWN WITH A HIGH
>> WE'VE SHOWN WITH A HIGH PRODUCTIVITY YOU CAN GET AWESOME
PRODUCTIVITY YOU CAN GET AWESOME
PRODUCTIVITY YOU CAN GET AWESOME PERFORMANCE AND ACCOMPLISH YOUR
PERFORMANCE AND ACCOMPLISH YOUR
PERFORMANCE AND ACCOMPLISH YOUR GOALS.
GOALS.
GOALS. >> GENETICS, COMESMOLOGIST, HIGH
>> GENETICS, COMESMOLOGIST, HIGH
>> GENETICS, COMESMOLOGIST, HIGH ENERGY PHYSICS THAT'S IMMENSELY
ENERGY PHYSICS THAT'S IMMENSELY
ENERGY PHYSICS THAT'S IMMENSELY EXCITING FOR ME.
EXCITING FOR ME.
EXCITING FOR ME. ♪
WORK A LOT ON HOW DO YOU DO DEEP
WORK A LOT ON HOW DO YOU DO DEEP
WORK A LOT ON HOW DO YOU DO DEEP NETWORKS AND BUILD, YOU KNOW,
NETWORKS AND BUILD, YOU KNOW,
NETWORKS AND BUILD, YOU KNOW, MACHINE LEARNING SYSTEMS THAT
MACHINE LEARNING SYSTEMS THAT
MACHINE LEARNING SYSTEMS THAT SCALE ON THE CLOUD WITH MINIMAL
SCALE ON THE CLOUD WITH MINIMAL
SCALE ON THE CLOUD WITH MINIMAL SUPERVISION.
SUPERVISION.
SUPERVISION. WE WORK IN LANGUAGE
WE WORK IN LANGUAGE
WE WORK IN LANGUAGE UNDERSTANDING MULTI-MODAL AM
UNDERSTANDING MULTI-MODAL AM
UNDERSTANDING MULTI-MODAL AM KEGS.
KEGS.
KEGS. HOW DO YOU TAKE ALL THESE ALGA
HOW DO YOU TAKE ALL THESE ALGA
HOW DO YOU TAKE ALL THESE ALGA RIT IMS AND COMPUTE THEM ON THE
RIT IMS AND COMPUTE THEM ON THE
RIT IMS AND COMPUTE THEM ON THE EDGE.
EDGE.
EDGE. I'M HERE TODAY WITH MY
I'M HERE TODAY WITH MY
I'M HERE TODAY WITH MY COLLEAGUE?
COLLEAGUE?
COLLEAGUE? >> I'M WORKING WITH SUJI AND ALL
>> I'M WORKING WITH SUJI AND ALL
>> I'M WORKING WITH SUJI AND ALL THE RELATED TOPICS.
THE RELATED TOPICS.
THE RELATED TOPICS. >> LET'S GET STARTED.
>> LET'S GET STARTED.
>> LET'S GET STARTED. YOU HAVE THE HONOR OF BEING HERE
YOU HAVE THE HONOR OF BEING HERE
YOU HAVE THE HONOR OF BEING HERE FOR THE LAST SESSION OF THE LAST
FOR THE LAST SESSION OF THE LAST
FOR THE LAST SESSION OF THE LAST DAY.
DAY.
DAY. KUDOS, AND BRAVO.
KUDOS, AND BRAVO.
KUDOS, AND BRAVO. I MEAN, YOU ARE REALLY
I MEAN, YOU ARE REALLY
I MEAN, YOU ARE REALLY DEDICATED.
DEDICATED.
DEDICATED. LET US BEGIN.
LET US BEGIN.
LET US BEGIN. WE ARE VERY EXCITED TO TALK TO
WE ARE VERY EXCITED TO TALK TO
WE ARE VERY EXCITED TO TALK TO YOU ABOUT NOODLE-STRUCTURED
YOU ABOUT NOODLE-STRUCTURED
YOU ABOUT NOODLE-STRUCTURED LEARNING, WHICH IS A NEW
LEARNING, WHICH IS A NEW
LEARNING, WHICH IS A NEW FRAMEWORK INTENSIVE FLOW THAT
FRAMEWORK INTENSIVE FLOW THAT
FRAMEWORK INTENSIVE FLOW THAT ALLOWS YOU TO TRAIN NOODLED
ALLOWS YOU TO TRAIN NOODLED
ALLOWS YOU TO TRAIN NOODLED FRAMEWORKS.
FRAMEWORKS.
FRAMEWORKS. FIRST, LET'S GO OVER SOME
FIRST, LET'S GO OVER SOME
FIRST, LET'S GO OVER SOME BASICS.
BASICS.
BASICS. IF YOU ARE IN THIS ROOM, AND YOU
IF YOU ARE IN THIS ROOM, AND YOU
IF YOU ARE IN THIS ROOM, AND YOU KNOW ABOUT DEEP LEARNING, AND
KNOW ABOUT DEEP LEARNING, AND
KNOW ABOUT DEEP LEARNING, AND YOU CARE DEEPLY ABOUT DEEP
YOU CARE DEEPLY ABOUT DEEP
YOU CARE DEEPLY ABOUT DEEP LEARNING YOU KNOW HOW TYPICAL
LEARNING YOU KNOW HOW TYPICAL
LEARNING YOU KNOW HOW TYPICAL NEURAL NETWORKS WORK.
NEURAL NETWORKS WORK.
NEURAL NETWORKS WORK. IF YOU WANT TO TAKE A NEURAL
IF YOU WANT TO TAKE A NEURAL
IF YOU WANT TO TAKE A NEURAL NETWORK AND TRAIN IT TO
NETWORK AND TRAIN IT TO
NETWORK AND TRAIN IT TO RECOGNIZE IMAGES AND DISTINGUISH
RECOGNIZE IMAGES AND DISTINGUISH
RECOGNIZE IMAGES AND DISTINGUISH BETWEEN CONCEPTS LIKE CATS AND
BETWEEN CONCEPTS LIKE CATS AND
BETWEEN CONCEPTS LIKE CATS AND DOGS, WHAT WOULD YOU DO?
DOGS, WHAT WOULD YOU DO?
DOGS, WHAT WOULD YOU DO? YOU FEED IMAGES ON THE LEFT SIDE
YOU FEED IMAGES ON THE LEFT SIDE
YOU FEED IMAGES ON THE LEFT SIDE AND DID GIVE THE LABEL DOG AND
AND DID GIVE THE LABEL DOG AND
AND DID GIVE THE LABEL DOG AND FEED IT TO THE NETWORK, AND THE
FEED IT TO THE NETWORK, AND THE
FEED IT TO THE NETWORK, AND THE PROCESS BY WHICH IT WORKS IS YOU
PROCESS BY WHICH IT WORKS IS YOU
PROCESS BY WHICH IT WORKS IS YOU ADJUST WEIGHTS IN THE NETWORKS
ADJUST WEIGHTS IN THE NETWORKS
ADJUST WEIGHTS IN THE NETWORKS SUCH THAT THE NETWORK LEARNS TO
SUCH THAT THE NETWORK LEARNS TO
SUCH THAT THE NETWORK LEARNS TO DISTINGUISH AND DISCRIINATE
DISTINGUISH AND DISCRIINATE
DISTINGUISH AND DISCRIINATE BETWEENS DIFFERENT CONAT THE
BETWEENS DIFFERENT CONAT THE
BETWEENS DIFFERENT CONAT THE PARTICULAR TIME AND TARGET THE
PARTICULAR TIME AND TARGET THE
PARTICULAR TIME AND TARGET THE IMAGE.
IMAGE.
IMAGE. THIS IS THE LAST SESSION, LAST
THIS IS THE LAST SESSION, LAST
THIS IS THE LAST SESSION, LAST DAY.
DAY.
DAY. STILL, VERY HAPPY THAT YOU ARE
STILL, VERY HAPPY THAT YOU ARE
STILL, VERY HAPPY THAT YOU ARE ALL WITH ME HERE.
ALL WITH ME HERE.
ALL WITH ME HERE. WHAT IS THE ONE THING?
WHAT IS THE ONE THING?
WHAT IS THE ONE THING? IT'S ALL GREAT.
IT'S ALL GREAT.
IT'S ALL GREAT. WE HAVE A LOT OF FANCY ALGA
WE HAVE A LOT OF FANCY ALGA
WE HAVE A LOT OF FANCY ALGA RHYTHMS AND FANCY NETWORKS.
RHYTHMS AND FANCY NETWORKS.
RHYTHMS AND FANCY NETWORKS. WHAT IS THE ONE CORPS INGREDIENT
WHAT IS THE ONE CORPS INGREDIENT
WHAT IS THE ONE CORPS INGREDIENT WE NEED WHEN WE BUILD A NETWORK?
WE NEED WHEN WE BUILD A NETWORK?
WE NEED WHEN WE BUILD A NETWORK? ALMOST MAJORITY OF THE
ALMOST MAJORITY OF THE
ALMOST MAJORITY OF THE APPLICATIONS WE WORK ON REQUIRE
APPLICATIONS WE WORK ON REQUIRE
APPLICATIONS WE WORK ON REQUIRE DATA.
DATA.
DATA. IT'S NOT ONE IMAGE YOU ARE
IT'S NOT ONE IMAGE YOU ARE
IT'S NOT ONE IMAGE YOU ARE FEEDING TO THIS NETWORK.
FEEDING TO THIS NETWORK.
FEEDING TO THIS NETWORK. YOU ARE TAKING A BUNCH OF IMAGES
YOU ARE TAKING A BUNCH OF IMAGES
YOU ARE TAKING A BUNCH OF IMAGES PAIRED WITH THE LABELS, CATS AND
PAIRED WITH THE LABELS, CATS AND
PAIRED WITH THE LABELS, CATS AND DOGS IN THIS CASE, BUT, OF
DOGS IN THIS CASE, BUT, OF
DOGS IN THIS CASE, BUT, OF COURSE, IT COULD BE WHATEVER,
COURSE, IT COULD BE WHATEVER,
COURSE, IT COULD BE WHATEVER, LIKE, DEPENDING ON THE
LIKE, DEPENDING ON THE
LIKE, DEPENDING ON THE APPLICATION.
APPLICATION.
APPLICATION. YOU FEED IT, LIKE, THOUSANDS OR
YOU FEED IT, LIKE, THOUSANDS OR
YOU FEED IT, LIKE, THOUSANDS OR HUNDREDS OF THANK YOUS THOUSANDS
HUNDREDS OF THANK YOUS THOUSANDS
HUNDREDS OF THANK YOUS THOUSANDS OR EVEN MILLIONS OF EXAMPLES
OR EVEN MILLIONS OF EXAMPLES
OR EVEN MILLIONS OF EXAMPLES INTO THE NETWORK TO TRAIN A GOOD
INTO THE NETWORK TO TRAIN A GOOD
INTO THE NETWORK TO TRAIN A GOOD CLASSIFIED.
CLASSIFIED.
CLASSIFIED. RIGHT?
RIGHT?
RIGHT? TODAY WE'RE GOING TO INTRODUCE
TODAY WE'RE GOING TO INTRODUCE
TODAY WE'RE GOING TO INTRODUCE NOODLE STRUCTURED LEARNING,
NOODLE STRUCTURED LEARNING,
NOODLE STRUCTURED LEARNING, WHICH IS A FRAMEWORK, WE'RE
WHICH IS A FRAMEWORK, WE'RE
WHICH IS A FRAMEWORK, WE'RE HAPPY TO TALK ABOUT IT INTENSIVE
HAPPY TO TALK ABOUT IT INTENSIVE
HAPPY TO TALK ABOUT IT INTENSIVE 2.0 AND IT ALLOWS YOU TO TRAIN
2.0 AND IT ALLOWS YOU TO TRAIN
2.0 AND IT ALLOWS YOU TO TRAIN BETTER AND MORROW BUST NETWORKS
BETTER AND MORROW BUST NETWORKS
BETTER AND MORROW BUST NETWORKS BY LEVERAGING STRUCTURE IN THE
BY LEVERAGING STRUCTURE IN THE
BY LEVERAGING STRUCTURE IN THE DATA.
DATA.
DATA. SO THE CORE IDEA BEHIND THE
SO THE CORE IDEA BEHIND THE
SO THE CORE IDEA BEHIND THE FRAMEWORK IS WE'RE GOING TO TAKE
FRAMEWORK IS WE'RE GOING TO TAKE
FRAMEWORK IS WE'RE GOING TO TAKE NOODLE NETWORKS AND FEED IS IN
NOODLE NETWORKS AND FEED IS IN
NOODLE NETWORKS AND FEED IS IN ADDITION TO FEATURE INPUTS,
ADDITION TO FEATURE INPUTS,
ADDITION TO FEATURE INPUTS, STRUCTURED SIGNALS.
STRUCTURED SIGNALS.
STRUCTURED SIGNALS. THINK OF THE IMAGE THAT I SHOWED
THINK OF THE IMAGE THAT I SHOWED
THINK OF THE IMAGE THAT I SHOWED YOU EARLIER.
YOU EARLIER.
YOU EARLIER. NOW, IN ADDITION TO THESE IMAGES
NOW, IN ADDITION TO THESE IMAGES
NOW, IN ADDITION TO THESE IMAGES PAIRED WITH LABELS, YOU ARE
PAIRED WITH LABELS, YOU ARE
PAIRED WITH LABELS, YOU ARE GOING TO FEED IT CONNECTIONS OR
GOING TO FEED IT CONNECTIONS OR
GOING TO FEED IT CONNECTIONS OR RELATIONSHIPS BETWEEN THE
RELATIONSHIPS BETWEEN THE
RELATIONSHIPS BETWEEN THE SAMPLES THEMSELVES.
SAMPLES THEMSELVES.
SAMPLES THEMSELVES. I WILL GET TO WHAT THESE
I WILL GET TO WHAT THESE
I WILL GET TO WHAT THESE RELATIONSHIPS MIGHT MEAN, BUT
RELATIONSHIPS MIGHT MEAN, BUT
RELATIONSHIPS MIGHT MEAN, BUT YOU HAVE THESE STRUCTURED
YOU HAVE THESE STRUCTURED
YOU HAVE THESE STRUCTURED SIGNALS, AND THE LABELS AND YOU
SIGNALS, AND THE LABELS AND YOU
SIGNALS, AND THE LABELS AND YOU FEED BOTH INTO THE NETWORK.
FEED BOTH INTO THE NETWORK.
FEED BOTH INTO THE NETWORK. YOU MIGHT ASK WHAT DO YOU MEAN
YOU MIGHT ASK WHAT DO YOU MEAN
YOU MIGHT ASK WHAT DO YOU MEAN BY STRUCTURE, RIGHT?
BY STRUCTURE, RIGHT?
BY STRUCTURE, RIGHT? STRUCTURE IS EVERYWHERE.
STRUCTURE IS EVERYWHERE.
STRUCTURE IS EVERYWHERE. IN THE EXAMPLE THAT I SHOWED YOU
IN THE EXAMPLE THAT I SHOWED YOU
IN THE EXAMPLE THAT I SHOWED YOU EARLIER, IF YOU LOOK AT IMAGES,
EARLIER, IF YOU LOOK AT IMAGES,
EARLIER, IF YOU LOOK AT IMAGES, JUST TAKE A LOOK AT THIS GRAPH
JUST TAKE A LOOK AT THIS GRAPH
JUST TAKE A LOOK AT THIS GRAPH HERE.
HERE.
HERE. THE IMAGE THAT IS ARE CONNECTED
THE IMAGE THAT IS ARE CONNECTED
THE IMAGE THAT IS ARE CONNECTED VIA EDGES IN THE PICTURE HERE
VIA EDGES IN THE PICTURE HERE
VIA EDGES IN THE PICTURE HERE BASICALLY REPRESENT THAT THERE'S
BASICALLY REPRESENT THAT THERE'S
BASICALLY REPRESENT THAT THERE'S SOME VISUAL SIMILARITIES BETWEEN
SOME VISUAL SIMILARITIES BETWEEN
SOME VISUAL SIMILARITIES BETWEEN THESE.
THESE.
THESE. IT IS ACTUALLY PRETTY EASY TO
IT IS ACTUALLY PRETTY EASY TO
IT IS ACTUALLY PRETTY EASY TO CONSTRUCT A STRUCTURED SIGNAL
CONSTRUCT A STRUCTURED SIGNAL
CONSTRUCT A STRUCTURED SIGNAL FROM DAY TO DAY SOURCES OF DATA.
FROM DAY TO DAY SOURCES OF DATA.
FROM DAY TO DAY SOURCES OF DATA. IN THE CASE OF IMAGES HERE, IT'S
IN THE CASE OF IMAGES HERE, IT'S
IN THE CASE OF IMAGES HERE, IT'S VISUAL SIMILARITY, BUT YOU COULD
VISUAL SIMILARITY, BUT YOU COULD
VISUAL SIMILARITY, BUT YOU COULD THINK OF WHAT IF YOU TAGGED YOUR
THINK OF WHAT IF YOU TAGGED YOUR
THINK OF WHAT IF YOU TAGGED YOUR IMAGES AND CREATED ALBUMS.
IMAGES AND CREATED ALBUMS.
IMAGES AND CREATED ALBUMS. THAT HAS SPECIFIC CONCEPTS.
THAT HAS SPECIFIC CONCEPTS.
THAT HAS SPECIFIC CONCEPTS. EVERYTHING WITHIN THAT REPTSZ
EVERYTHING WITHIN THAT REPTSZ
EVERYTHING WITHIN THAT REPTSZ ANOTHER TYPE OF STRUCTURE.
ANOTHER TYPE OF STRUCTURE.
ANOTHER TYPE OF STRUCTURE. IT'S NOT JUST FOR IMAGES.
IT'S NOT JUST FOR IMAGES.
IT'S NOT JUST FOR IMAGES. WE CAN GO TO, LIKE, MORE
WE CAN GO TO, LIKE, MORE
WE CAN GO TO, LIKE, MORE ADVANCED OR, LIKE, YOU KNOW,
ADVANCED OR, LIKE, YOU KNOW,
ADVANCED OR, LIKE, YOU KNOW, DIFFERENT -- COMPLETELY
DIFFERENT -- COMPLETELY
DIFFERENT -- COMPLETELY DIFFERENT APPLICATIONS LIKE IF
DIFFERENT APPLICATIONS LIKE IF
DIFFERENT APPLICATIONS LIKE IF YOU WANT TO TAKE, LIKE,
YOU WANT TO TAKE, LIKE,
YOU WANT TO TAKE, LIKE, PUBLICATIONS, SCIENTIFIC
PUBLICATIONS, SCIENTIFIC
PUBLICATIONS, SCIENTIFIC PUBLICATIONS OR NEWS ARTICLES,
PUBLICATIONS OR NEWS ARTICLES,
PUBLICATIONS OR NEWS ARTICLES, AND YOU WANT TO TAG THEM WITH
AND YOU WANT TO TAG THEM WITH
AND YOU WANT TO TAG THEM WITH THE TOPIC, ONE SIMPLE THING,
THE TOPIC, ONE SIMPLE THING,
THE TOPIC, ONE SIMPLE THING, TAKE BIOMEDICAL LITERATURE.
TAKE BIOMEDICAL LITERATURE.
TAKE BIOMEDICAL LITERATURE. ALL THE PAPERS THAT ARE
ALL THE PAPERS THAT ARE
ALL THE PAPERS THAT ARE PUBLISHED WHETHER IT'S NATURE OR
PUBLISHED WHETHER IT'S NATURE OR
PUBLISHED WHETHER IT'S NATURE OR ANY OF THE CONFERENCES, THEY
ANY OF THE CONFERENCES, THEY
ANY OF THE CONFERENCES, THEY HAVE REFERENCES AND CITATIONS TO
HAVE REFERENCES AND CITATIONS TO
HAVE REFERENCES AND CITATIONS TO OTHER PAPERS.
OTHER PAPERS.
OTHER PAPERS. RELATIONSHIPS THAT ARE MODELLED
RELATIONSHIPS THAT ARE MODELLED
RELATIONSHIPS THAT ARE MODELLED BETWEEN DIFFERENT TYPES OF
BETWEEN DIFFERENT TYPES OF
BETWEEN DIFFERENT TYPES OF OBJECTS.
OBJECTS.
OBJECTS. IN THE NATURAL LANGUAGE SPACE,
IN THE NATURAL LANGUAGE SPACE,
IN THE NATURAL LANGUAGE SPACE, THIS OCCURS EVERYWHERE.
THIS OCCURS EVERYWHERE.
THIS OCCURS EVERYWHERE. IF YOU ARE TALKING ABOUT DOING
IF YOU ARE TALKING ABOUT DOING
IF YOU ARE TALKING ABOUT DOING SEARCH, EVERYBODY HAS HEARD OF
SEARCH, EVERYBODY HAS HEARD OF
SEARCH, EVERYBODY HAS HEARD OF KNOWLEDGE GRAPH, WHICH IS A RICH
KNOWLEDGE GRAPH, WHICH IS A RICH
KNOWLEDGE GRAPH, WHICH IS A RICH SOURCE OF INFORMATION WHICH
SOURCE OF INFORMATION WHICH
SOURCE OF INFORMATION WHICH CAP
CAP
CAP CAPTURES.
CAPTURES.
CAPTURES. THE RELATIONSHIP IS ONE IS A
THE RELATIONSHIP IS ONE IS A
THE RELATIONSHIP IS ONE IS A CAPITAL OF THE OTHER.
CAPITAL OF THE OTHER.
CAPITAL OF THE OTHER. RIGHT?
RIGHT?
RIGHT? THESE SORT OF RELATIONSHIPS
THESE SORT OF RELATIONSHIPS
THESE SORT OF RELATIONSHIPS THESE ARE THE KIND OF
THESE ARE THE KIND OF
THESE ARE THE KIND OF RELATIONSHIPS WHICH ALREADY
RELATIONSHIPS WHICH ALREADY
RELATIONSHIPS WHICH ALREADY EXIST IN DAY TO DAY DATA
EXIST IN DAY TO DAY DATA
EXIST IN DAY TO DAY DATA SOURCES.
SOURCES.
SOURCES. WHY NOT LEVERAGE THEM.
WHY NOT LEVERAGE THEM.
WHY NOT LEVERAGE THEM. THAT IS WHAT YOU TRY TO DO WITH
THAT IS WHAT YOU TRY TO DO WITH
THAT IS WHAT YOU TRY TO DO WITH THE NOODLED STRUCTURED LEARNING.
THE NOODLED STRUCTURED LEARNING.
THE NOODLED STRUCTURED LEARNING. BEFORE WE TALK ABOUT WHAT IT
BEFORE WE TALK ABOUT WHAT IT
BEFORE WE TALK ABOUT WHAT IT DOES AND HOW WE DO IT, WHY DO
DOES AND HOW WE DO IT, WHY DO
DOES AND HOW WE DO IT, WHY DO YOU EVEN WANT TO CARE ABOUT IT?
YOU EVEN WANT TO CARE ABOUT IT?
YOU EVEN WANT TO CARE ABOUT IT? RIGHT?
RIGHT?
RIGHT? WHAT DOES IT BENEFIT.
WHAT DOES IT BENEFIT.
WHAT DOES IT BENEFIT. IT ALLOWS YOU TO TAKE THE
IT ALLOWS YOU TO TAKE THE
IT ALLOWS YOU TO TAKE THE STRUCTURE AND USE IT TO TRAIN
STRUCTURE AND USE IT TO TRAIN
STRUCTURE AND USE IT TO TRAIN LECTURES WITH LESS LABEL DATA,
LECTURES WITH LESS LABEL DATA,
LECTURES WITH LESS LABEL DATA, AND THAT'S THE COSTLY PROCESS.
AND THAT'S THE COSTLY PROCESS.
AND THAT'S THE COSTLY PROCESS. EVERY APPLICATION THAT YOU WANT
EVERY APPLICATION THAT YOU WANT
EVERY APPLICATION THAT YOU WANT TO TRAIN, IF YOU HAD TO COLLECT
TO TRAIN, IF YOU HAD TO COLLECT
TO TRAIN, IF YOU HAD TO COLLECT A LOT OF RICH ANNOTATED DATE
A LOT OF RICH ANNOTATED DATE
A LOT OF RICH ANNOTATED DATE WRA.
WRA.
WRA. IF YOU ARE ABLE TO USE A
IF YOU ARE ABLE TO USE A
IF YOU ARE ABLE TO USE A FRAMEWORK LIKE NEURAL STRUCTURED
FRAMEWORK LIKE NEURAL STRUCTURED
FRAMEWORK LIKE NEURAL STRUCTURED LEARNING, THAT AUTOMATICALLY
LEARNING, THAT AUTOMATICALLY
LEARNING, THAT AUTOMATICALLY CAPTURES THE DATA STRUCTURE AND
CAPTURES THE DATA STRUCTURE AND
CAPTURES THE DATA STRUCTURE AND RELATIONSHIP AND WITH MINIMAL
RELATIONSHIP AND WITH MINIMAL
RELATIONSHIP AND WITH MINIMAL SUPERVISION ARE YOU ABLE TO
SUPERVISION ARE YOU ABLE TO
SUPERVISION ARE YOU ABLE TO TRAIN PREDICTIVE SYSTEMS WITH
TRAIN PREDICTIVE SYSTEMS WITH
TRAIN PREDICTIVE SYSTEMS WITH THE SAME ACCURACY, THAT WOULD BE
THE SAME ACCURACY, THAT WOULD BE
THE SAME ACCURACY, THAT WOULD BE A HUGE THING.
A HUGE THING.
A HUGE THING. WHO WOULDN'T WANT THAT, RIGHT?
WHO WOULDN'T WANT THAT, RIGHT?
WHO WOULDN'T WANT THAT, RIGHT? THAT'S ONE TYPE OF BENEFIT.
THAT'S ONE TYPE OF BENEFIT.
THAT'S ONE TYPE OF BENEFIT. ANOTHER ONE IS THAT TYPICALLY
ANOTHER ONE IS THAT TYPICALLY
ANOTHER ONE IS THAT TYPICALLY WHEN YOU DEPLOY THESE SYSTEMS IN
WHEN YOU DEPLOY THESE SYSTEMS IN
WHEN YOU DEPLOY THESE SYSTEMS IN PRACTICE IN REAL WORLD
PRACTICE IN REAL WORLD
PRACTICE IN REAL WORLD APPLICATIONS, BE YOU WANT THE
APPLICATIONS, BE YOU WANT THE
APPLICATIONS, BE YOU WANT THE SYSTEMS OR NETWORKS TO BE
SYSTEMS OR NETWORKS TO BE
SYSTEMS OR NETWORKS TO BE REBUST.
REBUST.
REBUST. THAT MEANS YOU TRAIN THEM ONCE.
THAT MEANS YOU TRAIN THEM ONCE.
THAT MEANS YOU TRAIN THEM ONCE. YOU DON'T WANT -- IF THE INPUT
YOU DON'T WANT -- IF THE INPUT
YOU DON'T WANT -- IF THE INPUT DISTRIBUTION CHANGES OR DATA
DISTRIBUTION CHANGES OR DATA
DISTRIBUTION CHANGES OR DATA SUDDENLY CHANGES OR SOMEBODY,
SUDDENLY CHANGES OR SOMEBODY,
SUDDENLY CHANGES OR SOMEBODY, YOU KNOW -- SUDDENLY THEY FLIP
YOU KNOW -- SUDDENLY THEY FLIP
YOU KNOW -- SUDDENLY THEY FLIP THE PREDICTIONS AND GO
THE PREDICTIONS AND GO
THE PREDICTIONS AND GO BONNINGERS, RIGHT?
BONNINGERS, RIGHT?
BONNINGERS, RIGHT? THIS IS ANOTHER BENEFIT WHERE IF
THIS IS ANOTHER BENEFIT WHERE IF
THIS IS ANOTHER BENEFIT WHERE IF YOU USE NEURAL STRUCTURED
YOU USE NEURAL STRUCTURED
YOU USE NEURAL STRUCTURED LEARNING, YOU CAN ACTUALLY
LEARNING, YOU CAN ACTUALLY
LEARNING, YOU CAN ACTUALLY IMPROVE THE QUALITY OF THE
IMPROVE THE QUALITY OF THE
IMPROVE THE QUALITY OF THE NETWORK AND ALSO THE ROBUSTNESS
NETWORK AND ALSO THE ROBUSTNESS
NETWORK AND ALSO THE ROBUSTNESS OF THE NETWORK.
OF THE NETWORK.
OF THE NETWORK. LET ME DIVE A LITTLE DEEPER AND
LET ME DIVE A LITTLE DEEPER AND
LET ME DIVE A LITTLE DEEPER AND GIVE YOU A LITTLE MORE, YOU
GIVE YOU A LITTLE MORE, YOU
GIVE YOU A LITTLE MORE, YOU KNOW, INSIGHT INTO, RIGHT?
KNOW, INSIGHT INTO, RIGHT?
KNOW, INSIGHT INTO, RIGHT? TAKE DOCUMENT CLASSIFICATION AS
TAKE DOCUMENT CLASSIFICATION AS
TAKE DOCUMENT CLASSIFICATION AS AN EXAMPLE.
AN EXAMPLE.
AN EXAMPLE. SO I'LL GIVE YOU PROBABLY SOME
SO I'LL GIVE YOU PROBABLY SOME
SO I'LL GIVE YOU PROBABLY SOME EXAMPLE THAT YOU PROBABLY HAVE
EXAMPLE THAT YOU PROBABLY HAVE
EXAMPLE THAT YOU PROBABLY HAVE AT YOUR HOME; RIGHT?
AT YOUR HOME; RIGHT?
AT YOUR HOME; RIGHT? IMAGINE YOU HAVE A CATALOG OR A
IMAGINE YOU HAVE A CATALOG OR A
IMAGINE YOU HAVE A CATALOG OR A LIBRARY OF BOOKS IN YOUR HOME.
LIBRARY OF BOOKS IN YOUR HOME.
LIBRARY OF BOOKS IN YOUR HOME. THESE ARE DIGITIZED CONAT THE
THESE ARE DIGITIZED CONAT THE
THESE ARE DIGITIZED CONAT THE PARTICULAR TIME, AND YOU WANT TO
PARTICULAR TIME, AND YOU WANT TO
PARTICULAR TIME, AND YOU WANT TO NEATLY ARRANGE THEM INTO
NEATLY ARRANGE THEM INTO
NEATLY ARRANGE THEM INTO SPECIFIC TOPICS OR CATEGORIES.
SPECIFIC TOPICS OR CATEGORIES.
SPECIFIC TOPICS OR CATEGORIES. NOW, ONE PERSON MIGHT WANT TO
NOW, ONE PERSON MIGHT WANT TO
NOW, ONE PERSON MIGHT WANT TO CATEGORIZE THEM BASED ON THE
CATEGORIZE THEM BASED ON THE
CATEGORIZE THEM BASED ON THE GENRE.
GENRE.
GENRE. A DIFFERENT PERSON MIGHT SAY,
A DIFFERENT PERSON MIGHT SAY,
A DIFFERENT PERSON MIGHT SAY, OH, I WANT, LIKE, IT TO BELONG
OH, I WANT, LIKE, IT TO BELONG
OH, I WANT, LIKE, IT TO BELONG TO, YOU KNOW, THE SAME PERIOD.
TO, YOU KNOW, THE SAME PERIOD.
TO, YOU KNOW, THE SAME PERIOD. A THIRD PERSON MIGHT SAY I WANT
A THIRD PERSON MIGHT SAY I WANT
A THIRD PERSON MIGHT SAY I WANT TO CAPTURE DIFFERENT BOOKS THAT
TO CAPTURE DIFFERENT BOOKS THAT
TO CAPTURE DIFFERENT BOOKS THAT HAVE THE SAME WORDS THAT APPEAR
HAVE THE SAME WORDS THAT APPEAR
HAVE THE SAME WORDS THAT APPEAR IN THE SAME -- BY THE SAME
IN THE SAME -- BY THE SAME
IN THE SAME -- BY THE SAME AUTHOR OR BASED ON SOME CERTAIN
AUTHOR OR BASED ON SOME CERTAIN
AUTHOR OR BASED ON SOME CERTAIN ASPECT, LIKE THERE IS LIKE A
ASPECT, LIKE THERE IS LIKE A
ASPECT, LIKE THERE IS LIKE A PARTICULAR PLOT TWIST THAT YOU
PARTICULAR PLOT TWIST THAT YOU
PARTICULAR PLOT TWIST THAT YOU HAVE THAT IS CAPTURED IN THE
HAVE THAT IS CAPTURED IN THE
HAVE THAT IS CAPTURED IN THE DIFFERENT BOOKS, AND NOW I WANT
DIFFERENT BOOKS, AND NOW I WANT
DIFFERENT BOOKS, AND NOW I WANT TO ARRANGE THEM BASED ON THAT.
TO ARRANGE THEM BASED ON THAT.
TO ARRANGE THEM BASED ON THAT. YOU CAN THINK OF NOT EVERYBODY'S
YOU CAN THINK OF NOT EVERYBODY'S
YOU CAN THINK OF NOT EVERYBODY'S NEEDS ARE THE SAME.
NEEDS ARE THE SAME.
NEEDS ARE THE SAME. ARE YOU GOING TO COLLECT ENOUGH
ARE YOU GOING TO COLLECT ENOUGH
ARE YOU GOING TO COLLECT ENOUGH LABEL DATA FOR EACH OF THOSE
LABEL DATA FOR EACH OF THOSE
LABEL DATA FOR EACH OF THOSE TASKS, YOU KNOW, TO CREATE A
TASKS, YOU KNOW, TO CREATE A
TASKS, YOU KNOW, TO CREATE A NETWORK THAT CAN HAVE VERY HAYEK
NETWORK THAT CAN HAVE VERY HAYEK
NETWORK THAT CAN HAVE VERY HAYEK ARE ASSY AND CLASSIFY THEM INTO
ARE ASSY AND CLASSIFY THEM INTO
ARE ASSY AND CLASSIFY THEM INTO THE GENRE?
THE GENRE?
THE GENRE? PROBABLY NOT.
PROBABLY NOT.
PROBABLY NOT. ON THE ONE HAND YOU HAVE PLENTY
ON THE ONE HAND YOU HAVE PLENTY
ON THE ONE HAND YOU HAVE PLENTY OF DATA, THE RAW BOOK CONTENT
OF DATA, THE RAW BOOK CONTENT
OF DATA, THE RAW BOOK CONTENT THAT'S AVAILABLE TO YOU FOR RAW
THAT'S AVAILABLE TO YOU FOR RAW
THAT'S AVAILABLE TO YOU FOR RAW NEWS ARTICLES.
NEWS ARTICLES.
NEWS ARTICLES. THERE'S PLENTY OF RAW TEXT
THERE'S PLENTY OF RAW TEXT
THERE'S PLENTY OF RAW TEXT AVAILABLE TO YOU, BUT IT'S HARD
AVAILABLE TO YOU, BUT IT'S HARD
AVAILABLE TO YOU, BUT IT'S HARD TO CONSTRUCT THIS LABEL
TO CONSTRUCT THIS LABEL
TO CONSTRUCT THIS LABEL ANNOTATIVEL.
ANNOTATIVEL.
ANNOTATIVEL. WE CAN USE STRUCTURE.
WE CAN USE STRUCTURE.
WE CAN USE STRUCTURE. AGAIN, I WILL TELL YOU WHAT THE
AGAIN, I WILL TELL YOU WHAT THE
AGAIN, I WILL TELL YOU WHAT THE STRUCTURE MEANS, BUT PAIR IT
STRUCTURE MEANS, BUT PAIR IT
STRUCTURE MEANS, BUT PAIR IT WITH A FEW LABEL EXAMPLES, AND
WITH A FEW LABEL EXAMPLES, AND
WITH A FEW LABEL EXAMPLES, AND THEN FRAME A NETWORK THAT ALMOST
THEN FRAME A NETWORK THAT ALMOST
THEN FRAME A NETWORK THAT ALMOST AS GOOD AS THE NETWORK THAT IS
AS GOOD AS THE NETWORK THAT IS
AS GOOD AS THE NETWORK THAT IS TRAINED AND MILLIONS OF
TRAINED AND MILLIONS OF
TRAINED AND MILLIONS OF EXAMPLES.
EXAMPLES.
EXAMPLES. IMAGINE YOU COULD -- FOR YOUR
IMAGINE YOU COULD -- FOR YOUR
IMAGINE YOU COULD -- FOR YOUR APPLICATION -- ONLY USE 5% OR
APPLICATION -- ONLY USE 5% OR
APPLICATION -- ONLY USE 5% OR 10% OF THE LABEL DATA AND TRAIN
10% OF THE LABEL DATA AND TRAIN
10% OF THE LABEL DATA AND TRAIN AS GOOD A CLASSIFIER OR LIKE A
AS GOOD A CLASSIFIER OR LIKE A
AS GOOD A CLASSIFIER OR LIKE A PREDICTION SYSTEM.
PREDICTION SYSTEM.
PREDICTION SYSTEM. I'M GOING TO GIVE YOU A SOLID
I'M GOING TO GIVE YOU A SOLID
I'M GOING TO GIVE YOU A SOLID REFERENCE HERE, AND WE'RE GOING
REFERENCE HERE, AND WE'RE GOING
REFERENCE HERE, AND WE'RE GOING TO TALK MORE ABOUT THAT AS WELL.
TO TALK MORE ABOUT THAT AS WELL.
TO TALK MORE ABOUT THAT AS WELL. THERE IS A HANDS-ON TUTORIAL FOR
THERE IS A HANDS-ON TUTORIAL FOR
THERE IS A HANDS-ON TUTORIAL FOR THE EXACT EXAMPLE THAT I TOLD
THE EXACT EXAMPLE THAT I TOLD
THE EXACT EXAMPLE THAT I TOLD YOU.
YOU.
YOU. DOCUMENT CLASSIFICATION WITH A
DOCUMENT CLASSIFICATION WITH A
DOCUMENT CLASSIFICATION WITH A GRAPH, AND YOU CAN GO TO THE
GRAPH, AND YOU CAN GO TO THE
GRAPH, AND YOU CAN GO TO THE NEURAL STRUCTURED LEARNING
NEURAL STRUCTURED LEARNING
NEURAL STRUCTURED LEARNING WEBSITE AND TRY THIS OUT FOR
WEBSITE AND TRY THIS OUT FOR
WEBSITE AND TRY THIS OUT FOR YOURSELF.
YOURSELF.
YOURSELF. ALL WITHIN JUST A FEW LINES OF
ALL WITHIN JUST A FEW LINES OF
ALL WITHIN JUST A FEW LINES OF CODE.
CODE.
CODE. ALL YOU NEED TO DO IS CONSTRUCT
ALL YOU NEED TO DO IS CONSTRUCT
ALL YOU NEED TO DO IS CONSTRUCT THE DATA IN THE FORMAT, AND WITH
THE DATA IN THE FORMAT, AND WITH
THE DATA IN THE FORMAT, AND WITH A FEW LINES OF CODE, YOU SHOULD
A FEW LINES OF CODE, YOU SHOULD
A FEW LINES OF CODE, YOU SHOULD BE ABLE TO RUN THE SYSTEM
BE ABLE TO RUN THE SYSTEM
BE ABLE TO RUN THE SYSTEM END-TO-END.
END-TO-END.
END-TO-END. WE HAVE SEEN THE CAPACITY TO
WE HAVE SEEN THE CAPACITY TO
WE HAVE SEEN THE CAPACITY TO TRAIN QUALITY MODELS, ESPECIALLY
TRAIN QUALITY MODELS, ESPECIALLY
TRAIN QUALITY MODELS, ESPECIALLY AS THE NUMBER OF PERIMETERS, BUT
AS THE NUMBER OF PERIMETERS, BUT
AS THE NUMBER OF PERIMETERS, BUT ROBUSTNESS IS AN IMPORTANT
ROBUSTNESS IS AN IMPORTANT
ROBUSTNESS IS AN IMPORTANT CONCEPT, AND IT'S IMPORTANT WHEN
CONCEPT, AND IT'S IMPORTANT WHEN
CONCEPT, AND IT'S IMPORTANT WHEN WE DEPLOY THIS TO REAL WORLD
WE DEPLOY THIS TO REAL WORLD
WE DEPLOY THIS TO REAL WORLD SCENARIOS.
SCENARIOS.
SCENARIOS. FOR EXAMPLE, IF YOU HAVE AN
FOR EXAMPLE, IF YOU HAVE AN
FOR EXAMPLE, IF YOU HAVE AN IMAGE RECOGNITION SYSTEM AND
IMAGE RECOGNITION SYSTEM AND
IMAGE RECOGNITION SYSTEM AND SUDDENLY THE NET WORLD WAR I,
SUDDENLY THE NET WORLD WAR I,
SUDDENLY THE NET WORLD WAR I, THIS IS NOT A GOOD THING, AND
THIS IS NOT A GOOD THING, AND
THIS IS NOT A GOOD THING, AND THIS IS JUST -- NOT JUST
THIS IS JUST -- NOT JUST
THIS IS JUST -- NOT JUST CLASSIFICATION.
CLASSIFICATION.
CLASSIFICATION. IT COULD BE FOR TEXT OR ANY KIND
IT COULD BE FOR TEXT OR ANY KIND
IT COULD BE FOR TEXT OR ANY KIND OF SENSOR DATA.
OF SENSOR DATA.
OF SENSOR DATA. HERE'S AN EXAMPLE.
HERE'S AN EXAMPLE.
HERE'S AN EXAMPLE. TAKE THE IMAGE ON THE LEFT.
TAKE THE IMAGE ON THE LEFT.
TAKE THE IMAGE ON THE LEFT. WHAT DO YOU THINK IT IS?
WHAT DO YOU THINK IT IS?
WHAT DO YOU THINK IT IS? IT'S A PANDA.
IT'S A PANDA.
IT'S A PANDA. TAKE THE IMAGE ON THE RIGHT.
TAKE THE IMAGE ON THE RIGHT.
TAKE THE IMAGE ON THE RIGHT. WHAT DO YOU THINK IT IS?
WHAT DO YOU THINK IT IS?
WHAT DO YOU THINK IT IS? ANYONE WHO DISAGREES IT'S A
ANYONE WHO DISAGREES IT'S A
ANYONE WHO DISAGREES IT'S A PANDA?
PANDA?
PANDA? BASICALLY YOUR NEURAL NETWORK IS
BASICALLY YOUR NEURAL NETWORK IS
BASICALLY YOUR NEURAL NETWORK IS SAYING BOTH OF THESE IMAGES ARE
SAYING BOTH OF THESE IMAGES ARE
SAYING BOTH OF THESE IMAGES ARE PANDAS, BUT THAT'S NOT WHAT
PANDAS, BUT THAT'S NOT WHAT
PANDAS, BUT THAT'S NOT WHAT HAPPENS IF YOU ACTUALLY CREATE A
HAPPENS IF YOU ACTUALLY CREATE A
HAPPENS IF YOU ACTUALLY CREATE A NEURAL NETWORK.
NEURAL NETWORK.
NEURAL NETWORK. THE REASON IT HAPPENS IS YOU
THE REASON IT HAPPENS IS YOU
THE REASON IT HAPPENS IS YOU ZOOM IN REALLY CLOSE.
ZOOM IN REALLY CLOSE.
ZOOM IN REALLY CLOSE. THE SECOND IMAGE IS AN
THE SECOND IMAGE IS AN
THE SECOND IMAGE IS AN ADVERSARIAL EXAMPLE.
ADVERSARIAL EXAMPLE.
ADVERSARIAL EXAMPLE. IT'S ADDING NOISE TO THE
IT'S ADDING NOISE TO THE
IT'S ADDING NOISE TO THE ORIGINAL IMAGE, BUT THESE ARE SO
ORIGINAL IMAGE, BUT THESE ARE SO
ORIGINAL IMAGE, BUT THESE ARE SO TINY.
TINY.
TINY. IT'S IMPERCEIVABLE FOR THE HUMAN
IT'S IMPERCEIVABLE FOR THE HUMAN
IT'S IMPERCEIVABLE FOR THE HUMAN EYE, BUT THE NETWORK BASED ON
EYE, BUT THE NETWORK BASED ON
EYE, BUT THE NETWORK BASED ON THE PIXELS FLIPS IT COMPLETELY.
THE PIXELS FLIPS IT COMPLETELY.
THE PIXELS FLIPS IT COMPLETELY. IMAGINE THIS HAPPENING IN AWE
IMAGINE THIS HAPPENING IN AWE
IMAGINE THIS HAPPENING IN AWE LIVE SYSTEM.
LIVE SYSTEM.
LIVE SYSTEM. YOU WANT THE NETWORKS TO BE
YOU WANT THE NETWORKS TO BE
YOU WANT THE NETWORKS TO BE REBUST.
REBUST.
REBUST. THIS IS WHERE AN ASSAULT COMES
THIS IS WHERE AN ASSAULT COMES
THIS IS WHERE AN ASSAULT COMES AGAIN, AND, AGAIN, THEY'RE GOING
AGAIN, AND, AGAIN, THEY'RE GOING
AGAIN, AND, AGAIN, THEY'RE GOING TO USE THE SAME CONCEPT.
TO USE THE SAME CONCEPT.
TO USE THE SAME CONCEPT. USE THE STRUCTURE IN THE DATA TO
USE THE STRUCTURE IN THE DATA TO
USE THE STRUCTURE IN THE DATA TO TRAIN MORROW BUST MODELS, AND
TRAIN MORROW BUST MODELS, AND
TRAIN MORROW BUST MODELS, AND HERE THE STRUCTURE IS LIKELY
HERE THE STRUCTURE IS LIKELY
HERE THE STRUCTURE IS LIKELY DIFFERENT FROM THE ONE THAT I
DIFFERENT FROM THE ONE THAT I
DIFFERENT FROM THE ONE THAT I MENTIONED EARLIER.
MENTIONED EARLIER.
MENTIONED EARLIER. EARLIER WE'RE TALKING ABOUT
EARLIER WE'RE TALKING ABOUT
EARLIER WE'RE TALKING ABOUT MODELIZED GRAPHS.
MODELIZED GRAPHS.
MODELIZED GRAPHS. HERE WE ARE GOING TO CONSTRUCT
HERE WE ARE GOING TO CONSTRUCT
HERE WE ARE GOING TO CONSTRUCT THAT STRUCTURE.
THAT STRUCTURE.
THAT STRUCTURE. WE TAKE THE ORIGINAL IMAGE, AND
WE TAKE THE ORIGINAL IMAGE, AND
WE TAKE THE ORIGINAL IMAGE, AND WE ALSO GENERATE A ADVERSARIAL
WE ALSO GENERATE A ADVERSARIAL
WE ALSO GENERATE A ADVERSARIAL EXAMPLE FOR THE IMAGE.
EXAMPLE FOR THE IMAGE.
EXAMPLE FOR THE IMAGE. THESE TWO IMAGES ARE JOINED VIA
THESE TWO IMAGES ARE JOINED VIA
THESE TWO IMAGES ARE JOINED VIA A LINK.
A LINK.
A LINK. THERE'S A RELATIONSHIP BETWEEN
THERE'S A RELATIONSHIP BETWEEN
THERE'S A RELATIONSHIP BETWEEN THEM.
THEM.
THEM. THE DIFFERENCE IS THIS IS
THE DIFFERENCE IS THIS IS
THE DIFFERENCE IS THIS IS DYNAMICALLY GENERATED USING THE
DYNAMICALLY GENERATED USING THE
DYNAMICALLY GENERATED USING THE LEARNING PROCESS AS OPPOSED TO
LEARNING PROCESS AS OPPOSED TO
LEARNING PROCESS AS OPPOSED TO THE EARLIER CASE WHERE SOMEBODY
THE EARLIER CASE WHERE SOMEBODY
THE EARLIER CASE WHERE SOMEBODY GIVES YOU A KNOWLEDGE GRAPH OR
GIVES YOU A KNOWLEDGE GRAPH OR
GIVES YOU A KNOWLEDGE GRAPH OR THE STRUCTURED SIGNALS OR YOU
THE STRUCTURED SIGNALS OR YOU
THE STRUCTURED SIGNALS OR YOU CAN STRIKE THEM FROM A DATA
CAN STRIKE THEM FROM A DATA
CAN STRIKE THEM FROM A DATA SOURCE.
SOURCE.
SOURCE. WHAT WE TRY TO DO HERE IS USING
WHAT WE TRY TO DO HERE IS USING
WHAT WE TRY TO DO HERE IS USING THIS STRUCTURE WE ALREADY KNOW
THIS STRUCTURE WE ALREADY KNOW
THIS STRUCTURE WE ALREADY KNOW THE LABEL FOR THE FIRST IMAGE IS
THE LABEL FOR THE FIRST IMAGE IS
THE LABEL FOR THE FIRST IMAGE IS A PANDA.
A PANDA.
A PANDA. WE FORCE THE NETWORK TO LEARN
WE FORCE THE NETWORK TO LEARN
WE FORCE THE NETWORK TO LEARN THAT THE THIRD IMAGE ALSO SHOULD
THAT THE THIRD IMAGE ALSO SHOULD
THAT THE THIRD IMAGE ALSO SHOULD BE CLASSIFIED AS A PANDA.
BE CLASSIFIED AS A PANDA.
BE CLASSIFIED AS A PANDA. THAT'S AT A VERY HIGH LEVEL OF
THAT'S AT A VERY HIGH LEVEL OF
THAT'S AT A VERY HIGH LEVEL OF HOW THIS WORKS.
HOW THIS WORKS.
HOW THIS WORKS. AGAIN, IF YOU WANT TO TRY THIS
AGAIN, IF YOU WANT TO TRY THIS
AGAIN, IF YOU WANT TO TRY THIS OUT FOR ANY NETWORK OR IN ANY
OUT FOR ANY NETWORK OR IN ANY
OUT FOR ANY NETWORK OR IN ANY APPLICATION, YOU CAN GO.
APPLICATION, YOU CAN GO.
APPLICATION, YOU CAN GO. THERE'S A TUTORIAL THAT ALLOWS
THERE'S A TUTORIAL THAT ALLOWS
THERE'S A TUTORIAL THAT ALLOWS YOU TO DO THIS LOOSELY API.
YOU TO DO THIS LOOSELY API.
YOU TO DO THIS LOOSELY API. NSL ENABLES BOTH THE GRAPH --
NSL ENABLES BOTH THE GRAPH --
NSL ENABLES BOTH THE GRAPH -- NEURAL GRAPH TYPE OF LEARNING
NEURAL GRAPH TYPE OF LEARNING
NEURAL GRAPH TYPE OF LEARNING AND THE ADVERSARIAL TYPE OF
AND THE ADVERSARIAL TYPE OF
AND THE ADVERSARIAL TYPE OF LEARNING, AND YOU CAN GO TO THE
LEARNING, AND YOU CAN GO TO THE
LEARNING, AND YOU CAN GO TO THE WEBSITE AND RUN THROUGH THE CODE
WEBSITE AND RUN THROUGH THE CODE
WEBSITE AND RUN THROUGH THE CODE EXAMPLE, ALL WITH JUST A FEW
EXAMPLE, ALL WITH JUST A FEW
EXAMPLE, ALL WITH JUST A FEW LINES OF CODE.
LINES OF CODE.
LINES OF CODE. YOU'LL SEE MORE DETAILS LATER IN
YOU'LL SEE MORE DETAILS LATER IN
YOU'LL SEE MORE DETAILS LATER IN THE TALK.
THE TALK.
THE TALK. THIS IS HOW WEPTD WE WOULD WANT
THIS IS HOW WEPTD WE WOULD WANT
THIS IS HOW WEPTD WE WOULD WANT THE FRAMEWORK LIKE NSL, AND THE
THE FRAMEWORK LIKE NSL, AND THE
THE FRAMEWORK LIKE NSL, AND THE POWER OF USING IT TO ENABLE
POWER OF USING IT TO ENABLE
POWER OF USING IT TO ENABLE MORROW BUST NETWORKS AND ALSO
MORROW BUST NETWORKS AND ALSO
MORROW BUST NETWORKS AND ALSO BUILD NETWORKS THAT CAN BE
BUILD NETWORKS THAT CAN BE
BUILD NETWORKS THAT CAN BE TRAINED WITH MINIMAL
TRAINED WITH MINIMAL
TRAINED WITH MINIMAL SUPERVISION.
SUPERVISION.
SUPERVISION. THESE ARE VERY, VERY HANDY WHEN
THESE ARE VERY, VERY HANDY WHEN
THESE ARE VERY, VERY HANDY WHEN YOU WANT TO BUILD, YOU KNOW,
YOU WANT TO BUILD, YOU KNOW,
YOU WANT TO BUILD, YOU KNOW, APPLICATIONS ON THE FLY, AND
APPLICATIONS ON THE FLY, AND
APPLICATIONS ON THE FLY, AND VERY, VERY CUSTOM APPLICATION
VERY, VERY CUSTOM APPLICATION
VERY, VERY CUSTOM APPLICATION THAT IS DO NOT SPREAD THE
THAT IS DO NOT SPREAD THE
THAT IS DO NOT SPREAD THE REGULAR MOLD.
REGULAR MOLD.
REGULAR MOLD. LET US DIVE DEEPER INTO HOW WE
LET US DIVE DEEPER INTO HOW WE
LET US DIVE DEEPER INTO HOW WE KNOW WHAT THE FRAMEWORK IS
KNOW WHAT THE FRAMEWORK IS
KNOW WHAT THE FRAMEWORK IS DOING.
DOING.
DOING. STRUCTURE IS GOING TO BE USED TO
STRUCTURE IS GOING TO BE USED TO
STRUCTURE IS GOING TO BE USED TO MODEL AND AS AN INPUT AND
MODEL AND AS AN INPUT AND
MODEL AND AS AN INPUT AND NETWORK.
NETWORK.
NETWORK. WE CALL THIS PARADIGM NEURAL
WE CALL THIS PARADIGM NEURAL
WE CALL THIS PARADIGM NEURAL GRAPH LEARNING.
GRAPH LEARNING.
GRAPH LEARNING. THE CORE IDEA IS THAT SHE MIGHT
THE CORE IDEA IS THAT SHE MIGHT
THE CORE IDEA IS THAT SHE MIGHT ASK, BY THE WAY, AT THIS POINT,
ASK, BY THE WAY, AT THIS POINT,
ASK, BY THE WAY, AT THIS POINT, LIKE, WHERE IS THE GRAPH COMING
LIKE, WHERE IS THE GRAPH COMING
LIKE, WHERE IS THE GRAPH COMING FROM IN THIS SETTING?
FROM IN THIS SETTING?
FROM IN THIS SETTING? IN SOME CASES IT MIGHT BE GIVEN
IN SOME CASES IT MIGHT BE GIVEN
IN SOME CASES IT MIGHT BE GIVEN TO YOU.
TO YOU.
TO YOU. YOU HAVE TO SAY IT WOULD PROVIDE
YOU HAVE TO SAY IT WOULD PROVIDE
YOU HAVE TO SAY IT WOULD PROVIDE TOOLS.
TOOLS.
TOOLS. AGAIN, YOU WILL HEAR MORE ABOUT
AGAIN, YOU WILL HEAR MORE ABOUT
AGAIN, YOU WILL HEAR MORE ABOUT THIS THAT ALLOW YOU TO CONSTRUCT
THIS THAT ALLOW YOU TO CONSTRUCT
THIS THAT ALLOW YOU TO CONSTRUCT THESE GRAPHS FROM, YOU KNOW,
THESE GRAPHS FROM, YOU KNOW,
THESE GRAPHS FROM, YOU KNOW, SOURCES OF DATA LIKE WHAT
SOURCES OF DATA LIKE WHAT
SOURCES OF DATA LIKE WHAT EMBEDDINGS AND IMAGE EMBEDDINGS.
EMBEDDINGS AND IMAGE EMBEDDINGS.
EMBEDDINGS AND IMAGE EMBEDDINGS. THE PHOTOGRAPH HERE IS THE
THE PHOTOGRAPH HERE IS THE
THE PHOTOGRAPH HERE IS THE NETWORK IS GOING TO BE FORCED TO
NETWORK IS GOING TO BE FORCED TO
NETWORK IS GOING TO BE FORCED TO JOINTLY OPTIMIZE BOTH THE
JOINTLY OPTIMIZE BOTH THE
JOINTLY OPTIMIZE BOTH THE FEATURE INPUT AND THE STRUCTURED
FEATURE INPUT AND THE STRUCTURED
FEATURE INPUT AND THE STRUCTURED SIGNAL SIMULTANEOUSLY.
SIGNAL SIMULTANEOUSLY.
SIGNAL SIMULTANEOUSLY. LET'S SEE HOW THAT HAPPENS,
LET'S SEE HOW THAT HAPPENS,
LET'S SEE HOW THAT HAPPENS, LIKE, DIVING IN DEEPER IF YOU
LIKE, DIVING IN DEEPER IF YOU
LIKE, DIVING IN DEEPER IF YOU WERE TRYING TO LOOK AT WHAT
WERE TRYING TO LOOK AT WHAT
WERE TRYING TO LOOK AT WHAT EXACTLY THE NETWORK IS LEARNING.
EXACTLY THE NETWORK IS LEARNING.
EXACTLY THE NETWORK IS LEARNING. EVERY NETWORK IS TRYING TO
EVERY NETWORK IS TRYING TO
EVERY NETWORK IS TRYING TO OPTIMIZE SOME LOSS.
OPTIMIZE SOME LOSS.
OPTIMIZE SOME LOSS. IN IMAGE CLASSIFICATION, WHAT IS
IN IMAGE CLASSIFICATION, WHAT IS
IN IMAGE CLASSIFICATION, WHAT IS THE LOSS WHEN YOU TAKE THE
THE LOSS WHEN YOU TAKE THE
THE LOSS WHEN YOU TAKE THE PIXELS, PASS THROUGH THE
PIXELS, PASS THROUGH THE
PIXELS, PASS THROUGH THE NETWORK, GET PREDICTIONS, WHERE
NETWORK, GET PREDICTIONS, WHERE
NETWORK, GET PREDICTIONS, WHERE IS THE ERROR BETWEEN THE
IS THE ERROR BETWEEN THE
IS THE ERROR BETWEEN THE PREDICTIONS AND THE TRUE LABEL.
PREDICTIONS AND THE TRUE LABEL.
PREDICTIONS AND THE TRUE LABEL. IN NSL, THE NEURAL GRAPH SETTING
IN NSL, THE NEURAL GRAPH SETTING
IN NSL, THE NEURAL GRAPH SETTING LEARNING, WE CALL THIS TRAINED
LEARNING, WE CALL THIS TRAINED
LEARNING, WE CALL THIS TRAINED IN THIS MODE NEURAL GRAPH
IN THIS MODE NEURAL GRAPH
IN THIS MODE NEURAL GRAPH MACHINE.
MACHINE.
MACHINE. WE TRY TO OPTIMIZE TWO
WE TRY TO OPTIMIZE TWO
WE TRY TO OPTIMIZE TWO COMPONENTS.
COMPONENTS.
COMPONENTS. ONE IS THE STANDARD LOSS, WHICH
ONE IS THE STANDARD LOSS, WHICH
ONE IS THE STANDARD LOSS, WHICH IS IMAGE CLASSIFICATIONS.
IS IMAGE CLASSIFICATIONS.
IS IMAGE CLASSIFICATIONS. THE LOSS INCURRED WHEN YOU PASS
THE LOSS INCURRED WHEN YOU PASS
THE LOSS INCURRED WHEN YOU PASS THE PIXELS THROUGH THE NETWORK
THE PIXELS THROUGH THE NETWORK
THE PIXELS THROUGH THE NETWORK AND THE PREDICTIONS ARE MEASURED
AND THE PREDICTIONS ARE MEASURED
AND THE PREDICTIONS ARE MEASURED IN THE ERROR.
IN THE ERROR.
IN THE ERROR. THE OTHER COMPONENT IS GOING TO
THE OTHER COMPONENT IS GOING TO
THE OTHER COMPONENT IS GOING TO BE BUSINESSED ON THE STRUCTURED
BE BUSINESSED ON THE STRUCTURED
BE BUSINESSED ON THE STRUCTURED SIGNAL OR THE GRAPH THAT YOU
SIGNAL OR THE GRAPH THAT YOU
SIGNAL OR THE GRAPH THAT YOU PROVIDED.
PROVIDED.
PROVIDED. WHERE THAT COMES IS IF I ARE AN
WHERE THAT COMES IS IF I ARE AN
WHERE THAT COMES IS IF I ARE AN IMAGE THAT LOOKED LIKE THE PIT
IMAGE THAT LOOKED LIKE THE PIT
IMAGE THAT LOOKED LIKE THE PIT BULL DOG, THAT'S LABELED AS A
BULL DOG, THAT'S LABELED AS A
BULL DOG, THAT'S LABELED AS A PIT BULL.
PIT BULL.
PIT BULL. IF I HAVE A DIFFERENT IMAGE,
IF I HAVE A DIFFERENT IMAGE,
IF I HAVE A DIFFERENT IMAGE, WHICH THROUGH MY STRUCTURED
WHICH THROUGH MY STRUCTURED
WHICH THROUGH MY STRUCTURED SIGNAL HAS AN EDGE WITH THE
SIGNAL HAS AN EDGE WITH THE
SIGNAL HAS AN EDGE WITH THE ORIGINAL IMAGE, THEN THE NETWORK
ORIGINAL IMAGE, THEN THE NETWORK
ORIGINAL IMAGE, THEN THE NETWORK IS FORCED TO LEARN THAT THE
IS FORCED TO LEARN THAT THE
IS FORCED TO LEARN THAT THE SOURCE IMAGE AND ITS NEIGHBOR IN
SOURCE IMAGE AND ITS NEIGHBOR IN
SOURCE IMAGE AND ITS NEIGHBOR IN THE GRAPH SHOULD LEARN SIMILAR
THE GRAPH SHOULD LEARN SIMILAR
THE GRAPH SHOULD LEARN SIMILAR REPRESENTATIONS.
REPRESENTATIONS.
REPRESENTATIONS. THAT MEANS YOU'RE TRYING TO SAY
THAT MEANS YOU'RE TRYING TO SAY
THAT MEANS YOU'RE TRYING TO SAY THAT -- NOW THIS IS FLEXIBLE, AS
THAT -- NOW THIS IS FLEXIBLE, AS
THAT -- NOW THIS IS FLEXIBLE, AS YOU CAN MABLG.
YOU CAN MABLG.
YOU CAN MABLG. YOU CAN USUALLY CHANGE THE
YOU CAN USUALLY CHANGE THE
YOU CAN USUALLY CHANGE THE FORMULATIONS.
FORMULATIONS.
FORMULATIONS. THAT MEANS INSTEAD OF A
THAT MEANS INSTEAD OF A
THAT MEANS INSTEAD OF A SUPERVISED LOGS, IF YOU WANT TO
SUPERVISED LOGS, IF YOU WANT TO
SUPERVISED LOGS, IF YOU WANT TO DO UNSUPERVISED LEARNING OR A
DO UNSUPERVISED LEARNING OR A
DO UNSUPERVISED LEARNING OR A VERY DIFFERENT KIND OF LOSS, YOU
VERY DIFFERENT KIND OF LOSS, YOU
VERY DIFFERENT KIND OF LOSS, YOU CAN ACTUALLY CHANGE THE FIRST
CAN ACTUALLY CHANGE THE FIRST
CAN ACTUALLY CHANGE THE FIRST COMPONENT YOU ARE CAN USE IT
COMPONENT YOU ARE CAN USE IT
COMPONENT YOU ARE CAN USE IT XEBDING ON THE APPLICATIONS.
XEBDING ON THE APPLICATIONS.
XEBDING ON THE APPLICATIONS. YOU KNOW, VERY, VERY EASY FOR
YOU KNOW, VERY, VERY EASY FOR
YOU KNOW, VERY, VERY EASY FOR YOU TO HAVE A WIDE RANGE OF
YOU TO HAVE A WIDE RANGE OF
YOU TO HAVE A WIDE RANGE OF APPLICATIONS IN A DIFFERENT
APPLICATIONS IN A DIFFERENT
APPLICATIONS IN A DIFFERENT LEARNING SETTING, LIKE WHETHER
LEARNING SETTING, LIKE WHETHER
LEARNING SETTING, LIKE WHETHER IT'S UNSUPERVISED, SUPERVISED,
IT'S UNSUPERVISED, SUPERVISED,
IT'S UNSUPERVISED, SUPERVISED, OR RANKING OR CLASSIFICATION,
OR RANKING OR CLASSIFICATION,
OR RANKING OR CLASSIFICATION, BUT AT THE SAME TIME YOU WOULD
BUT AT THE SAME TIME YOU WOULD
BUT AT THE SAME TIME YOU WOULD BE ABLE TO PASS INFRASTRUCTURE
BE ABLE TO PASS INFRASTRUCTURE
BE ABLE TO PASS INFRASTRUCTURE IN A SEAMLESS MANNER.
IN A SEAMLESS MANNER.
IN A SEAMLESS MANNER. HERE'S AN EXAMPLE.
HERE'S AN EXAMPLE.
HERE'S AN EXAMPLE. TAKES IMAGE CLASSIFICATION WITH
TAKES IMAGE CLASSIFICATION WITH
TAKES IMAGE CLASSIFICATION WITH SOME SAMPLES, AS I SAID, THE
SOME SAMPLES, AS I SAID, THE
SOME SAMPLES, AS I SAID, THE PIXELS, AND YOU HAVE A
PIXELS, AND YOU HAVE A
PIXELS, AND YOU HAVE A STRUCTURE.
STRUCTURE.
STRUCTURE. IN THIS CASE, THE IMAGES ARE
IN THIS CASE, THE IMAGES ARE
IN THIS CASE, THE IMAGES ARE CONNECTED IN THE GRAPH BASED ON
CONNECTED IN THE GRAPH BASED ON
CONNECTED IN THE GRAPH BASED ON SOME USER AND TRAFFIC SIGNAL OR
SOME USER AND TRAFFIC SIGNAL OR
SOME USER AND TRAFFIC SIGNAL OR BASICALLY AN EXAMPLE AS I SHOWED
BASICALLY AN EXAMPLE AS I SHOWED
BASICALLY AN EXAMPLE AS I SHOWED YOU, LIKE, THEY BELONG TO THE
YOU, LIKE, THEY BELONG TO THE
YOU, LIKE, THEY BELONG TO THE SAME ALBUM OR THERE'S SOME SORT
SAME ALBUM OR THERE'S SOME SORT
SAME ALBUM OR THERE'S SOME SORT OF STRUCTURE TYING THEM
OF STRUCTURE TYING THEM
OF STRUCTURE TYING THEM TOGETHER.
TOGETHER.
TOGETHER. AS YOU MEAN, THIS IS GIVEN TO
AS YOU MEAN, THIS IS GIVEN TO
AS YOU MEAN, THIS IS GIVEN TO YOU.
YOU.
YOU. WE PASS THIS THROUGH THE
WE PASS THIS THROUGH THE
WE PASS THIS THROUGH THE NETWORK.
NETWORK.
NETWORK. BOTH THIS AND IT'S NEIGHBORS ARE
BOTH THIS AND IT'S NEIGHBORS ARE
BOTH THIS AND IT'S NEIGHBORS ARE PASSED SIMULTANEOUSLY THROUGH
PASSED SIMULTANEOUSLY THROUGH
PASSED SIMULTANEOUSLY THROUGH THE SAME NETWORK, AND THE
THE SAME NETWORK, AND THE
THE SAME NETWORK, AND THE NETWORK IS LEARNING TO OPTIMIZE
NETWORK IS LEARNING TO OPTIMIZE
NETWORK IS LEARNING TO OPTIMIZE WITHIN, YOU KNOW, EACH LAYER,
WITHIN, YOU KNOW, EACH LAYER,
WITHIN, YOU KNOW, EACH LAYER, AND THIS IS ALSO CONFIGURED, BY
AND THIS IS ALSO CONFIGURED, BY
AND THIS IS ALSO CONFIGURED, BY THE WAY, TO PUSH THE EMBEDDING
THE WAY, TO PUSH THE EMBEDDING
THE WAY, TO PUSH THE EMBEDDING FOR NEIGHBORS CLOSER TO EACH
FOR NEIGHBORS CLOSER TO EACH
FOR NEIGHBORS CLOSER TO EACH OTHER.
OTHER.
OTHER. THAT MEANS TWO IMAGES THAT ARE
THAT MEANS TWO IMAGES THAT ARE
THAT MEANS TWO IMAGES THAT ARE CONNECTED IN THE GRAPH SMUD
CONNECTED IN THE GRAPH SMUD
CONNECTED IN THE GRAPH SMUD LEARN SIMILAR EMBEDDINGS WHEN
LEARN SIMILAR EMBEDDINGS WHEN
LEARN SIMILAR EMBEDDINGS WHEN PASSED THROUGH THE NETWORK.
PASSED THROUGH THE NETWORK.
PASSED THROUGH THE NETWORK. SIMULTANEOUSY, YOU SHOULD ALSO
SIMULTANEOUSY, YOU SHOULD ALSO
SIMULTANEOUSY, YOU SHOULD ALSO OPTIMIZE THAT THEY SHOULD LEARN
OPTIMIZE THAT THEY SHOULD LEARN
OPTIMIZE THAT THEY SHOULD LEARN THE CORRECT PREDICTION, SO ONE
THE CORRECT PREDICTION, SO ONE
THE CORRECT PREDICTION, SO ONE OF THEM WAS LABEL AS A TIME, AND
OF THEM WAS LABEL AS A TIME, AND
OF THEM WAS LABEL AS A TIME, AND YOU ALSO WANT THE PREDICTION
YOU ALSO WANT THE PREDICTION
YOU ALSO WANT THE PREDICTION ERROR TO BE MINIMAL.
ERROR TO BE MINIMAL.
ERROR TO BE MINIMAL. BOTH HAVE THESE ARE BEING
BOTH HAVE THESE ARE BEING
BOTH HAVE THESE ARE BEING OPTIMIZED JOINTLY.
OPTIMIZED JOINTLY.
OPTIMIZED JOINTLY. HOPEFULLY THIS GIVES YOU AN IDEA
HOPEFULLY THIS GIVES YOU AN IDEA
HOPEFULLY THIS GIVES YOU AN IDEA OF HOW WE USE GRAPH LEARNING OR
OF HOW WE USE GRAPH LEARNING OR
OF HOW WE USE GRAPH LEARNING OR NEURAL GRAPH LEARNING, AND
NEURAL GRAPH LEARNING, AND
NEURAL GRAPH LEARNING, AND ENABLE THIS STRUCTURED LEARNING
ENABLE THIS STRUCTURED LEARNING
ENABLE THIS STRUCTURED LEARNING FRAMEWORK.
FRAMEWORK.
FRAMEWORK. AS I MENTIONED, STRUCTURE CAN
AS I MENTIONED, STRUCTURE CAN
AS I MENTIONED, STRUCTURE CAN COME IN DIFFERENT FORMS.
COME IN DIFFERENT FORMS.
COME IN DIFFERENT FORMS. THAT WAS AN EXPLICIT STRUCTURE
THAT WAS AN EXPLICIT STRUCTURE
THAT WAS AN EXPLICIT STRUCTURE AS A GRAPH INPUT, BUT WE CAN
AS A GRAPH INPUT, BUT WE CAN
AS A GRAPH INPUT, BUT WE CAN ALSO DO IMPLICIT STRUCTURES, AND
ALSO DO IMPLICIT STRUCTURES, AND
ALSO DO IMPLICIT STRUCTURES, AND THIS IS WHERE THE ADVERSARIAL
THIS IS WHERE THE ADVERSARIAL
THIS IS WHERE THE ADVERSARIAL LEARNING TYPE OF PARADIGMS ARE
LEARNING TYPE OF PARADIGMS ARE
LEARNING TYPE OF PARADIGMS ARE ENABLED USING THE NSL FRAMEWORK.
ENABLED USING THE NSL FRAMEWORK.
ENABLED USING THE NSL FRAMEWORK. YOU CREATE NXI PRIME AS AN
YOU CREATE NXI PRIME AS AN
YOU CREATE NXI PRIME AS AN ADVERSARIAL VERSION OF THAT.
ADVERSARIAL VERSION OF THAT.
ADVERSARIAL VERSION OF THAT. THERE IS SOME SORT OF WAIT BASED
THERE IS SOME SORT OF WAIT BASED
THERE IS SOME SORT OF WAIT BASED ON THE BUSINESS, AND THIS IS
ON THE BUSINESS, AND THIS IS
ON THE BUSINESS, AND THIS IS PASSED THROUGH THE NETWORK, AND
PASSED THROUGH THE NETWORK, AND
PASSED THROUGH THE NETWORK, AND THE NETWORK IS FORCED TO WANT ON
THE NETWORK IS FORCED TO WANT ON
THE NETWORK IS FORCED TO WANT ON MIZ BOTH OF THEM TO THE SAME
MIZ BOTH OF THEM TO THE SAME
MIZ BOTH OF THEM TO THE SAME EMBEDDINGS AND REPRESENTATIONS
EMBEDDINGS AND REPRESENTATIONS
EMBEDDINGS AND REPRESENTATIONS INSIDE.
INSIDE.
INSIDE. THESE ARE ALL GREAT.
THESE ARE ALL GREAT.
THESE ARE ALL GREAT. AS I MENTIONED, THIS SORT OF
AS I MENTIONED, THIS SORT OF
AS I MENTIONED, THIS SORT OF OPENS UP A HOST OF NEW KIND OF
OPENS UP A HOST OF NEW KIND OF
OPENS UP A HOST OF NEW KIND OF APPLICATION OR TRAINING CENTERS.
APPLICATION OR TRAINING CENTERS.
APPLICATION OR TRAINING CENTERS. THE BEST PART ABOUT THIS IS IF
THE BEST PART ABOUT THIS IS IF
THE BEST PART ABOUT THIS IS IF YOU ARE THINKING, OH, NOW HOW
YOU ARE THINKING, OH, NOW HOW
YOU ARE THINKING, OH, NOW HOW DOES THIS WORK WITH TRANSFORMERS
DOES THIS WORK WITH TRANSFORMERS
DOES THIS WORK WITH TRANSFORMERS OR DIFFERENT KINDS OF NETWORK
OR DIFFERENT KINDS OF NETWORK
OR DIFFERENT KINDS OF NETWORK STRUCTURE DOESN'T MATTER HERE.
STRUCTURE DOESN'T MATTER HERE.
STRUCTURE DOESN'T MATTER HERE. YOU CAN USE THIS WITH ANY TYPE
YOU CAN USE THIS WITH ANY TYPE
YOU CAN USE THIS WITH ANY TYPE OF NETWORK STRUCTURE.
OF NETWORK STRUCTURE.
OF NETWORK STRUCTURE. THAT'S THE BEST PART.
THAT'S THE BEST PART.
THAT'S THE BEST PART. RNNs, TRANSFORMERS, LIKE -- YOU
RNNs, TRANSFORMERS, LIKE -- YOU
RNNs, TRANSFORMERS, LIKE -- YOU HAVE A COMBINATIONS.
HAVE A COMBINATIONS.
HAVE A COMBINATIONS. IT DOESN'T MATTER.
IT DOESN'T MATTER.
IT DOESN'T MATTER. THESE ARE SORT OF A LEARNING
THESE ARE SORT OF A LEARNING
THESE ARE SORT OF A LEARNING STRATEGY WHERE YOU CAN ACTUALLY
STRATEGY WHERE YOU CAN ACTUALLY
STRATEGY WHERE YOU CAN ACTUALLY BUILD A NETWORK AND ENABLE NSL
BUILD A NETWORK AND ENABLE NSL
BUILD A NETWORK AND ENABLE NSL WORKING ADVERSARIAL AND THE
WORKING ADVERSARIAL AND THE
WORKING ADVERSARIAL AND THE NEURALRAPH SETTING VERY EASILY
NEURALRAPH SETTING VERY EASILY
NEURALRAPH SETTING VERY EASILY WITH VERY FEW LINES OF CODES IN
WITH VERY FEW LINES OF CODES IN
WITH VERY FEW LINES OF CODES IN 2.0, AND TO TELL YOU MORE ABOUT
2.0, AND TO TELL YOU MORE ABOUT
2.0, AND TO TELL YOU MORE ABOUT THAT I'M HANDING IT OVER
THAT I'M HANDING IT OVER
THAT I'M HANDING IT OVER >> ALL RIGHT.
>> ALL RIGHT.
>> ALL RIGHT. THANK YOU, SUJIT.
THANK YOU, SUJIT.
THANK YOU, SUJIT. NEXT, WE ARE GOING TO INTRODUCE
NEXT, WE ARE GOING TO INTRODUCE
NEXT, WE ARE GOING TO INTRODUCE THE LIBRARY TOOLS AND TRAINERS
THE LIBRARY TOOLS AND TRAINERS
THE LIBRARY TOOLS AND TRAINERS PROVIDED BY THE NEURAL
PROVIDED BY THE NEURAL
PROVIDED BY THE NEURAL STRUCTURED LEARNING FRAME WORLD
STRUCTURED LEARNING FRAME WORLD
STRUCTURED LEARNING FRAME WORLD WAR I.
WAR I.
WAR I. EVERYTHING HERE IS COMPATIBLE
EVERYTHING HERE IS COMPATIBLE
EVERYTHING HERE IS COMPATIBLE WITH TENSORFLOW 2.0.
WITH TENSORFLOW 2.0.
WITH TENSORFLOW 2.0. THIS IS THE TRAINING WORKFLOW WE
THIS IS THE TRAINING WORKFLOW WE
THIS IS THE TRAINING WORKFLOW WE JUST MENTIONED PREVIOUSLY.
JUST MENTIONED PREVIOUSLY.
JUST MENTIONED PREVIOUSLY. EVERY SEGMENT IN RED HERE IS A
EVERY SEGMENT IN RED HERE IS A
EVERY SEGMENT IN RED HERE IS A NEW STAT INTRODUCED TO THE
NEW STAT INTRODUCED TO THE
NEW STAT INTRODUCED TO THE WORKFLOW TO TRAIN WITH STRUCTURE
WORKFLOW TO TRAIN WITH STRUCTURE
WORKFLOW TO TRAIN WITH STRUCTURE SIGNALS.
SIGNALS.
SIGNALS. NEURAL STRUCTURE LEARNING
NEURAL STRUCTURE LEARNING
NEURAL STRUCTURE LEARNING PROVIDE LIBRARIES AND TOOLS FOR
PROVIDE LIBRARIES AND TOOLS FOR
PROVIDE LIBRARIES AND TOOLS FOR THESE SETS.
THESE SETS.
THESE SETS. LET'S FIRST TAKE A LOOK AT THE
LET'S FIRST TAKE A LOOK AT THE
LET'S FIRST TAKE A LOOK AT THE LAB PART OF THE WORKFLOW.
LAB PART OF THE WORKFLOW.
LAB PART OF THE WORKFLOW. THE TRAINING SAMPLES IN
THE TRAINING SAMPLES IN
THE TRAINING SAMPLES IN NEIGHBORS FROM THE SAME
NEIGHBORS FROM THE SAME
NEIGHBORS FROM THE SAME NEIGHBORHOOD ARE PACKED TO FORM
NEIGHBORHOOD ARE PACKED TO FORM
NEIGHBORHOOD ARE PACKED TO FORM THE NEW BATCH.
THE NEW BATCH.
THE NEW BATCH. NOTICE THAT IN THE BATCH EACH
NOTICE THAT IN THE BATCH EACH
NOTICE THAT IN THE BATCH EACH TRAINING SAMPLE IS EXTENDED TO
TRAINING SAMPLE IS EXTENDED TO
TRAINING SAMPLE IS EXTENDED TO INCLUDE THE NEIGHBORHOOD
INCLUDE THE NEIGHBORHOOD
INCLUDE THE NEIGHBORHOOD INFORMATION.
INFORMATION.
INFORMATION. TO ACHIEVE THIS IN THE NEURAL
TO ACHIEVE THIS IN THE NEURAL
TO ACHIEVE THIS IN THE NEURAL STRUCTURE LEARNING FRAMEWORK, WE
STRUCTURE LEARNING FRAMEWORK, WE
STRUCTURE LEARNING FRAMEWORK, WE PROVIDE TOOLS SUCH AS GRAPH AND
PROVIDE TOOLS SUCH AS GRAPH AND
PROVIDE TOOLS SUCH AS GRAPH AND PACK NEIGHBORS THAT A USING
PACK NEIGHBORS THAT A USING
PACK NEIGHBORS THAT A USING COULD USE DIRECTLY.
COULD USE DIRECTLY.
COULD USE DIRECTLY. WE ALSO PROVIDE FUNCTIONS THAT
WE ALSO PROVIDE FUNCTIONS THAT
WE ALSO PROVIDE FUNCTIONS THAT USERS COULD INTEGRATE INTO THEIR
USERS COULD INTEGRATE INTO THEIR
USERS COULD INTEGRATE INTO THEIR OWN CUSTOM PIPE LOOIRN, AND YOU
OWN CUSTOM PIPE LOOIRN, AND YOU
OWN CUSTOM PIPE LOOIRN, AND YOU MAY NOTICE NEW GRAPH AND PACK
MAY NOTICE NEW GRAPH AND PACK
MAY NOTICE NEW GRAPH AND PACK NEIGHBORS HERE ARE LISTED BOTH
NEIGHBORS HERE ARE LISTED BOTH
NEIGHBORS HERE ARE LISTED BOTH AS FINERIES AND FUNCTIONS.
AS FINERIES AND FUNCTIONS.
AS FINERIES AND FUNCTIONS. THIS IS NOT A TYPO.
THIS IS NOT A TYPO.
THIS IS NOT A TYPO. THIS MEANS THAT THEY COULD BE
THIS MEANS THAT THEY COULD BE
THIS MEANS THAT THEY COULD BE INVOLVED BOTH AS BINARY OR AS A
INVOLVED BOTH AS BINARY OR AS A
INVOLVED BOTH AS BINARY OR AS A FUNCTION.
FUNCTION.
FUNCTION. NEXT, LET'S TAKE A LOOK AT A
NEXT, LET'S TAKE A LOOK AT A
NEXT, LET'S TAKE A LOOK AT A RIGHT PART OF THE FIGURE.
RIGHT PART OF THE FIGURE.
RIGHT PART OF THE FIGURE. AGAIN, WE PROVIDE LIBRARIES FOR
AGAIN, WE PROVIDE LIBRARIES FOR
AGAIN, WE PROVIDE LIBRARIES FOR THESE NEW STATS INTRODUCED IN
THESE NEW STATS INTRODUCED IN
THESE NEW STATS INTRODUCED IN GRAPH -- BOTH THE TRAINING
GRAPH -- BOTH THE TRAINING
GRAPH -- BOTH THE TRAINING SAMPLE AND ITS NEIGHBOR WILL BE
SAMPLE AND ITS NEIGHBOR WILL BE
SAMPLE AND ITS NEIGHBOR WILL BE FED TO THE NEURAL NETWORK.
FED TO THE NEURAL NETWORK.
FED TO THE NEURAL NETWORK. IN ALL PACK NEIGHBOR FEATURES IS
IN ALL PACK NEIGHBOR FEATURES IS
IN ALL PACK NEIGHBOR FEATURES IS FOR THIS PURPOSE.
FOR THIS PURPOSE.
FOR THIS PURPOSE. IT CAN BE ANY TYPE OF NEURAL
IT CAN BE ANY TYPE OF NEURAL
IT CAN BE ANY TYPE OF NEURAL NETWORK NOT JUST LIMITED TO THE
NETWORK NOT JUST LIMITED TO THE
NETWORK NOT JUST LIMITED TO THE COMMERCIAL.
COMMERCIAL.
COMMERCIAL. THE DIFFERENCE BETWEEN THE
THE DIFFERENCE BETWEEN THE
THE DIFFERENCE BETWEEN THE SAMPLE AND ITS NEIGHBOR IS
SAMPLE AND ITS NEIGHBOR IS
SAMPLE AND ITS NEIGHBOR IS CALCULATED AND ADDED TO THE
CALCULATED AND ADDED TO THE
CALCULATED AND ADDED TO THE FINAL LOSS.
FINAL LOSS.
FINAL LOSS. IN ADDITION, WE ALSO PROVIDE
IN ADDITION, WE ALSO PROVIDE
IN ADDITION, WE ALSO PROVIDE LIBRARIES TO GENERATE THE
LIBRARIES TO GENERATE THE
LIBRARIES TO GENERATE THE ADVERSARIAL NEIGHBORS AS
ADVERSARIAL NEIGHBORS AS
ADVERSARIAL NEIGHBORS AS IMPLICIT STRUCTURE SIGNALS.
IMPLICIT STRUCTURE SIGNALS.
IMPLICIT STRUCTURE SIGNALS. FINALLY, WE ALSO PROVIDE SDPT
FINALLY, WE ALSO PROVIDE SDPT
FINALLY, WE ALSO PROVIDE SDPT ALL THREE TYPES OF MODELS EITHER
ALL THREE TYPES OF MODELS EITHER
ALL THREE TYPES OF MODELS EITHER VIA SEE CONTROVERSIAL OR
VIA SEE CONTROVERSIAL OR
VIA SEE CONTROVERSIAL OR FUNCTIONAL API OR VIA
FUNCTIONAL API OR VIA
FUNCTIONAL API OR VIA SUBCLASSING.
SUBCLASSING.
SUBCLASSING. THE FIRST STEP, IF YOU WANT TO
THE FIRST STEP, IF YOU WANT TO
THE FIRST STEP, IF YOU WANT TO USE NEURAL STRURGTAL LEARNING IS
USE NEURAL STRURGTAL LEARNING IS
USE NEURAL STRURGTAL LEARNING IS TO DO A PIT INSTALL.
TO DO A PIT INSTALL.
TO DO A PIT INSTALL. HERE WE PROVIDE A CODE EXAMPLE
HERE WE PROVIDE A CODE EXAMPLE
HERE WE PROVIDE A CODE EXAMPLE ILLUSTRATING THE API FROM THE
ILLUSTRATING THE API FROM THE
ILLUSTRATING THE API FROM THE NEURAL STRUCTURAL LEARNING
NEURAL STRUCTURAL LEARNING
NEURAL STRUCTURAL LEARNING LIBRARY.
LIBRARY.
LIBRARY. WE FIRST NEED TO READ THE
WE FIRST NEED TO READ THE
WE FIRST NEED TO READ THE TRAINING DATA.
TRAINING DATA.
TRAINING DATA. NOTE THAT THE DATA HERE ARE
NOTE THAT THE DATA HERE ARE
NOTE THAT THE DATA HERE ARE PROCESSED BY THE TOOLS OR
PROCESSED BY THE TOOLS OR
PROCESSED BY THE TOOLS OR FUNCTIONS TO INCORPORATE THE
FUNCTIONS TO INCORPORATE THE
FUNCTIONS TO INCORPORATE THE GRAPHS INTO TRAINING SAMPLES.
GRAPHS INTO TRAINING SAMPLES.
GRAPHS INTO TRAINING SAMPLES. NEXT, THE USER COULD BUILD
NEXT, THE USER COULD BUILD
NEXT, THE USER COULD BUILD CUSTOM MODELS, SUCH AS -- AND
CUSTOM MODELS, SUCH AS -- AND
CUSTOM MODELS, SUCH AS -- AND TREAT IT AS THE BASE MODEL.
TREAT IT AS THE BASE MODEL.
TREAT IT AS THE BASE MODEL. USING -- UT USER COULD BUILD
USING -- UT USER COULD BUILD
USING -- UT USER COULD BUILD THIS MODEL, THE BASE MODEL, USE
THIS MODEL, THE BASE MODEL, USE
THIS MODEL, THE BASE MODEL, USE ANY OF THEIR FAVORITE CURRENT
ANY OF THEIR FAVORITE CURRENT
ANY OF THEIR FAVORITE CURRENT API, LIKE WE JUST MENTIONED.
API, LIKE WE JUST MENTIONED.
API, LIKE WE JUST MENTIONED. SEE CONTROVERSIAL, FUNCTIONAL,
SEE CONTROVERSIAL, FUNCTIONAL,
SEE CONTROVERSIAL, FUNCTIONAL, OR SUBCLASING.
OR SUBCLASING.
OR SUBCLASING. AFTER THE BASE MODEL IS BUILT,
AFTER THE BASE MODEL IS BUILT,
AFTER THE BASE MODEL IS BUILT, WE USE THE API TO WRAP AROUND
WE USE THE API TO WRAP AROUND
WE USE THE API TO WRAP AROUND THE BASE MODEL TO ENABLE THE
THE BASE MODEL TO ENABLE THE
THE BASE MODEL TO ENABLE THE GRAPH REGULARIZATION.
GRAPH REGULARIZATION.
GRAPH REGULARIZATION. THERE ARE PERIMETERS WE NEED TO
THERE ARE PERIMETERS WE NEED TO
THERE ARE PERIMETERS WE NEED TO CONFIGURE.
CONFIGURE.
CONFIGURE. FOR EXAMPLE, WE NEED TO SPECIFY
FOR EXAMPLE, WE NEED TO SPECIFY
FOR EXAMPLE, WE NEED TO SPECIFY THE MAXIMUM NUMBER OF NEIGHBORS
THE MAXIMUM NUMBER OF NEIGHBORS
THE MAXIMUM NUMBER OF NEIGHBORS CONSIDERED DURING THE
CONSIDERED DURING THE
CONSIDERED DURING THE REGULARIZATION.
REGULARIZATION.
REGULARIZATION. ALSO, FOR EACH TYPE OF PROGRAM
ALSO, FOR EACH TYPE OF PROGRAM
ALSO, FOR EACH TYPE OF PROGRAM METER, WE PROVIDE DEEPER VALUES
METER, WE PROVIDE DEEPER VALUES
METER, WE PROVIDE DEEPER VALUES SET TO A CERTAIN NUMBER THAT WE
SET TO A CERTAIN NUMBER THAT WE
SET TO A CERTAIN NUMBER THAT WE KNOW IMPEERICLY THEY WORK WELL.
KNOW IMPEERICLY THEY WORK WELL.
KNOW IMPEERICLY THEY WORK WELL. AFTER WE ENABLE THE GRAPH, THE
AFTER WE ENABLE THE GRAPH, THE
AFTER WE ENABLE THE GRAPH, THE NEXT IS TO EXTEND OUR WORKFLOW,
NEXT IS TO EXTEND OUR WORKFLOW,
NEXT IS TO EXTEND OUR WORKFLOW, COMPILE, FIT, AND THEN EVAL.
COMPILE, FIT, AND THEN EVAL.
COMPILE, FIT, AND THEN EVAL. THAT'S IT.
THAT'S IT.
THAT'S IT. WITHIN FIVE LINES, WE ARE ABLE
WITHIN FIVE LINES, WE ARE ABLE
WITHIN FIVE LINES, WE ARE ABLE TO ENABLE GRAPH REGULARIZATION,
TO ENABLE GRAPH REGULARIZATION,
TO ENABLE GRAPH REGULARIZATION, AND IN FIVE LINES ACTUALLY
AND IN FIVE LINES ACTUALLY
AND IN FIVE LINES ACTUALLY INCLUDE ONE LINE AS A TOP.
INCLUDE ONE LINE AS A TOP.
INCLUDE ONE LINE AS A TOP. NOT AN ACTUAL LOGIC
NOT AN ACTUAL LOGIC
NOT AN ACTUAL LOGIC IMPLEMENTATION.
IMPLEMENTATION.
IMPLEMENTATION. HERE LET US SHOW SOME RESULTS OF
HERE LET US SHOW SOME RESULTS OF
HERE LET US SHOW SOME RESULTS OF A MODEL TREND WITH STRUCTURE
A MODEL TREND WITH STRUCTURE
A MODEL TREND WITH STRUCTURE SIGNALS.
SIGNALS.
SIGNALS. THE TASK IS TO CONDUCT A
THE TASK IS TO CONDUCT A
THE TASK IS TO CONDUCT A SENTIMENT ANALYSIS ON THE IMBB
SENTIMENT ANALYSIS ON THE IMBB
SENTIMENT ANALYSIS ON THE IMBB MANUFACTURE REVIEWS.
MANUFACTURE REVIEWS.
MANUFACTURE REVIEWS. WE WANT TO POINT OUT THAT THIS
WE WANT TO POINT OUT THAT THIS
WE WANT TO POINT OUT THAT THIS RESULT IS JUST FROM ONE OF OUR
RESULT IS JUST FROM ONE OF OUR
RESULT IS JUST FROM ONE OF OUR INTERNAL EXPERIMENTS.
INTERNAL EXPERIMENTS.
INTERNAL EXPERIMENTS. YOUR ACTUAL KNOWLEDGE MAY VARY
YOUR ACTUAL KNOWLEDGE MAY VARY
YOUR ACTUAL KNOWLEDGE MAY VARY FROM TASK TO TASK, TO DATA TO
FROM TASK TO TASK, TO DATA TO
FROM TASK TO TASK, TO DATA TO DATA OR MODEL TO MODEL.
DATA OR MODEL TO MODEL.
DATA OR MODEL TO MODEL. THE X EXIT HERE REPRESENTS THE
THE X EXIT HERE REPRESENTS THE
THE X EXIT HERE REPRESENTS THE AMOUNT OF SUPERVISION.
AMOUNT OF SUPERVISION.
AMOUNT OF SUPERVISION. WHICH COULD BE CONVERTED INTO
WHICH COULD BE CONVERTED INTO
WHICH COULD BE CONVERTED INTO THE AMOUNT OF LABEL EXAMPLES.
THE AMOUNT OF LABEL EXAMPLES.
THE AMOUNT OF LABEL EXAMPLES. AND THE Y EXIT HERE REPRESENTS
AND THE Y EXIT HERE REPRESENTS
AND THE Y EXIT HERE REPRESENTS THE MODEL ACCURACY.
THE MODEL ACCURACY.
THE MODEL ACCURACY. THE LAP FEATURE SHOWS THE
THE LAP FEATURE SHOWS THE
THE LAP FEATURE SHOWS THE PERFORMANCE OF A BIDIRECTIONAL
PERFORMANCE OF A BIDIRECTIONAL
PERFORMANCE OF A BIDIRECTIONAL DM, AND THE RIGHT FIGURE SHOWS
DM, AND THE RIGHT FIGURE SHOWS
DM, AND THE RIGHT FIGURE SHOWS THE PERFORMANCE OF A -- AS YOU
THE PERFORMANCE OF A -- AS YOU
THE PERFORMANCE OF A -- AS YOU CAN SEE, WHEN THERE -- WHEN WE
CAN SEE, WHEN THERE -- WHEN WE
CAN SEE, WHEN THERE -- WHEN WE HAVE LOTS OF TRAINING EXAMPLES,
HAVE LOTS OF TRAINING EXAMPLES,
HAVE LOTS OF TRAINING EXAMPLES, WHEN THE AMOUNT OF SUPERVISION
WHEN THE AMOUNT OF SUPERVISION
WHEN THE AMOUNT OF SUPERVISION IS HIGH, THERE IS NOT MUCH
IS HIGH, THERE IS NOT MUCH
IS HIGH, THERE IS NOT MUCH PERFORMANCE DIFFERENCE.
PERFORMANCE DIFFERENCE.
PERFORMANCE DIFFERENCE. AS SOON AS THE AMOUNT OF
AS SOON AS THE AMOUNT OF
AS SOON AS THE AMOUNT OF SUPERVISION DROPPED TO 5% OR
SUPERVISION DROPPED TO 5% OR
SUPERVISION DROPPED TO 5% OR EVEN 1%, TRAINING WITH STRUCTURE
EVEN 1%, TRAINING WITH STRUCTURE
EVEN 1%, TRAINING WITH STRUCTURE SIGNALS LEAD TO MORE ACCURATE
SIGNALS LEAD TO MORE ACCURATE
SIGNALS LEAD TO MORE ACCURATE MODELS, USUALLY BY THE
MODELS, USUALLY BY THE
MODELS, USUALLY BY THE IMPROVEMENT THAT'S MORE THAN
IMPROVEMENT THAT'S MORE THAN
IMPROVEMENT THAT'S MORE THAN 10%.
10%.
10%. >> WHAT SHOULD WE DO?
>> WHAT SHOULD WE DO?
>> WHAT SHOULD WE DO? NEURAL STRUCTURED LEARNING
NEURAL STRUCTURED LEARNING
NEURAL STRUCTURED LEARNING PROVIDES TWO METHODS.
PROVIDES TWO METHODS.
PROVIDES TWO METHODS. THE FIRST ONE IS TO CONSTRUCT
THE FIRST ONE IS TO CONSTRUCT
THE FIRST ONE IS TO CONSTRUCT THE GRAPH OR THE CONSTRUCT THE
THE GRAPH OR THE CONSTRUCT THE
THE GRAPH OR THE CONSTRUCT THE STRUCTURE VIA DATA PROCESSING,
STRUCTURE VIA DATA PROCESSING,
STRUCTURE VIA DATA PROCESSING, AND THE SECOND ONE IS TO
AND THE SECOND ONE IS TO
AND THE SECOND ONE IS TO CONSTRUCT SUCH STRUCTURE VIA
CONSTRUCT SUCH STRUCTURE VIA
CONSTRUCT SUCH STRUCTURE VIA ADVERSARIAL NEIGHBORS.
ADVERSARIAL NEIGHBORS.
ADVERSARIAL NEIGHBORS. LET'S FOCUS ON THE DATA
LET'S FOCUS ON THE DATA
LET'S FOCUS ON THE DATA PROCESSING ONE FIRST.
PROCESSING ONE FIRST.
PROCESSING ONE FIRST. AGAIN, LET'S TAKE DOCUMENT
AGAIN, LET'S TAKE DOCUMENT
AGAIN, LET'S TAKE DOCUMENT CLASSIFICATION AS AN EXAMPLE.
CLASSIFICATION AS AN EXAMPLE.
CLASSIFICATION AS AN EXAMPLE. GIVEN THE SAMPLE DOCUMENT, HOW
GIVEN THE SAMPLE DOCUMENT, HOW
GIVEN THE SAMPLE DOCUMENT, HOW DO WE KNOW IF ANOTHER DOCUMENT
DO WE KNOW IF ANOTHER DOCUMENT
DO WE KNOW IF ANOTHER DOCUMENT IS SIMILAR ENOUGH TO BE THE
IS SIMILAR ENOUGH TO BE THE
IS SIMILAR ENOUGH TO BE THE NEIGHBOR DOCUMENT?
NEIGHBOR DOCUMENT?
NEIGHBOR DOCUMENT? THESE DOCUMENTS WILL BE
THESE DOCUMENTS WILL BE
THESE DOCUMENTS WILL BE PROJECTED TO THE EMBEDDING
PROJECTED TO THE EMBEDDING
PROJECTED TO THE EMBEDDING SPACE.
SPACE.
SPACE. FOR EXAMPLE, WE COULD USE THE
FOR EXAMPLE, WE COULD USE THE
FOR EXAMPLE, WE COULD USE THE PRETREND BIRD EMBEDDING.
PRETREND BIRD EMBEDDING.
PRETREND BIRD EMBEDDING. THAT'S MENTIONED IN THE EARLIER
THAT'S MENTIONED IN THE EARLIER
THAT'S MENTIONED IN THE EARLIER ENS E TENSORFLOW TALK, AND TO
ENS E TENSORFLOW TALK, AND TO
ENS E TENSORFLOW TALK, AND TO GET ALL THESE DOCUMENTS
GET ALL THESE DOCUMENTS
GET ALL THESE DOCUMENTS EMBEDDED.
EMBEDDED.
EMBEDDED. DOCUMENTS THAT ARE CLOSER IN THE
DOCUMENTS THAT ARE CLOSER IN THE
DOCUMENTS THAT ARE CLOSER IN THE EMBEDDING SPACE WITH ASSUMED TO
EMBEDDING SPACE WITH ASSUMED TO
EMBEDDING SPACE WITH ASSUMED TO HAVE SIMILAR SCHEMATICS.
HAVE SIMILAR SCHEMATICS.
HAVE SIMILAR SCHEMATICS. NEXT, WE EXAMINE THE SIMILARITY
NEXT, WE EXAMINE THE SIMILARITY
NEXT, WE EXAMINE THE SIMILARITY BETWEEN TWO EMBEDS, PRECISE
BETWEEN TWO EMBEDS, PRECISE
BETWEEN TWO EMBEDS, PRECISE SIMILARITY OR OTHER METRIC TO BE
SIMILARITY OR OTHER METRIC TO BE
SIMILARITY OR OTHER METRIC TO BE USED HERE.
USED HERE.
USED HERE. IF THE SIMILARITY IS HIGHER THAN
IF THE SIMILARITY IS HIGHER THAN
IF THE SIMILARITY IS HIGHER THAN THE PREDEFINED THRESHOLD, WE
THE PREDEFINED THRESHOLD, WE
THE PREDEFINED THRESHOLD, WE TREAT THESE TWO DOCUMENTS ARE
TREAT THESE TWO DOCUMENTS ARE
TREAT THESE TWO DOCUMENTS ARE SIMILAR ENOUGH, AND, THEREFORE,
SIMILAR ENOUGH, AND, THEREFORE,
SIMILAR ENOUGH, AND, THEREFORE, WE ADD AN EDGE BETWEEN THESE TWO
WE ADD AN EDGE BETWEEN THESE TWO
WE ADD AN EDGE BETWEEN THESE TWO DOCUMENTS TO MAKE THEM
DOCUMENTS TO MAKE THEM
DOCUMENTS TO MAKE THEM NEIGHBORS.
NEIGHBORS.
NEIGHBORS. BY REPEATING THIS PROCESS, WE
BY REPEATING THIS PROCESS, WE
BY REPEATING THIS PROCESS, WE COULD CONSTRUCT A STRUCTURE OR
COULD CONSTRUCT A STRUCTURE OR
COULD CONSTRUCT A STRUCTURE OR CONSTRUCT A GRAPH ABOVE ALL THE
CONSTRUCT A GRAPH ABOVE ALL THE
CONSTRUCT A GRAPH ABOVE ALL THE DATA, VIA THE DATA FOR
DATA, VIA THE DATA FOR
DATA, VIA THE DATA FOR PROCESSING.
PROCESSING.
PROCESSING. AFTER WE HAVE THE GRAPH, THE
AFTER WE HAVE THE GRAPH, THE
AFTER WE HAVE THE GRAPH, THE REST OF THE TRAINING FLOW IS
REST OF THE TRAINING FLOW IS
REST OF THE TRAINING FLOW IS EXACTLY THE SAME AS WE MENTIONED
EXACTLY THE SAME AS WE MENTIONED
EXACTLY THE SAME AS WE MENTIONED BEFORE.
BEFORE.
BEFORE. LET'S, AGAIN, TAKE A LOOK AT THE
LET'S, AGAIN, TAKE A LOOK AT THE
LET'S, AGAIN, TAKE A LOOK AT THE ACTUAL CODE EXAMPLE.
ACTUAL CODE EXAMPLE.
ACTUAL CODE EXAMPLE. WE FIRST LOAD THE TRAINING DATA
WE FIRST LOAD THE TRAINING DATA
WE FIRST LOAD THE TRAINING DATA AND TEST SAMPLES FROM THE IMBD
AND TEST SAMPLES FROM THE IMBD
AND TEST SAMPLES FROM THE IMBD DATA SET.
DATA SET.
DATA SET. NEXT, WE LOAD THE PRETREND
NEXT, WE LOAD THE PRETREND
NEXT, WE LOAD THE PRETREND EMBEDDING MODEL FROM.
EMBEDDING MODEL FROM.
EMBEDDING MODEL FROM. THE EMBEDDING MODEL WE USE HERE
THE EMBEDDING MODEL WE USE HERE
THE EMBEDDING MODEL WE USE HERE IS A SWEEPER MODEL, BUT FEEL
IS A SWEEPER MODEL, BUT FEEL
IS A SWEEPER MODEL, BUT FEEL FREE TO REPLACE THEM WITH YOUR
FREE TO REPLACE THEM WITH YOUR
FREE TO REPLACE THEM WITH YOUR AVORITE PRETREND EMBEDDING
AVORITE PRETREND EMBEDDING
AVORITE PRETREND EMBEDDING MODEL SUCH AS WE CAN CALCULATE A
MODEL SUCH AS WE CAN CALCULATE A
MODEL SUCH AS WE CAN CALCULATE A SIMILARITY BETWEEN TWO REVIEWS
SIMILARITY BETWEEN TWO REVIEWS
SIMILARITY BETWEEN TWO REVIEWS IN AN EMBEDDING SPACE.
IN AN EMBEDDING SPACE.
IN AN EMBEDDING SPACE. REMEMBER, WHEN TWO REVIEWS ARE
REMEMBER, WHEN TWO REVIEWS ARE
REMEMBER, WHEN TWO REVIEWS ARE CLOSER IN THE EMBEDDING SPACE,
CLOSER IN THE EMBEDDING SPACE,
CLOSER IN THE EMBEDDING SPACE, WE ASSUME THEY SHARE SIMILAR
WE ASSUME THEY SHARE SIMILAR
WE ASSUME THEY SHARE SIMILAR SCHEMATICS.
SCHEMATICS.
SCHEMATICS. AFTER WE PROJECT TEXTS TO THE
AFTER WE PROJECT TEXTS TO THE
AFTER WE PROJECT TEXTS TO THE EMBEDDING, WE USE GRAPH
EMBEDDING, WE USE GRAPH
EMBEDDING, WE USE GRAPH FUNCTION, PROVIDED BY THE NEURAL
FUNCTION, PROVIDED BY THE NEURAL
FUNCTION, PROVIDED BY THE NEURAL STRUCTURAL LESCHING TO CONSTRUCT
STRUCTURAL LESCHING TO CONSTRUCT
STRUCTURAL LESCHING TO CONSTRUCT THE GRAPH.
THE GRAPH.
THE GRAPH. WE PROVIDE A THRESHOLD, WHICH IS
WE PROVIDE A THRESHOLD, WHICH IS
WE PROVIDE A THRESHOLD, WHICH IS 0.8 IN THIS SPACE.
0.8 IN THIS SPACE.
0.8 IN THIS SPACE. AFTER WE HAVE THE GRAPH, WE CALL
AFTER WE HAVE THE GRAPH, WE CALL
AFTER WE HAVE THE GRAPH, WE CALL PACK NEIGHBOR FUNCTION TO
PACK NEIGHBOR FUNCTION TO
PACK NEIGHBOR FUNCTION TO INCORPORATE THE NEIGHBOR SAMPLES
INCORPORATE THE NEIGHBOR SAMPLES
INCORPORATE THE NEIGHBOR SAMPLES INTO EACH TRAINING SAMPLE.
INTO EACH TRAINING SAMPLE.
INTO EACH TRAINING SAMPLE. HERE FOR EACH SAMPLE, THREE
HERE FOR EACH SAMPLE, THREE
HERE FOR EACH SAMPLE, THREE NEIGHBORS ARE CONSIDERED.
NEIGHBORS ARE CONSIDERED.
NEIGHBORS ARE CONSIDERED. AFTER WE AUGMENT IT WITH GRAPH
AFTER WE AUGMENT IT WITH GRAPH
AFTER WE AUGMENT IT WITH GRAPH SIGNALS, EVERYTHING IS JUST LIKE
SIGNALS, EVERYTHING IS JUST LIKE
SIGNALS, EVERYTHING IS JUST LIKE THE FIRST CODE EXAMPLE WE SHOW.
THE FIRST CODE EXAMPLE WE SHOW.
THE FIRST CODE EXAMPLE WE SHOW. DATA, BUILD A BASE MODEL VERSUS
DATA, BUILD A BASE MODEL VERSUS
DATA, BUILD A BASE MODEL VERSUS EITHER SEE CONTROVERSIAL,
EITHER SEE CONTROVERSIAL,
EITHER SEE CONTROVERSIAL, FUNCTIONAL API, OR SUBCLASSING.
FUNCTIONAL API, OR SUBCLASSING.
FUNCTIONAL API, OR SUBCLASSING. THEN WE USE THE NEURAL
THEN WE USE THE NEURAL
THEN WE USE THE NEURAL STRUCTURAL LEARNING API TO WRAP
STRUCTURAL LEARNING API TO WRAP
STRUCTURAL LEARNING API TO WRAP AROUND THE BASE MODEL.
AROUND THE BASE MODEL.
AROUND THE BASE MODEL. GERN THE REST OF THE WORKFLOW IS
GERN THE REST OF THE WORKFLOW IS
GERN THE REST OF THE WORKFLOW IS JUST TO EXTEND OUR FLOW,
JUST TO EXTEND OUR FLOW,
JUST TO EXTEND OUR FLOW, COMPILE, FIT, AND EVALVE.
COMPILE, FIT, AND EVALVE.
COMPILE, FIT, AND EVALVE. SO WE ALSO PROVIDE HANDS-ON
SO WE ALSO PROVIDE HANDS-ON
SO WE ALSO PROVIDE HANDS-ON TUTORIAL.
TUTORIAL.
TUTORIAL. STEP BY STEP TUTORIAL ON OUR
STEP BY STEP TUTORIAL ON OUR
STEP BY STEP TUTORIAL ON OUR WEBSITE.
WEBSITE.
WEBSITE. FEEL FREE TO VISIT THE WEBSITE
FEEL FREE TO VISIT THE WEBSITE
FEEL FREE TO VISIT THE WEBSITE AND PLAY IT BY YOURSELF.
AND PLAY IT BY YOURSELF.
AND PLAY IT BY YOURSELF. THE SECOND METHOD TO CONSTRUCT
THE SECOND METHOD TO CONSTRUCT
THE SECOND METHOD TO CONSTRUCT THE STRUCTURE OR CONSTRUCT A
THE STRUCTURE OR CONSTRUCT A
THE STRUCTURE OR CONSTRUCT A GRAPH SIGNAL IS TO BUILD A GRAPH
GRAPH SIGNAL IS TO BUILD A GRAPH
GRAPH SIGNAL IS TO BUILD A GRAPH DYNAMICALLY BY ADDING
DYNAMICALLY BY ADDING
DYNAMICALLY BY ADDING ADVERSARIAL NEIGHBORS.
ADVERSARIAL NEIGHBORS.
ADVERSARIAL NEIGHBORS. FOR EACH TRAINING SAMPLE WE FIND
FOR EACH TRAINING SAMPLE WE FIND
FOR EACH TRAINING SAMPLE WE FIND A MALICIOUS -- BASED ON THE
A MALICIOUS -- BASED ON THE
A MALICIOUS -- BASED ON THE REVERSE GRADIENT DIRECTION.
REVERSE GRADIENT DIRECTION.
REVERSE GRADIENT DIRECTION. IN OTHER WORDS, THAT IS DESIGND
IN OTHER WORDS, THAT IS DESIGND
IN OTHER WORDS, THAT IS DESIGND TO CONFUSE THE MODEL, WHICH
TO CONFUSE THE MODEL, WHICH
TO CONFUSE THE MODEL, WHICH MEANS TO MAXIMIZE THE LOSS.
MEANS TO MAXIMIZE THE LOSS.
MEANS TO MAXIMIZE THE LOSS. THEN THIS MALICIOUS -- IS -- IS
THEN THIS MALICIOUS -- IS -- IS
THEN THIS MALICIOUS -- IS -- IS ADDING ORIGINAL TRAINING SAMPLE
ADDING ORIGINAL TRAINING SAMPLE
ADDING ORIGINAL TRAINING SAMPLE TO CREATE AN ADVERSARIAL
TO CREATE AN ADVERSARIAL
TO CREATE AN ADVERSARIAL NEIGHBOR.
NEIGHBOR.
NEIGHBOR. THEN WE ADD IT BETWEEN THESE
THEN WE ADD IT BETWEEN THESE
THEN WE ADD IT BETWEEN THESE ADVERSARIAL NEIGHBORS AND THE
ADVERSARIAL NEIGHBORS AND THE
ADVERSARIAL NEIGHBORS AND THE ORIGINAL TRAINING EXAMPLE.
ORIGINAL TRAINING EXAMPLE.
ORIGINAL TRAINING EXAMPLE. THEREFORE, WE HAVE CONSTRUCTED A
THEREFORE, WE HAVE CONSTRUCTED A
THEREFORE, WE HAVE CONSTRUCTED A GRAPH OR CONSTRUCTED A
GRAPH OR CONSTRUCTED A
GRAPH OR CONSTRUCTED A STRUCTURE.
STRUCTURE.
STRUCTURE. THIS IS THE CODE EXAMPLE USING
THIS IS THE CODE EXAMPLE USING
THIS IS THE CODE EXAMPLE USING ADVERSARIAL MODEL FROM THE
ADVERSARIAL MODEL FROM THE
ADVERSARIAL MODEL FROM THE NEURAL STRUCTURAL LEARNING.
NEURAL STRUCTURAL LEARNING.
NEURAL STRUCTURAL LEARNING. AGAIN, FEEL FAMILIAR?
AGAIN, FEEL FAMILIAR?
AGAIN, FEEL FAMILIAR? IN ADDITION TO THESE THREE
IN ADDITION TO THESE THREE
IN ADDITION TO THESE THREE LINES, EVERYTHING ELSE FOLLOWED
LINES, EVERYTHING ELSE FOLLOWED
LINES, EVERYTHING ELSE FOLLOWED THE SAME WORKFLOW INTRODUCED
THE SAME WORKFLOW INTRODUCED
THE SAME WORKFLOW INTRODUCED BEFORE.
BEFORE.
BEFORE. IT HAS BEEN WIDELY USED IN MANY
IT HAS BEEN WIDELY USED IN MANY
IT HAS BEEN WIDELY USED IN MANY PRODUCTS AND SERVICES IN GOOGLE.
PRODUCTS AND SERVICES IN GOOGLE.
PRODUCTS AND SERVICES IN GOOGLE. FOR EXAMPLE, LEARNING IMAGE
FOR EXAMPLE, LEARNING IMAGE
FOR EXAMPLE, LEARNING IMAGE SCHEMATIC EMBEDDING.
SCHEMATIC EMBEDDING.
SCHEMATIC EMBEDDING. HERE WE PROVIDE SIX EXAMPLES.
HERE WE PROVIDE SIX EXAMPLES.
HERE WE PROVIDE SIX EXAMPLES. TWO FOR EACH SCHEMATIC
TWO FOR EACH SCHEMATIC
TWO FOR EACH SCHEMATIC GRANULARITY TO ILLUSTRATE A
GRANULARITY TO ILLUSTRATE A
GRANULARITY TO ILLUSTRATE A DIFFERENCE -- THE OBJECT IN THE
DIFFERENCE -- THE OBJECT IN THE
DIFFERENCE -- THE OBJECT IN THE RIGHT IS TO GOLDEN GATE BRIDGE.
RIGHT IS TO GOLDEN GATE BRIDGE.
RIGHT IS TO GOLDEN GATE BRIDGE. THE GOLDEN GATE BRIDGE IS THE
THE GOLDEN GATE BRIDGE IS THE
THE GOLDEN GATE BRIDGE IS THE STEEL RED BRIDGE.
STEEL RED BRIDGE.
STEEL RED BRIDGE. ALL THE STEEL RED BRIBLG, SUCH
ALL THE STEEL RED BRIBLG, SUCH
ALL THE STEEL RED BRIBLG, SUCH AS AN IMAGE IN THE MIDDLE OR THE
AS AN IMAGE IN THE MIDDLE OR THE
AS AN IMAGE IN THE MIDDLE OR THE GOLDEN GATE BRIDGE.
GOLDEN GATE BRIDGE.
GOLDEN GATE BRIDGE. LEARNING TO CAPTURE SCHEMATICS,
LEARNING TO CAPTURE SCHEMATICS,
LEARNING TO CAPTURE SCHEMATICS, HOWEVER, IS THE CORE OF MANY
HOWEVER, IS THE CORE OF MANY
HOWEVER, IS THE CORE OF MANY IMAGE-RELATED APPLICATION SUCH
IMAGE-RELATED APPLICATION SUCH
IMAGE-RELATED APPLICATION SUCH AS IMAGE SEARCH.
AS IMAGE SEARCH.
AS IMAGE SEARCH. EITHER CARRIED BY TRADITIONAL
EITHER CARRIED BY TRADITIONAL
EITHER CARRIED BY TRADITIONAL KEYWORDS OR THE EXAMPLE QUERY
KEYWORDS OR THE EXAMPLE QUERY
KEYWORDS OR THE EXAMPLE QUERY IMAGE.
IMAGE.
IMAGE. THIS IS THE OVERALL NEURAL
THIS IS THE OVERALL NEURAL
THIS IS THE OVERALL NEURAL ARCHITECTURE USED TO LEARN THE
ARCHITECTURE USED TO LEARN THE
ARCHITECTURE USED TO LEARN THE IMAGE EMBEDDING, AND AGAIN, FEEL
IMAGE EMBEDDING, AND AGAIN, FEEL
IMAGE EMBEDDING, AND AGAIN, FEEL FAMILIAR.
FAMILIAR.
FAMILIAR. THIS IS EXACTLY THE SAME
THIS IS EXACTLY THE SAME
THIS IS EXACTLY THE SAME WORKFLOW USED AGAIN AND AGAIN IN
WORKFLOW USED AGAIN AND AGAIN IN
WORKFLOW USED AGAIN AND AGAIN IN THIS TALK.
THIS TALK.
THIS TALK. SINCE THAT TALK, WE WILL NOT
SINCE THAT TALK, WE WILL NOT
SINCE THAT TALK, WE WILL NOT INTRODUCE IN DETAIL ABOUT OTHER
INTRODUCE IN DETAIL ABOUT OTHER
INTRODUCE IN DETAIL ABOUT OTHER TECHNIQUES SUCH AS SIMPLE USED
TECHNIQUES SUCH AS SIMPLE USED
TECHNIQUES SUCH AS SIMPLE USED TO TRAIN THE MODEL.
TO TRAIN THE MODEL.
TO TRAIN THE MODEL. IF YOU ARE INTERESTED, PLEASE
IF YOU ARE INTERESTED, PLEASE
IF YOU ARE INTERESTED, PLEASE REFER TO OUR PAPER.
REFER TO OUR PAPER.
REFER TO OUR PAPER. LET'S ZOOM IN INTO THE STRUCTURE
LET'S ZOOM IN INTO THE STRUCTURE
LET'S ZOOM IN INTO THE STRUCTURE PART.
PART.
PART. THE GRAPH USED HERE IS A CORE
THE GRAPH USED HERE IS A CORE
THE GRAPH USED HERE IS A CORE CURRENT GRAPH.
CURRENT GRAPH.
CURRENT GRAPH. ESSENTIALLY THE CORE CURRENTS
ESSENTIALLY THE CORE CURRENTS
ESSENTIALLY THE CORE CURRENTS GRAPH IS TRYING TO ANSWER THE
GRAPH IS TRYING TO ANSWER THE
GRAPH IS TRYING TO ANSWER THE FOLLOWING QUESTION.
FOLLOWING QUESTION.
FOLLOWING QUESTION. GIVEN ONE IMAGE IS SELECTED,
GIVEN ONE IMAGE IS SELECTED,
GIVEN ONE IMAGE IS SELECTED, WHAT ARE OTHER IMAGES THAT ARE
WHAT ARE OTHER IMAGES THAT ARE
WHAT ARE OTHER IMAGES THAT ARE SUFFICIENTLY SIMILAR THAT WOULD
SUFFICIENTLY SIMILAR THAT WOULD
SUFFICIENTLY SIMILAR THAT WOULD ALSO BE SELECTED AND SAY IT IS
ALSO BE SELECTED AND SAY IT IS
ALSO BE SELECTED AND SAY IT IS THE WHITE ENGLISH BULLDOG.
THE WHITE ENGLISH BULLDOG.
THE WHITE ENGLISH BULLDOG. IF TWO IMAGES ARE CALLED FOR
IF TWO IMAGES ARE CALLED FOR
IF TWO IMAGES ARE CALLED FOR MANY, MANY TIMES, WE ADD AN EDGE
MANY, MANY TIMES, WE ADD AN EDGE
MANY, MANY TIMES, WE ADD AN EDGE BETWEEN THESE TWO IMAGES MAKING
BETWEEN THESE TWO IMAGES MAKING
BETWEEN THESE TWO IMAGES MAKING THEM NEIGHBORS.
THEM NEIGHBORS.
THEM NEIGHBORS. SO HERE ARE SOME EXPERIMENTAL
SO HERE ARE SOME EXPERIMENTAL
SO HERE ARE SOME EXPERIMENTAL RESULTS FOR EACH QUERY IMAGE WE
RESULTS FOR EACH QUERY IMAGE WE
RESULTS FOR EACH QUERY IMAGE WE PROVIDE A TOP THREE NEIGHBOR
PROVIDE A TOP THREE NEIGHBOR
PROVIDE A TOP THREE NEIGHBOR BASED ON THE IMAGE EMBEDDING
BASED ON THE IMAGE EMBEDDING
BASED ON THE IMAGE EMBEDDING LEARNING.
LEARNING.
LEARNING. THE IMAGE COLOR TO BE STRONGLY
THE IMAGE COLOR TO BE STRONGLY
THE IMAGE COLOR TO BE STRONGLY SIMILAR WITH THE QUERY IMAGE,
SIMILAR WITH THE QUERY IMAGE,
SIMILAR WITH THE QUERY IMAGE, AND IT'S NOT SO SIMILAR.
AND IT'S NOT SO SIMILAR.
AND IT'S NOT SO SIMILAR. FOR EXAMPLE, IN THE LAST FIGURE
FOR EXAMPLE, IN THE LAST FIGURE
FOR EXAMPLE, IN THE LAST FIGURE WHEN THE IMAGE QUERY IS A WHITE
WHEN THE IMAGE QUERY IS A WHITE
WHEN THE IMAGE QUERY IS A WHITE SQUIRREL, ALL THE WHITE SQUIRREL
SQUIRREL, ALL THE WHITE SQUIRREL
SQUIRREL, ALL THE WHITE SQUIRREL USING AN EMBEDDING LEARNINGED
USING AN EMBEDDING LEARNINGED
USING AN EMBEDDING LEARNINGED FROM THE NEURAL STRUCTURAL
FROM THE NEURAL STRUCTURAL
FROM THE NEURAL STRUCTURAL LEARNING FRAMEWORK.
LEARNING FRAMEWORK.
LEARNING FRAMEWORK. IN OTHER WORDS, LEARNING WITH
IN OTHER WORDS, LEARNING WITH
IN OTHER WORDS, LEARNING WITH STRUCTURE IS ABLE TO CAPTURE
STRUCTURE IS ABLE TO CAPTURE
STRUCTURE IS ABLE TO CAPTURE IMAGE SCHEMATICS MUCH CLOSER TO
IMAGE SCHEMATICS MUCH CLOSER TO
IMAGE SCHEMATICS MUCH CLOSER TO ACTUAL HUMAN PERCEPTION.
ACTUAL HUMAN PERCEPTION.
ACTUAL HUMAN PERCEPTION. SO TO RECAP, TRAINING WITH
SO TO RECAP, TRAINING WITH
SO TO RECAP, TRAINING WITH STRUCTURE IS VERY USEFUL.
STRUCTURE IS VERY USEFUL.
STRUCTURE IS VERY USEFUL. LAST LABEL DATA TOO REQUIRE TO
LAST LABEL DATA TOO REQUIRE TO
LAST LABEL DATA TOO REQUIRE TO EFFECTIVELY TRAIN THE MODEL.
EFFECTIVELY TRAIN THE MODEL.
EFFECTIVELY TRAIN THE MODEL. ALSO, LEARNING WITH STRUCTURE
ALSO, LEARNING WITH STRUCTURE
ALSO, LEARNING WITH STRUCTURE SIGNALS LEAD TO THE MORROW BUST
SIGNALS LEAD TO THE MORROW BUST
SIGNALS LEAD TO THE MORROW BUST MODEL.
MODEL.
MODEL. ALSO, IT WORKS FOR ALL TYPES OF
ALSO, IT WORKS FOR ALL TYPES OF
ALSO, IT WORKS FOR ALL TYPES OF NEURAL NETS, EITHER COMMERCIAL
NEURAL NETS, EITHER COMMERCIAL
NEURAL NETS, EITHER COMMERCIAL NEURAL NET, OR RECURRENT NEURAL
NEURAL NET, OR RECURRENT NEURAL
NEURAL NET, OR RECURRENT NEURAL NET OR ANY CUSTOM NEURAL NET YOU
NET OR ANY CUSTOM NEURAL NET YOU
NET OR ANY CUSTOM NEURAL NET YOU DESIGN.
DESIGN.
DESIGN. THIS IS PROBABLY THE MOST
THIS IS PROBABLY THE MOST
THIS IS PROBABLY THE MOST INFORMATIVE SLIDE OF THIS TALK.
INFORMATIVE SLIDE OF THIS TALK.
INFORMATIVE SLIDE OF THIS TALK. YOU CAN LEARN TOOLS, LIBRARIES,
YOU CAN LEARN TOOLS, LIBRARIES,
YOU CAN LEARN TOOLS, LIBRARIES, OR HANDS-ON TUTORIALS IN DETAIL
OR HANDS-ON TUTORIALS IN DETAIL
OR HANDS-ON TUTORIALS IN DETAIL IN OUR WEBSITE.
IN OUR WEBSITE.
IN OUR WEBSITE. ALSO, PLEASE START.
ALSO, PLEASE START.
ALSO, PLEASE START. WE DO TAKE THE ISSUES, AND WE
WE DO TAKE THE ISSUES, AND WE
WE DO TAKE THE ISSUES, AND WE WOULD LOVE TO HEAR FROM YOU.
WOULD LOVE TO HEAR FROM YOU.
WOULD LOVE TO HEAR FROM YOU. WE ARE LOOKING FORWARD TO
WE ARE LOOKING FORWARD TO
WE ARE LOOKING FORWARD TO DEVELOPING THIS FRAMEWORK WITH
DEVELOPING THIS FRAMEWORK WITH
DEVELOPING THIS FRAMEWORK WITH ALL OF YOU TO MAKE THIS
ALL OF YOU TO MAKE THIS
ALL OF YOU TO MAKE THIS FRAMEWORK MORE COMPREHENSIVE.
FRAMEWORK MORE COMPREHENSIVE.
FRAMEWORK MORE COMPREHENSIVE. WE WILL BE WAITING FOR YOUR
WE WILL BE WAITING FOR YOUR
WE WILL BE WAITING FOR YOUR REQUEST.
REQUEST.
REQUEST. THANK YOU.
THANK YOU.
THANK YOU. (APP
(APP
(APP (APPLAUSE).
(APPLAUSE).
(APPLAUSE). >> DO WE STILL HAVE TIME
>> DO WE STILL HAVE TIME
>> DO WE STILL HAVE TIME FOR QUESTIONS?
FOR QUESTIONS?
FOR QUESTIONS? OKAY.
OKAY.
OKAY. I THINK WE STILL HAVE SOME TIME
I THINK WE STILL HAVE SOME TIME
I THINK WE STILL HAVE SOME TIME FOR QUESTIONS.
FOR QUESTIONS.
FOR QUESTIONS. >> YOU MAY HAVE TO USE THE MIK
>> YOU MAY HAVE TO USE THE MIK
>> YOU MAY HAVE TO USE THE MIK >> THE GRAPH FUNCTION THAT YOU
>> THE GRAPH FUNCTION THAT YOU
>> THE GRAPH FUNCTION THAT YOU MENTIONED, DOES IT DO, LIKE,
MENTIONED, DOES IT DO, LIKE,
MENTIONED, DOES IT DO, LIKE, COMPARISON ACROSS ALL THE, LIKE,
COMPARISON ACROSS ALL THE, LIKE,
COMPARISON ACROSS ALL THE, LIKE, ALL THE ITEMS
ALL THE ITEMS
ALL THE ITEMS >> ALL THE TRAINING -- ALL THE
>> ALL THE TRAINING -- ALL THE
>> ALL THE TRAINING -- ALL THE SAMPLES.
SAMPLES.
SAMPLES. NOT ONLY TRAINING, YEAH
NOT ONLY TRAINING, YEAH
NOT ONLY TRAINING, YEAH >> I SEE.
>> I SEE.
>> I SEE. SO FOR --
SO FOR --
SO FOR -- >> LABEL AND ALL LABEL
>> LABEL AND ALL LABEL
>> LABEL AND ALL LABEL CENTERS.
CENTERS.
CENTERS. >> I SEE.
>> I SEE.
>> I SEE. SO FOR -- DOES IT, LIKE -- HOW
SO FOR -- DOES IT, LIKE -- HOW
SO FOR -- DOES IT, LIKE -- HOW LONG DID IT TAKE TO BUILD GRAPH?
LONG DID IT TAKE TO BUILD GRAPH?
LONG DID IT TAKE TO BUILD GRAPH? >> IT'S A RELATIVELY SMALL DATA
>> IT'S A RELATIVELY SMALL DATA
>> IT'S A RELATIVELY SMALL DATA SET.
SET.
SET. IT WILL NOT TAKE TOO LONG?
IT WILL NOT TAKE TOO LONG?
IT WILL NOT TAKE TOO LONG? >> INSTEAD OF THAT, YOU THEN
>> INSTEAD OF THAT, YOU THEN
>> INSTEAD OF THAT, YOU THEN USE --
USE --
USE -- >> GREAT.
>> GREAT.
>> GREAT. >> WE'RE GOING TO RELEASE IT
>> WE'RE GOING TO RELEASE IT
>> WE'RE GOING TO RELEASE IT >> THANK YOU
>> THANK YOU
>> THANK YOU >> VERY NICE.
>> VERY NICE.
>> VERY NICE. JUST I UNDERSTAND WHAT YOU ARE
JUST I UNDERSTAND WHAT YOU ARE
JUST I UNDERSTAND WHAT YOU ARE DOING.
DOING.
DOING. IT LOOKS LIKE YOU'RE TAKING
IT LOOKS LIKE YOU'RE TAKING
IT LOOKS LIKE YOU'RE TAKING IMAGES AND THEN GATHERING IMAGES
IMAGES AND THEN GATHERING IMAGES
IMAGES AND THEN GATHERING IMAGES FROM THE GENERATOR ASSOCIATING
FROM THE GENERATOR ASSOCIATING
FROM THE GENERATOR ASSOCIATING THEM RECEIVING THE NEURAL
THEM RECEIVING THE NEURAL
THEM RECEIVING THE NEURAL NETWORK.
NETWORK.
NETWORK. IS THAT CORRECT?
IS THAT CORRECT?
IS THAT CORRECT? >> IT DOES NOT NECESSARILY
>> IT DOES NOT NECESSARILY
>> IT DOES NOT NECESSARILY HAVE TO BE THE STRUCTURE.
HAVE TO BE THE STRUCTURE.
HAVE TO BE THE STRUCTURE. THE IDEA IS ANY NETWORK THAT YOU
THE IDEA IS ANY NETWORK THAT YOU
THE IDEA IS ANY NETWORK THAT YOU ARE TRYING TO LEARN, WITH USING
ARE TRYING TO LEARN, WITH USING
ARE TRYING TO LEARN, WITH USING THE GRADIENTS AS A BACK DROP AND
THE GRADIENTS AS A BACK DROP AND
THE GRADIENTS AS A BACK DROP AND REVERSING THE GRADIENTS TO
REVERSING THE GRADIENTS TO
REVERSING THE GRADIENTS TO SKRUBLGT SORT OF A NOISY
SKRUBLGT SORT OF A NOISY
SKRUBLGT SORT OF A NOISY EXAMPLE, IF YOU WILL, AND THE
EXAMPLE, IF YOU WILL, AND THE
EXAMPLE, IF YOU WILL, AND THE IDEA -- THE REASON TO DO THIS IS
IDEA -- THE REASON TO DO THIS IS
IDEA -- THE REASON TO DO THIS IS IN A WHILE DOING THE TRAINING IS
IN A WHILE DOING THE TRAINING IS
IN A WHILE DOING THE TRAINING IS THAT NEXT TIME THE NETWORK SEES
THAT NEXT TIME THE NETWORK SEES
THAT NEXT TIME THE NETWORK SEES THIS NOISY EXAMPLE, RIGHT, IT
THIS NOISY EXAMPLE, RIGHT, IT
THIS NOISY EXAMPLE, RIGHT, IT WILL STILL LEARN TO CORRECT IT
WILL STILL LEARN TO CORRECT IT
WILL STILL LEARN TO CORRECT IT RATHER THAN, YOU KNOW, FLIPPING
RATHER THAN, YOU KNOW, FLIPPING
RATHER THAN, YOU KNOW, FLIPPING THE PREDICTIONS.
THE PREDICTIONS.
THE PREDICTIONS. WE TRY TO MAKE THAT INTERMEDIATE
WE TRY TO MAKE THAT INTERMEDIATE
WE TRY TO MAKE THAT INTERMEDIATE LAYER AND ALSO THE PREDICTIONS.
LAYER AND ALSO THE PREDICTIONS.
LAYER AND ALSO THE PREDICTIONS. >> IN THAT LAST FUNCTION, IT
>> IN THAT LAST FUNCTION, IT
>> IN THAT LAST FUNCTION, IT LOOKED LIKE WE HAD SOMETHING
LOOKED LIKE WE HAD SOMETHING
LOOKED LIKE WE HAD SOMETHING FROM A -- PLUS, THE GENERATOR
FROM A -- PLUS, THE GENERATOR
FROM A -- PLUS, THE GENERATOR >> THE EQUATION --
>> THE EQUATION --
>> THE EQUATION -- >> THAT LOSS THAT YOU HAD AND
>> THAT LOSS THAT YOU HAD AND
>> THAT LOSS THAT YOU HAD AND TRYING TO MINIMIZE
TRYING TO MINIMIZE
TRYING TO MINIMIZE >> OH, THE LOSS FUNCTION JUST
>> OH, THE LOSS FUNCTION JUST
>> OH, THE LOSS FUNCTION JUST HAS TWO COMPONENTS.
HAS TWO COMPONENTS.
HAS TWO COMPONENTS. THE FIRST IS THE SUPERVISED LOSS
THE FIRST IS THE SUPERVISED LOSS
THE FIRST IS THE SUPERVISED LOSS OR WHATEVER LOSS YOU HAD
OR WHATEVER LOSS YOU HAD
OR WHATEVER LOSS YOU HAD APPLICATION HAS.
APPLICATION HAS.
APPLICATION HAS. THE SECOND ONE IS FACTORIZING
THE SECOND ONE IS FACTORIZING
THE SECOND ONE IS FACTORIZING THE LOS OR NEIGHBORS, LIKE
THE LOS OR NEIGHBORS, LIKE
THE LOS OR NEIGHBORS, LIKE SOURCE IMAGES AND NEIGHBORS.
SOURCE IMAGES AND NEIGHBORS.
SOURCE IMAGES AND NEIGHBORS. AND IN THE CASE OF ADVERSARIAL
AND IN THE CASE OF ADVERSARIAL
AND IN THE CASE OF ADVERSARIAL NEIGHBOR IMAGE, IT'S BASICALLY
NEIGHBOR IMAGE, IT'S BASICALLY
NEIGHBOR IMAGE, IT'S BASICALLY CONSTRUCTED, AND THE WAY TO
CONSTRUCTED, AND THE WAY TO
CONSTRUCTED, AND THE WAY TO BASICALLY WHATEVER ASSIGN IT
BASICALLY WHATEVER ASSIGN IT
BASICALLY WHATEVER ASSIGN IT >> VERY GOOD.
>> VERY GOOD.
>> VERY GOOD. THANK YOU
THANK YOU
THANK YOU >> I THINK IN PAGE 54 GRAPH --
>> I THINK IN PAGE 54 GRAPH --
>> I THINK IN PAGE 54 GRAPH -- CAN YOU OPEN
CAN YOU OPEN
CAN YOU OPEN >> 34
>> 34
>> 34 >> YEAH, THIS ONE.
>> YEAH, THIS ONE.
>> YEAH, THIS ONE. YEAH.
YEAH.
YEAH. THIS GRAPH LOOKS LIKE IT'S
THIS GRAPH LOOKS LIKE IT'S
THIS GRAPH LOOKS LIKE IT'S CHANGED TO ME.
CHANGED TO ME.
CHANGED TO ME. HOW MANY SAMPLES AT THIS POINT
HOW MANY SAMPLES AT THIS POINT
HOW MANY SAMPLES AT THIS POINT HAVE YOU GENERATED HERE?
HAVE YOU GENERATED HERE?
HAVE YOU GENERATED HERE? >> BASICALLY, THIS IS --
>> BASICALLY, THIS IS --
>> BASICALLY, THIS IS -- IT'S -- IT'S NOT FROM THE --
IT'S -- IT'S NOT FROM THE --
IT'S -- IT'S NOT FROM THE -- >> HOW MANY TRAINING
>> HOW MANY TRAINING
>> HOW MANY TRAINING TRIALS -- HOW MANY TIMES DID YOU
TRIALS -- HOW MANY TIMES DID YOU
TRIALS -- HOW MANY TIMES DID YOU TRAIN?
TRAIN?
TRAIN? >> HOW MANY TRAINING
>> HOW MANY TRAINING
>> HOW MANY TRAINING TRIALS?
TRIALS?
TRIALS? I DO BELIEVE WE HAVE FIVE
I DO BELIEVE WE HAVE FIVE
I DO BELIEVE WE HAVE FIVE TRIALS.
TRIALS.
TRIALS. >> EACH POINT IS FIVE SAMPLES.
>> EACH POINT IS FIVE SAMPLES.
>> EACH POINT IS FIVE SAMPLES. EVEN IF THE NETWORK IS POWERFUL,
EVEN IF THE NETWORK IS POWERFUL,
EVEN IF THE NETWORK IS POWERFUL, LIKE FOR THE GAP TO INCREASE.
LIKE FOR THE GAP TO INCREASE.
LIKE FOR THE GAP TO INCREASE. THERE'S SOME 2% THING YOU SEE.
THERE'S SOME 2% THING YOU SEE.
THERE'S SOME 2% THING YOU SEE. THE GAP IS LOWER THAN WHAT THE
THE GAP IS LOWER THAN WHAT THE
THE GAP IS LOWER THAN WHAT THE 5% IS, BUT THAT'S JUST BASED ON
5% IS, BUT THAT'S JUST BASED ON
5% IS, BUT THAT'S JUST BASED ON THIS DATA CENTER.
THIS DATA CENTER.
THIS DATA CENTER. TYPICALLY WHAT YOU SEE IS THE
TYPICALLY WHAT YOU SEE IS THE
TYPICALLY WHAT YOU SEE IS THE GAP INCREASE AND SEPARATINGS
GAP INCREASE AND SEPARATINGS
GAP INCREASE AND SEPARATINGS RATIO GOES LOWER
RATIO GOES LOWER
RATIO GOES LOWER >> IF YOU HAVE A 2% --
>> IF YOU HAVE A 2% --
>> IF YOU HAVE A 2% -- >> YES.
>> YES.
>> YES. THIS IS JUST ONE EXAMPLE.
THIS IS JUST ONE EXAMPLE.
THIS IS JUST ONE EXAMPLE. >> WE WOULD RECOMMEND THAT YOU
>> WE WOULD RECOMMEND THAT YOU
>> WE WOULD RECOMMEND THAT YOU TRY IT ON YOUR OWN DATA SET AND
TRY IT ON YOUR OWN DATA SET AND
TRY IT ON YOUR OWN DATA SET AND ON ONE OF THE NETWORKS THAT YOU
ON ONE OF THE NETWORKS THAT YOU
ON ONE OF THE NETWORKS THAT YOU HAVE
HAVE
HAVE >> HI.
>> HI.
>> HI. HOW ARE YOU DOING?
HOW ARE YOU DOING?
HOW ARE YOU DOING? >> I WAS CURIOUS HOW YOU
>> I WAS CURIOUS HOW YOU
>> I WAS CURIOUS HOW YOU HAVE SEGMENTATION OR TO VIDEO
HAVE SEGMENTATION OR TO VIDEO
HAVE SEGMENTATION OR TO VIDEO CLASSIFICATION.
CLASSIFICATION.
CLASSIFICATION. >> I THINK THERE ARE A COUPLE OF
>> I THINK THERE ARE A COUPLE OF
>> I THINK THERE ARE A COUPLE OF DIFFERENT WAYS.
DIFFERENT WAYS.
DIFFERENT WAYS. LIKE, IN THE VIDEO
LIKE, IN THE VIDEO
LIKE, IN THE VIDEO CLASSIFICATION, YOU CAN LOOK AT
CLASSIFICATION, YOU CAN LOOK AT
CLASSIFICATION, YOU CAN LOOK AT VIDEOS THAT ARE SORT OF RELATED.
VIDEOS THAT ARE SORT OF RELATED.
VIDEOS THAT ARE SORT OF RELATED. RIGHT?
RIGHT?
RIGHT? LIKE I MEAN, SIMILAR, LET'S SAY,
LIKE I MEAN, SIMILAR, LET'S SAY,
LIKE I MEAN, SIMILAR, LET'S SAY, FROM THE SAME CHANNEL, AND YOU
FROM THE SAME CHANNEL, AND YOU
FROM THE SAME CHANNEL, AND YOU HAVE SIMILAR KIND OF CONTENT.
HAVE SIMILAR KIND OF CONTENT.
HAVE SIMILAR KIND OF CONTENT. OR THE METADATA KIND OF MATCHES.
OR THE METADATA KIND OF MATCHES.
OR THE METADATA KIND OF MATCHES. YOU CAN CREATE THESE LINKS
YOU CAN CREATE THESE LINKS
YOU CAN CREATE THESE LINKS BETWEEN DIFFERENT VIDEOS.
BETWEEN DIFFERENT VIDEOS.
BETWEEN DIFFERENT VIDEOS. THESE RELATED VIDEOS SHOULD BE
THESE RELATED VIDEOS SHOULD BE
THESE RELATED VIDEOS SHOULD BE OPTIMIZED TO LEARN THE SAME
OPTIMIZED TO LEARN THE SAME
OPTIMIZED TO LEARN THE SAME PREDICTION?
PREDICTION?
PREDICTION? >> THERE ARE MANY OTHER
>> THERE ARE MANY OTHER
>> THERE ARE MANY OTHER WAYS WE CAN TALK OFF LINE.
WAYS WE CAN TALK OFF LINE.
WAYS WE CAN TALK OFF LINE. >> THANK YOU
>> THANK YOU
>> THANK YOU >> IF WE'RE TALKING ABOUT --
>> IF WE'RE TALKING ABOUT --
>> IF WE'RE TALKING ABOUT -- >> LAST QUESTION.
>> LAST QUESTION.
>> LAST QUESTION. >> LAST QUESTION
>> LAST QUESTION
>> LAST QUESTION >> CLASSICAL PRINCIPLES, LIKE
>> CLASSICAL PRINCIPLES, LIKE
>> CLASSICAL PRINCIPLES, LIKE DISTORTION AND BLURRING,
DISTORTION AND BLURRING,
DISTORTION AND BLURRING, STRETCHING, AND ALL THIS KIND OF
STRETCHING, AND ALL THIS KIND OF
STRETCHING, AND ALL THIS KIND OF STUFF COMPARED TO THE
STUFF COMPARED TO THE
STUFF COMPARED TO THE ADVERSARIAL GENERATED SAMPLES.
ADVERSARIAL GENERATED SAMPLES.
ADVERSARIAL GENERATED SAMPLES. IS THERE ANY BENEFIT OF USING
IS THERE ANY BENEFIT OF USING
IS THERE ANY BENEFIT OF USING ONE AUTO OVER ANOTHER IN THIS
ONE AUTO OVER ANOTHER IN THIS
ONE AUTO OVER ANOTHER IN THIS CASE?
CASE?
CASE? >> SO --
>> SO --
>> SO -- >> WHY ADVERSARIAL EXAMPLES.
>> WHY ADVERSARIAL EXAMPLES.
>> WHY ADVERSARIAL EXAMPLES. THERE ARE SOME CLASSICAL
THERE ARE SOME CLASSICAL
THERE ARE SOME CLASSICAL PRINCIPLES
PRINCIPLES
PRINCIPLES >> BECAUSE THE DISTORTION AND
>> BECAUSE THE DISTORTION AND
>> BECAUSE THE DISTORTION AND ROTATION ARE PREDEFINED, LIKE,
ROTATION ARE PREDEFINED, LIKE,
ROTATION ARE PREDEFINED, LIKE, TRANSMISSION FUNCTIONS THAT YOU
TRANSMISSION FUNCTIONS THAT YOU
TRANSMISSION FUNCTIONS THAT YOU HAVE, RIGHT?
HAVE, RIGHT?
HAVE, RIGHT? THE ADVERSARIAL -- IT DEPENDS ON
THE ADVERSARIAL -- IT DEPENDS ON
THE ADVERSARIAL -- IT DEPENDS ON THE DATA AND THE NETWORK, AND
THE DATA AND THE NETWORK, AND
THE DATA AND THE NETWORK, AND YOU ARE USING THE GREATENTS FOR
YOU ARE USING THE GREATENTS FOR
YOU ARE USING THE GREATENTS FOR THE BACK DROPS TO CREATE AN
THE BACK DROPS TO CREATE AN
THE BACK DROPS TO CREATE AN EQUAL EXAMPLE.
EQUAL EXAMPLE.
EQUAL EXAMPLE. THE HARD EXAMPLE, IF YOU WILL.
THE HARD EXAMPLE, IF YOU WILL.
THE HARD EXAMPLE, IF YOU WILL. THIS IS THE ROTATION OR BRURING,
THIS IS THE ROTATION OR BRURING,
THIS IS THE ROTATION OR BRURING, RIGHT?
RIGHT?
RIGHT? WHAT IF SOMEBODY SAYS IT'S NOT A
WHAT IF SOMEBODY SAYS IT'S NOT A
WHAT IF SOMEBODY SAYS IT'S NOT A ROTATED IMAGE, BUT SAYS I TAKE,
ROTATED IMAGE, BUT SAYS I TAKE,
ROTATED IMAGE, BUT SAYS I TAKE, LIKE, PIXELS HERE OR THERE AND
LIKE, PIXELS HERE OR THERE AND
LIKE, PIXELS HERE OR THERE AND TRANSFORM THEM OR BLUR THEM
TRANSFORM THEM OR BLUR THEM
TRANSFORM THEM OR BLUR THEM >> WHEN NETWORK HAS THE FEEDBACK
>> WHEN NETWORK HAS THE FEEDBACK
>> WHEN NETWORK HAS THE FEEDBACK FROM THE GRAPH THAT ACTUALLY
FROM THE GRAPH THAT ACTUALLY
FROM THE GRAPH THAT ACTUALLY ADVERSARIAL NEIGHBOR GIVES YOU A
ADVERSARIAL NEIGHBOR GIVES YOU A
ADVERSARIAL NEIGHBOR GIVES YOU A COMPLETELY, LIKE, LOW SCORE ON
COMPLETELY, LIKE, LOW SCORE ON
COMPLETELY, LIKE, LOW SCORE ON THE EXAMPLE, BUT THE NETWORK
THE EXAMPLE, BUT THE NETWORK
THE EXAMPLE, BUT THE NETWORK ITSELF IS PRETTY SURE THAT THE
ITSELF IS PRETTY SURE THAT THE
ITSELF IS PRETTY SURE THAT THE CLASSES IS THERE.
CLASSES IS THERE.
CLASSES IS THERE. HOW DOES IT RESULT ON THIS LEVEL
HOW DOES IT RESULT ON THIS LEVEL
HOW DOES IT RESULT ON THIS LEVEL WHEN YOU HAVE A PRETTY CONVINCED
WHEN YOU HAVE A PRETTY CONVINCED
WHEN YOU HAVE A PRETTY CONVINCED SCENARIO VERSUS ADVERSARIAL.
SCENARIO VERSUS ADVERSARIAL.
SCENARIO VERSUS ADVERSARIAL. >> AS AN EXAMPLE, THERE'S
>> AS AN EXAMPLE, THERE'S
>> AS AN EXAMPLE, THERE'S NOT NECESSARILY HAVE THAT WRONG
NOT NECESSARILY HAVE THAT WRONG
NOT NECESSARILY HAVE THAT WRONG LABEL DURING THE LEARNING
LABEL DURING THE LEARNING
LABEL DURING THE LEARNING PROCESS.
PROCESS.
PROCESS. --
--
-- >> YOU TRY TO REGULARIZE
>> YOU TRY TO REGULARIZE
>> YOU TRY TO REGULARIZE IT, AND BY THIS MOMENT THEN, THE
IT, AND BY THIS MOMENT THEN, THE
IT, AND BY THIS MOMENT THEN, THE NETWORK IS ALREADY CONVINCED
NETWORK IS ALREADY CONVINCED
NETWORK IS ALREADY CONVINCED THAT IT'S NOT GIVEN, FOR
THAT IT'S NOT GIVEN, FOR
THAT IT'S NOT GIVEN, FOR EXAMPLE.
EXAMPLE.
EXAMPLE. THE GRAPH TELLS THAT IT'S STILL,
THE GRAPH TELLS THAT IT'S STILL,
THE GRAPH TELLS THAT IT'S STILL, LIKE, THE CLASS IS WRONG.
LIKE, THE CLASS IS WRONG.
LIKE, THE CLASS IS WRONG. HOW DOES THIS RESULT.
HOW DOES THIS RESULT.
HOW DOES THIS RESULT. >> A DMREET ADD ON THIS THAT IN
>> A DMREET ADD ON THIS THAT IN
>> A DMREET ADD ON THIS THAT IN SOME EXPERIMENTS, WE APPLY THE
SOME EXPERIMENTS, WE APPLY THE
SOME EXPERIMENTS, WE APPLY THE CORRECT LABEL TO THE ADVERSARIAL
CORRECT LABEL TO THE ADVERSARIAL
CORRECT LABEL TO THE ADVERSARIAL EXAMPLE.
EXAMPLE.
EXAMPLE. IN A PREVIOUS LIFE, ACTUALLY,
IN A PREVIOUS LIFE, ACTUALLY,
IN A PREVIOUS LIFE, ACTUALLY, THE ADVERSARY EXAMPLE, WE WOULD
THE ADVERSARY EXAMPLE, WE WOULD
THE ADVERSARY EXAMPLE, WE WOULD HAVE THE LABEL.
HAVE THE LABEL.
HAVE THE LABEL. YOU COULD THINK IN THIS WAY.
YOU COULD THINK IN THIS WAY.
YOU COULD THINK IN THIS WAY. THIS SAMPLE IS VERY CONFUSING TO
THIS SAMPLE IS VERY CONFUSING TO
THIS SAMPLE IS VERY CONFUSING TO THE MODEL.
THE MODEL.
THE MODEL. WE STILL SUPERVISE MODELS TO
WE STILL SUPERVISE MODELS TO
WE STILL SUPERVISE MODELS TO LEARN THE LABELING, AND LATER ON
LEARN THE LABELING, AND LATER ON
LEARN THE LABELING, AND LATER ON IT WILL BE MORROW BUST
IT WILL BE MORROW BUST
IT WILL BE MORROW BUST >> OKAY.
>> OKAY.
>> OKAY. CLEAR.
CLEAR.
CLEAR. THANK YOU VERY MUCH.