Placeholder Image

Subtitles section Play video

  • Hello, I'm Martin Kronberg and welcome to the IOT Developer

  • Show.

  • In these episodes, we're taking a deep dive

  • into OpenVINO, Intel's new toolkit for AI

  • and computer vision development.

  • In the previous episode, we gave a high level

  • overview of OpenVINO.

  • And in this one, we will take a look

  • at OpenVINO in much more detail.

  • [MUSIC PLAYING]

  • First things first, let's take a look

  • at the OpenVINO architecture.

  • OpenVINO is a set of tools designed

  • to help you build smart video applications.

  • These tools can be broken down in two parts, a deep learning

  • deployment toolkit and a traditional computer vision

  • toolkit.

  • The deep learning toolkit consists

  • of the inference engine which runs the deep learning model,

  • a model optimizer used to convert and optimize

  • existing models from other frameworks,

  • and a set of prebuilt deep learning models.

  • Also included are libraries to optimize the running of models

  • with the Math Kernal Library for deep neural networks

  • and the compute library for deep neural networks

  • to optimize deep neural networks on CPU and GPU, respectively.

  • On the other hand, the traditional computer vision

  • toolkit consists of OpenCV 3.3, which

  • is a popular library for computer vision,

  • the Intel Media SDK used to leverage fast hardware

  • encode and decode of video and OpenCL drivers and runtimes

  • in order to access the onboard Intel GPU effectively.

  • In order to give you guys a better understanding of how

  • all of these features work together,

  • I want to walk you through a sample deep neural network

  • computer vision work flow.

  • Let's say that I have a specific computer vision

  • application in mind.

  • And that I want to use OpenVINO.

  • The first thing I can do is look online

  • to see what pretrained models exist for me to use.

  • If you can find a pretrained model that meets your needs,

  • it's going to save you a lot of time, versus having

  • to train one yourself.

  • I can go under software at Intle.com/OpenVINO and look

  • at all the models available.

  • We have models that detect people,

  • license plates, road side objects,

  • even models that detect emotion on faces,

  • like we saw in the last episode.

  • If one of these does not fit my needs,

  • I can search for more models from any

  • of the popular frameworks including Caffe, TensorFlow,

  • or MXNet.

  • OpenVINO has a tool called the model downloader which

  • is a script that pulls all the necessary files for a model,

  • including topology, weights, and labels

  • and makes sure that their naming conventions are

  • compatible with the model optimizer.

  • Once I have that model downloaded,

  • I can use the model optimizer, which is a Python script,

  • to convert the model into the intermediate representation

  • format that the OpenVINO inference engine uses.

  • For this workflow example, let's say

  • that I'm building out a people tracker

  • and the pedestrian tracking model works for me.

  • So I'm going to use that.

  • Now that I have an idea of the model that I'm using,

  • how should I go about developing an OpenVINO app?

  • The first thing to think about is what ID you will be using.

  • While there are many options, I would

  • suggest trying Intel System Studio 2019.

  • In this newly released version of our development platform

  • integration with OpenVINO is super simple.

  • And if you have used eclipse based IDs in the past,

  • it'll be very familiar to you.

  • In addition to the debugging capabilities

  • that you get with Intel System Studios,

  • you can also use it to leverage VTune, a powerful Performance

  • Analyzer.

  • If you want to optimize your application

  • VTunes give you a lot of insight into how

  • various process threads are performing on the CPU.

  • In fact, it can even tell you how

  • the various layers of your inference model are running.

  • So if you have a bottleneck happening

  • on one particular layer, a convolutional layer,

  • for example, you can work to reduce its complexity

  • or even send it to the GPU for processing to increase

  • overall performance.

  • After you have your ID set up, I would

  • say go on our GitHub to explore some of the reference

  • implementations there.

  • To get an idea of how to leverage this model,

  • I could take a look at the store traffic monitor, reference

  • implementation or installation and deployment information.

  • Right now the sample is using OpenCV and FFMpeg

  • to do the video stream encoding and decoding.

  • However, I could use the Media SDK encode/decode functionality

  • to get a more optimized performance.

  • What decoding does is transform an MP4 or other video format

  • into pixel value arrays for each frame.

  • I'm going be doing every operation on the image

  • on a frame by frame basis.

  • Now before I can run my inference model,

  • I want to do some image preprocessing,

  • let's say denoise, convert to grayscale, or resize.

  • I would use OpenCV to perform all those initial image

  • transformations on each of my frames.

  • Next, I will put my process frame

  • into the inference engine using the model I found earlier.

  • This will analyze the image and will

  • identify people in the frame as well as

  • the bounding boxes around them.

  • Now, if I want to display those labels

  • and bounding boxes on my image, I

  • will use OpenCV again to draw that information

  • onto the frame.

  • And finally, I want to encode all of those frames

  • into an output video file or stream.

  • Once again, I can either use OpenCV and FFMpeg or Media SDK.

  • Now, let's take a look at the end result

  • of that whole pipeline.

  • Here we see a retail environment where

  • we can keep track of people as they enter and leave.

  • We can also track inventory by seeing when people pick bottles

  • off of a shelf.

  • And that's it for today's episode.

  • Tune in two weeks and we'll discuss the various hardware

  • kits that you can use to develop OpenVINO applications.

  • Thanks for watching.

  • Follow the links provided.

  • And see you guys next time.

  • [DING]

Hello, I'm Martin Kronberg and welcome to the IOT Developer

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it