Placeholder Image

Subtitles section Play video

  • hello world it's Siraj in this video we're going to compare the most popular deep

  • learning frameworks out there right now to see what works best

  • the deep-learning space is exploding with frameworks right now ,it's like every

  • single week some major tech company decides to open source their own deep

  • learning library and that's not including the dozens of deep learning

  • frameworks being released every single week on github by cowboy developers how

  • *how many layers you have?*

  • let's start off with scikit-learn scikit it was made to provide an easy-to-use

  • interface for developers to use off-the-shelf general-purpose machine

  • learning algorithms for both supervised and unsupervised learning , scikit provides

  • functions that like you apply classic machine learning algorithms like support

  • vector machines logistic regressions and k nearest neighbor very easily but the

  • one type of machine learning algorithm he doesn't let you implement is a neural

  • network it doesn't provide GPU support either which is what helps neural

  • networks scale , since like two months ago

  • pretty much every single general-purpose algorithm that psyche learned

  • implemented has since been implemented in tensorflow sidekick you just got

  • LEARNED , there's also caffe which was basically the first mainstream

  • production grade deep learning library started in 2014, the cafe isn't

  • very flexible think of a neural network as a computational graph in cafe each

  • note is considered a layer so if you want new layer types you define the full

  • forward backward and gradient updates these layers are building blocks that

  • are unnecessarily big there's an endless list of them that you can pick from

  • intensive flow each note is considered a tensor operation like matrix add,

  • matrix multiply or convolution and a layer can be defined as a composition of

  • those operations so tender flows building blocks are smaller which allows

  • for more modularity ,cafe also requires a lot of unnecessary verbosity if you want

  • to support both the CPU and the GPU you need to implement extra functions for

  • each and you have to define your model using a plain text editor that is just

  • ghetto model should be defined programmatically because it's better for

  • modularity between different components , also caffe main architect now works on

  • the tensorflow team we're all out of caffe

  • speaking of modularity let's talk about keras : Keras has been the go-to

  • source to get started with deep learning for a while because it provides a very

  • high level API to build deep learning models Kera sits on top of the other

  • deep learning libraries like Theano and tensorflow it uses an

  • object-oriented design so everything is considered an object be that layers

  • models optimizers and all the parameters of the model can be access object

  • properties like model.layers[3].output will give you the output tensor

  • for the third layer in the model and model.layers[3].weights is a list

  • of symbolic weight tensors this is a cleaner interface as opposed to the

  • functional approach of making layers function that create weights when being

  • called great documentation it's all gucci yes i'm bringing that back but

  • because it's so general-purpose it lacks on the side of performance Keras has been

  • known to have performance issues when used with a tensorflow backend since

  • it's not really optimized for it but it does work pretty well with the Theano

  • backend , the two frameworks that are neck-and-neck right now in the race to

  • be the best library for both research and Industry are tensorflow and Theano

  • Theano currently outperforms tensorflow on a single GPU potential flow

  • outperforms piano for parallel execution across multiple gpus , Theano has got more

  • documentation because it's been around for a while and it's got native windows

  • support which tensorflow doesn't yet dammit windows in terms of syntax let's

  • just take a look at some code to see some differences

  • we're going to compare two scripts in tensorflow and beyond

  • they both do the same thing initializing phony data and then learn the line of

  • best fit for that data is it can predict future data points let's look at the

  • first step in both tensorflow and Theano for generating the data pretty much the

  • same way using numpy arrays so there's not really a difference there

  • let's look at the model initialization parts ,this is the basic "y=mx+b"

  • formula in tensorflow it doesn't require any special treatment of the x

  • and y variables

  • they're just they're natively but Theano we have to specifically say that

  • the variables are symbolic inputs to the function the tensorflow syntax for

  • defining the B&W variables is cleaner than we implement our gradient descent

  • function which is what helps us learn or trying to minimize the mean squared

  • error over time which is what makes our model more accurate as we train the

  • syntax for defining what we're minimizing is pretty similar then we

  • look at the actual optimizer which helps us do that will notice a difference in

  • syntax again

  • the flow just gives you access to a bunch of optimizers right out-of-the-box

  • things like gradient descent or Adam Theano makes you do this from scratch

  • then we have our training function which is again more verbose see the trend here

  • theano so far is making us implement more code than tensorflow so it seems to

  • give us more fine-grained control but at the cost of readability finally we'll

  • get to the actual training part itself they look pretty identical but

  • tensorflow methodology of encapsulating the computational graph feels conceptually

  • cleaner than pianos tensorflow is just growing so fast that it seems inevitable

  • that whatever feature it lacks right now because of how new it is it will gain

  • very rapidly

  • I mean just look at the amount of activity happening in the tension flow

  • repo versus the Theano repo on get up right now and while keras serves as an

  • easy use wrapper around different libraries it's not optimized for

  • tensorflow a better alternative you want to learn and get started

  • easily with deep learning is TF learn which is basically keras but optimized

  • for tensorflow so to sum things up the best library to use for research is

  • tensorflow the world-class researchers at both open AI and deep mine are now

  • using it for production best library to use is still tensorflow is it scaled

  • better across multiple GPUs and its closest competitor Theano .Lastly for

  • learning the best library to use is TF learn which is a high-level wrapper

  • around tensorflow that lets you get started really easily also shout out to

  • rahul do for being able to generate an upbeat midnight file badass of the

  • week please subscribe for more programming videos for now I've got to

  • go worship tensorflow some more so thanks for watching

hello world it's Siraj in this video we're going to compare the most popular deep

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it