Placeholder Image

Subtitles section Play video

  • JOSH GORDON: Hey, everyone.

  • Thanks for joining us.

  • My name's Josh Gordon, and I'm very, very

  • happy to be here today with Jeremy Howard, who

  • is the co-founder of a course I'm sure many of you

  • have heard of called fast.ai, and he's faculty at USF.

  • So, Jeremy, thank you so much for joining us today.

  • JEREMY HOWARD: Oh, it's a pleasure.

  • JOSH GORDON: And, today, I'd love to talk about a new course

  • that you're working on for Swift.

  • So I am totally new to Swift.

  • So could you tell me a little bit maybe first about fast.ai

  • in general, and then maybe a little bit

  • about your plans for Swift?

  • JEREMY HOWARD: Yeah, absolutely.

  • So fast.ai, as you know, is a self-funded nonprofit.

  • We do a few things.

  • We do research, we do teaching, we do software development,

  • and we do community development.

  • So the research we do is all about how

  • do we make world class deep learning more

  • accessible to regular people.

  • So we do lots of algorithm development and testing

  • and curation.

  • That research ends up in courses that we

  • provide online through fast.ai.

  • There are a couple of main ones people know.

  • One is Practical Deep Learning for Coders,

  • which is taking that research and saying, how can that

  • be used to make somebody with a year of coding background,

  • but no particular math background,

  • into an effective deep learning practitioner.

  • And then there's a more advanced course,

  • which is about research level deep learning.

  • And then we have our community online, which

  • we're super excited about.

  • But then there's a really interesting thing we do,

  • which is to help all that stuff become easier,

  • we have a software library called

  • fast.ai, which currently sits on top of PyTorch and Python.

  • But we're very interested by what Chris Lattner and his team

  • are doing with Swift now and Swift for TensorFlow.

  • We think there are big opportunities to help all four

  • of those areas by embracing Swift for TensorFlow

  • as well as PyTorch and Python.

  • JOSH GORDON: OK, so moving on to Swift, so traditionally,

  • almost all of machine learning development

  • is done in Python, and of course languages

  • like R and now JavaScript.

  • So why Swift?

  • Could you tell me a little bit about it?

  • JEREMY HOWARD: Sure.

  • So I'm a real programming language nerd.

  • I like studying programming languages,

  • and I've been involved in lots of different programming

  • languages.

  • I actually was the chair of the working group that

  • tried to bring scientific programming

  • capabilities to Perl 6 back in the late '90s, early 2000s.

  • And my programming language nerdship

  • is driven by a deep discontent with any programming language

  • when it comes to numerical programming.

  • I did a lot of work in C++ in the early 2000s and got really

  • turned off by the long compile times and the complexity

  • of expression templates.

  • I quite like C# and F#, but there's always been--

  • cross platform took a long time to come,

  • and then cross platform has come with some performance

  • regressions in some way.

  • So I've always been looking for a really good programming

  • language for numerical computing.

  • Python isn't that language.

  • Python is a great glue language to sit on top of other

  • languages, so like NumPy is basically C.

  • And TensorFlow Python is just wrapping up C++,

  • which sits on top of Cuda C libraries and Icon and the C++

  • library.

  • And, in the end, it's dissatisfying for a student

  • and for a teacher, because at some point,

  • I have to say, this is the point at which we can stop finding

  • out what's going on, because beneath here,

  • it's kind of assembly code, or machine code,

  • or compiled stuff.

  • It's very frustrating, as a researcher,

  • like an NLP, where you keep on wanting

  • to do more stuff with RNNs.

  • And we keep on hitting the point where it's like, oh,

  • we can't really implement this different RNN cell,

  • because if we do, then it's not going to use cuDNN anymore,

  • and the performance is going to fall apart.

  • Swift offers a way past all this.

  • It lets us write on a language that's

  • a thin layer over the amazing LLVM infrastructure, compiler

  • infrastructure, which can get lots

  • of different bits of an algorithm pulled together,

  • optimize the whole thing, for CPU and for GPU.

  • And, at the same time, because it

  • has the view of everything going on from top to bottom,

  • it can tell me all the times I've screwed up.

  • So, hey, you thought that this dimension was batch size,

  • but over here, you use this other dimension as batch size,

  • and I tried to multiply those two tensors.

  • At compile time, I could tell the shapes

  • aren't going to match.

  • JOSH GORDON: It's huge.

  • JEREMY HOWARD: Yeah.

  • So I'm very excited to learn how it's

  • going to make me more productive as a programmer.

  • It's going to allow me to turn my research ideas into code.

  • At the moment, I keep butting up against things

  • where I just can't do it.

  • And it's going to let us teach things

  • deeper, which is what I'm always trying to do.

  • JOSH GORDON: Right.

  • So how does this relate to fast.ai?

  • JEREMY HOWARD: So then, the next course, which

  • is going to be in a couple of weeks

  • at the University of San Francisco,

  • we'll be recording it, and then that recording will

  • become our next MOOC in June.

  • It's going to be a seven part course.

  • It's going to be called Deeper Deep Learning.

  • And Deeper Deep Learning is going

  • to be all about how do we take a practical practitioner

  • of deep learning and turn them into somebody who

  • can go further, like do cutting edge research,

  • get things into production, make things run fast.

  • So of the seven lessons in the next course,

  • the last two are going to be about Swift for TensorFlow.

  • And it's going to be really cool, because actually, I'm

  • going to be co-teaching it with Chris Lattner.

  • JOSH GORDON: And so for people that don't know,

  • so Chris Lattner is the inventor of Swift.

  • JEREMY HOWARD: The inventor of Swift.

  • So, for me, I'm kind of having a little flashy moment now,

  • where I'm like, oh my god, I can't

  • believe I'll be standing next to Chris Lattner, the Chris

  • Lattner.

  • So I'm very excited about that.

  • And I'm very excited about what it

  • means in terms of what we can say

  • to these students who are at the very

  • advanced level at this point.

  • They've done 100 plus hours of study,

  • and they're competent coders.

  • And we will be able to say, here's

  • a deeply well-designed, thoughtful, fast, brilliant

  • language, which has had very, very little numerical computing

  • so far, but has the might now of Google, and Chris Lattner,

  • and a brilliant team behind it.

  • The world's your oyster.

  • Almost nothing's been implemented.

  • So the class projects are going to be things like, create

  • this layer that hasn't been implemented yet,

  • or implement this architecture, or be

  • the first person that's run this model end to end on ImageNet.

  • So the class projects-- and there'll also

  • be things like, help us create the fast.ai library for Swift

  • for TensorFlow.

  • JOSH GORDON: Yes.

  • JEREMY HOWARD: So I can tell you the fast.ai library for Swift

  • for TensorFlow even has a code name now.

  • JOSH GORDON: Awesome.

  • JEREMY HOWARD: The code name is harebrain.

  • JOSH GORDON: OK.

  • JEREMY HOWARD: Harebrain, because it's a crazy idea.

  • It's this amazingly crazy idea, which

  • is, we can take something that's so early,

  • but it's going to be such potential,

  • and actually start writing the dev UX layer from the start.

  • JOSH GORDON: You're right.

  • So this is all new territory.

  • JEREMY HOWARD: So it's this crazy idea.

  • It's super exciting.

  • But also hare, because hares are fast.

  • And Swift's fast, and fast.ai is fast, right.

  • JOSH GORDON: It sounds cool.

  • JEREMY HOWARD: And then brain, because we're working

  • with Google Brain on this.

  • So that's going to be our little internal code name for this.

  • So anybody who's getting involved in the next course,

  • through the MOOC or in person, can actually

  • help contribute to this codename harebrain library

  • from the very earliest days.

  • And so for a language nerd like me, it's so exciting.

  • JOSH GORDON: Awesome.

  • And what's the best way for people

  • to find out about the new course and follow it, keep updated,

  • sign up?

  • JEREMY HOWARD: Yeah, so if you want

  • to sign up for the in-person course, do it right now.

  • So just go to the University of San Francisco Data Institute.

  • But that's in San Francisco, so, obviously, a lot of people

  • won't be able to make it in person.

  • Otherwise, keep an eye on fast.ai in June,

  • where we'll be launching that course as a MOOC.

  • And yeah, at that point, it's still

  • going to be super early days for everything.

  • So there'll be lots of opportunities

  • for people to become part of what

  • I think is going to be a very, very, very

  • impactful project for scientific programming,

  • and for deep learning, and for differentiable computing

  • more generally, being Swift for TensorFlow.

  • JOSH GORDON: I'm really looking forward to it.

  • I have one last question for you, too.

  • JEREMY HOWARD: OK, hit me.

  • JOSH GORDON: So will Swift be primarily

  • for researchers, novices?

  • Long-term, how do you see the evolution of Swift

  • for deep learning?

  • JEREMY HOWARD: Because it's so versatile,

  • I plan to do a lot of research in Swift myself,

  • to get past that boundary I described of places where

  • you can't go with Python as a researcher.

  • I also expect it to be great in production,

  • because it's going to allow you to take the stuff that you

  • wrote at prototyping time and have something that's

  • super fast straight away, particularly because, for most

  • people--

  • maybe not most Googlers, but most normal people--

  • production inference means CPU, not GPU.

  • And Swift will be a really great option for that

  • because we're going to get all that compilation

  • niceness of LLVM to make that CPU code super fast.

  • JOSH GORDON: Awesome.

  • So thank you so much for joining us.

  • And thank you, everyone.

  • I learned a lot from this talk, and I'm really

  • looking forward to your course.

  • JEREMY HOWARD: Cheers.

  • JOSH GORDON: OK.

  • Thanks very much.

  • [MUSIC PLAYING]

JOSH GORDON: Hey, everyone.

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it