Placeholder Image

Subtitles section Play video

  • MICHIEL BACCHIANI: My name is Michiel Bacchiani,

  • I am part of the Google speech team.

  • So we make that microphone button on your Android phone

  • do something.

  • So more specifically, I am a senior staff research scientist

  • here at Google, and I lead the acoustic modeling group.

  • That's part of the speech team.

  • And so I've been doing speech recognition

  • research for about 20 years.

  • I've been at other places before here.

  • I worked for about three years in Japan in the ATR--

  • the Advanced Telecommunications Research labs,

  • which is sort of like a Bell Labs of Japan.

  • And I worked at AT&T research for about five and 1/2 years,

  • and I had a bit of a stint at IBM research

  • in upstate New York for about a year and a half.

  • And I've been at Google for almost exactly eight years.

  • And speech recognition is sort of a typical hard

  • applied science.

  • So there's a lot of math a lot of statistical modeling.

  • And as a result of it, we like to process

  • large amounts of data, and so that

  • brings about a large amount of computer science.

  • And so Google looked like a very interesting place to work.

  • So I was intrigued by this place when

  • I was offered the opportunity to do something there.

  • About eight years ago, I looked at it,

  • and it certainly is a very different culture

  • from what I was used to.

  • So when I came from is the typical research

  • lab where there are somewhat of the ivory tower syndrome

  • where you sit together with all the other PhDs

  • and, you know, you're doing exhaustive research,

  • and then there's the development group that's somewhere else,

  • and we don't really talk to each other.

  • And there's a bit of finger pointing

  • going back and forth where the researchers would say,

  • they never take our new idea.

  • And the development group would say

  • that whatever code we get from research is such a mess,

  • we have to all re-implement this.

  • And both of those is true, but it doesn't really work well.

  • And as a result of it, you don't really

  • have an outlet that is built for your product.

  • So you would live off publications, largely,

  • and that's one way to get your stuff out

  • and that would be exciting.

  • But, so now I'm looking at this Google place

  • that is clearly very different.

  • Actually, one of the concerns I had for real

  • is that, are these guys all sitting on colorful skippy

  • balls high fiving each other and saying awesome all day long?

  • Which is not really something I enjoy,

  • because I did enjoy the hard science of what I was doing.

  • Also in retrospect, what I think is very different from Google

  • to do other places, I think, is sort of the three aspects,

  • I think.

  • There's a people component to it,

  • which is we are extremely collaborative

  • between engineering and research.

  • In fact, we make hardly any distinction,

  • where it's hard to distinguish whether somebody is a software

  • engineer or a researcher.

  • And it's just because, if you want to get your stuff out,

  • it has to be engineered.

  • And if you're an engineer and you engineer something,

  • you might run into a problem and you

  • have to research on how to solve that correctly.

  • And so that's a nice environment where people are collaborative

  • and there's no hang ups about what department

  • you're in or what your title is exactly.

  • and you really jointly get something out.

  • I think the second aspect of it is a product.

  • Your stuff really gets used by a lot of people.

  • There's a real path to get whatever

  • you code to get in the hands of people,

  • and actually, people use it.

  • And not only is it sort of a satisfaction

  • that somebody will use your code.

  • There's some interesting stuff that it brings up for research.

  • I mean, it gives you signals to look at on how people are using

  • things, whether it's working for them or not.

  • And you can really focus on a user

  • and sort of make their experience

  • better, which is an interesting research setting in itself.

  • Sort of, how do you make use of those signals

  • to improve your product?

  • And the third thing is sort of impact, because you know,

  • you really can improve people's lives.

  • We did some, you know, I work in the speech teams.

  • Working with some deaf people and giving them

  • opportunities they never had before is exciting.

  • You really, I think, have a more profound impact

  • on somebody's life.

  • And I think a lot of Googlers have

  • this mean altruistic streak where they really

  • like the fact that their stuff can

  • do something good for somebody.

  • And that brings a lot of excitement about.

  • So that might sound a little naive,

  • and naive can be bad, because I have colleagues that

  • look like they're 13-year olds.

  • And I know they're not, but they look like that to me.

  • And they think that the normal world is whatever our Google

  • environment is, and even if you tell them that's not true,

  • you can't really convince them.

  • And we have this rich food culture,

  • as you might have noticed.

  • I literally heard a complaint to somebody that said,

  • do they have to really put truffle oil and everything?

  • There's somewhat of a bubble here, that's true.

  • But in general, I think it's a pretty happy place to work.

  • So more specifically to speech, I

  • think that it is really this kind of company in this mindset

  • that it's not surprising that it took speech, I think,

  • from when I joined eight years ago from sort of user torture.

  • You know, where it's like, say or press one for sales,

  • or say or press two for, you know.

  • To now, we're sitting at home and looking at an ad,

  • and people are like, I like this cellphone

  • because it has voice recognition,

  • as a feature that they actually want.

  • So again, sort of this user focus, I think,

  • is a positive thing.

  • And I think, in more general, it's not just the speech thing

  • where it's been lucky that they had some success.

  • Information retrieval, Google search has been applied

  • and really is useful to people, I think machine translation,

  • I think there are many, many more areas to come.

  • So don't just take it from me.

  • I think we have a very interesting panel

  • with various other people in various other areas

  • that you can quiz and we can discuss things.

  • Some.

  • Of the things I think that we will talk about a little

  • before we go in our Q&A session is

  • to discuss this whole collaboration, which I think

  • is contrasting some of our relationships internally,

  • as well as with the research community outside.

  • So let's start with introductions, I guess,

  • of the panel.

  • So how about we go from left to right.

  • So I'll let you guys do it.

  • UMESH SHANKAR: My name's Umesh Shankar.

  • I've been at Google for about seven years, previous to which

  • I did my Ph.D. at UC Berkeley.

  • I entered Berkeley as a systems student

  • and fairly quickly ended up focusing

  • on security and privacy, which is

  • what I still do here at Google.

  • I'm a software engineer.

  • I lead the data protection engineering group,

  • so we spend pretty much all our days

  • figuring out how to make sure that all

  • the private information that Google stores stays that way.

  • And as far as I'm concerned, Google

  • is really the most interesting place

  • to work on security and privacy, because we don't really

  • have that many theoretical attacks.

  • Most of the stuff that most people think of as theoretical

  • is real.

  • And there's really no playbook for how

  • to do data protection at the scale

  • that we want to do it at with the performance and usability

  • requirements that we have.

  • I mean, it's just been an incredible learning experience

  • for me here.

  • And to a degree that I never really thought I would,

  • I've used almost everything I learned in graduate school.

  • EUGENE WEINSTEIN: Hi, I'm Eugene Weinstein,

  • and I think I'm one of the 13-year-old looking people that

  • Michiel was--

  • MICHIEL BACCHIANI: You wish.

  • EUGENE WEINSTEIN: Well, I don't know, you might disagree,

  • but in the sense that I've never really worked-- well,

  • some small internships aside, I've

  • never really worked at another company other than Google.

  • So I was lucky enough to be able to do

  • kind of the applied part of my Ph.D. Here as an intern

  • while I was a Ph.D. Student at NYU,

  • and I joined here full time about 3 and 1/2 years ago,

  • and I work on speech with Michiel.

  • So I work a lot on the internationalization side

  • of speech.

  • So at Google, internationalization

  • is a big thing.

  • We can't just get away with making our products

  • work in English anymore.

  • But for speech, this is like a whole new ball game.

  • Because it's not just, like, translating some strings,

  • right?

  • We have to model new languages, right?

  • So this is a difficult thing.

  • So I kind of enjoyed being able to scale things up

  • in that respect.

  • Now, voice search supports 40 languages,

  • we just launched four new ones a couple of weeks ago.

  • So that's been so that's been really fun to work on.

  • And the other scale issue that I've

  • gotten to work on that I think has been really

  • meaningful to me is the work that I've

  • done with keeping speech models fresh.

  • The things that people are talking about

  • and the ways in which they say them are changing, right?

  • As demographics change, things come

  • into and out of popularity, and people

  • change the way they search for things with their voice.

  • And we have to take the petabytes of data that's

  • coming in from users of our products

  • and we have to make sure that we're

  • servicing the user in a way that's

  • still current and relevant.

  • And so to me, it's been really fun to take the, let's

  • say, theoretical things that I learned about in grad school

  • and apply them at this ridiculous scale

  • that I could never even conceive before I came here.

  • So it's been really fun.

  • OKSANA YAKHNENKO: Can you hear me?

  • I can't hear myself.

  • So, my name is Oksana Yakhnenko.

  • I've been at Google for a little bit over two years.

  • I joined in like December, 2010.

  • I have my Ph.D. In computer science

  • and I was mostly working on machine learning

  • and applications to various things

  • like text classification, computational biology,

  • and computer vision.

  • And then I've been [INAUDIBLE] in computer vision

  • for about a year and a half.

  • And one of the reasons, when I got on the job market,

  • I was a little bit disillusioned with academia, in terms of,

  • I publish a paper and you spend so much time writing this paper

  • and you so much work in the paper.

  • And then you have like four citations and that's kind

  • of the end of it.

  • I mean, sometimes you have more than that,

  • but I actually felt like my work was affecting

  • a very small niche of people.

  • So basically, I started looking for a place where

  • I could actually do some more impact.

  • And now I have joined a group in Knowledge,

  • so we work in search.

  • And specifically, my group, we don't really

  • do anything that you guys see directly,

  • but we work sort of behind the scenes,

  • and we work on information extraction.

  • So there's this effort called Knowledge Graph,

  • and this is basically the graph of related entities

  • that Knowledge Panels uses to answer

  • the questions and the queries that you

  • guys type in the search engine.

  • And basically, our project is to automatically populate

  • this graph with a lot of unknown entities

  • and unknown relationships.

  • So things that we do, we do a lot of experimentations

  • with like, hey, here I have this model.

  • How good is this model?

  • Not only on paper, that like, this model

  • is going to give me 95% accuracy, which is like, OK,

  • cool.

  • But like, how many users is it going to impact, right?

  • And it's like, even if my model says that it's 95% accurate,

  • do I know that it's 95% accurate?

  • Because not everything that's on the web is true, right?

  • So that was basically some of the more challenging things

  • that I've been working on, things

  • that I enjoyed doing here.

  • GIDEON MANN: Thank you.

  • And thank you, Michiel.

  • I don't think I've heard such an eloquent description of what

  • it's like to work at Google.

  • And I think I would echo a lot of the same things.

  • Especially, Michiel talked about a mean altruistic streak.

  • So I think that, if I have an affliction, or one

  • of the afflictions I might have would be that,

  • and so I've always been looking for opportunities at Google

  • to use the work that I've done to have some kind of a broader

  • impact.

  • And one of the nice things about this place

  • is that you have a platform to do that,

  • and the opportunity to do that, and the support to do that.

  • So in particular, I have worked on machine learning.

  • My background's in natural language processing,

  • but I'm moving a little bit more into machine learning.

  • And in particular, I've been looking

  • at how to make machine learning something that everybody can

  • use, and something that is easy to use,

  • and something that you can drop into your web service,

  • and something that you can training easily.

  • And so, it's been really exciting

  • to pursue that here, because there's so many opportunities

  • to get feedback on whether you're doing the right thing

  • and going in the right direction.

  • So I'll cut myself off there and turn it over,

  • but I'd love to talk about that more with you guys.

  • OMAR BENJELLOUN: So my name is Omar Benjelloun.

  • Can you hear me?

  • So yes, I've been at Google for about seven years.

  • Before that, I did a Ph.D. In databases

  • and I spent two years doing up [INAUDIBLE]

  • also, working on databases and uncertain databases

  • and reference reconciliation problems.

  • And I also joined Google for similar reasons

  • to what was discussed earlier, like having an impact

  • and doing things that are really useful to more

  • than the community of researchers that would maybe

  • read my papers.

  • So at Google, I've been also working

  • on projects related to structured data.

  • And so, I just want to mention two projects that I worked on.

  • One is about, so I lead a project

  • about public data and statistics.

  • And the goal of that project was to

  • and continues to be to get official data

  • about, important statistics about, the world, countries.

  • Your city gets that official data

  • directly from the sources of the data, which

  • are international or national organizations,

  • or local organizations.

  • There are lots of reconciliation data integration problems

  • to solve in order to do that, and provide also means

  • for any provider to give the data themselves

  • and using an easy to understand format.

  • And then reconsolidate all that data in a place

  • where we can make it useful to users through visual tools

  • to explore the data.

  • Visualizations, ways to drill down,

  • compare, and so on in a tool called Public Data Explore.

  • And more recently, we've been surfacing that data directly

  • in Google search.

  • There's a statistics feature that we just

  • launched about three months ago where you can,

  • if you search for a population of India,

  • you'll see graph that's interactive,

  • and you can move your mouse and see the value over time.

  • But we also try to give you context by figuring out

  • what are the most relevant other countries to compare

  • this data to in order to help you understand the content

  • and help you explore.

  • And currently, I'm working on another project in parallel,

  • which is about, again, structured data,

  • but figuring out what is important,

  • how to rank the information that we show you.

  • So if you search for, say, Brad Pitt,

  • we show you a knowledge panel about this actor,

  • and we'll select a few facts that are important about him.

  • So how to select the right set of information

  • and rank that information is this different problem

  • than search ranking, because it's the structured data world.

  • It's a very sort of both research-y and challenging

  • problem, but it has direct applications

  • and direct impact on our users.

  • And so, that's what I find exciting about our work.

  • MICHIEL BACCHIANI: All right, so you have some idea, I think,

  • who is on our panel.

  • So I would like to bring up some of the topics

  • that I brought up, and you can chime in

  • whether you think this is appropriate or not.

  • One of the things is always the perpetual question

  • about researcher versus engineering, and how it's

  • organized, and why they can never get a straight answer out

  • of any of us, whether they're a software

  • engineer or a researcher.

  • So anybody wants to chime in on that?

  • GIDEON MANN: I'll take the bait.

  • So if you can't get a straight answer,

  • I think that's almost by design.

  • I think that at the company, there are really

  • a tremendous number PhDs, and the PhDs

  • are in every part of the organization.

  • I took a training today from someone

  • who had a psychology PhD.

  • And she was in the people [INAUDIBLE].

  • And I was surprised, though I probably shouldn't have been.

  • Because that's really the nature of this organization.

  • It's an amazing place in terms of the level of education

  • and intelligent [INAUDIBLE] and everybody here.

  • So as far as the distinction between these different roles,

  • I don't think there is much, except it's more determined

  • by your local contacts, who your local manager is,

  • and your local group is.

  • Alfred is one of the people that's

  • been thinking about research a lot at Google.

  • Likes to talk about that research

  • being done opportunistically.

  • [INAUDIBLE]

  • I think that that's probably about right.

  • MICHIEL BACCHIANI: I think you want

  • to use the microphone so that we can hear you.

  • when

  • OKSANA YAKHNENKO: So we have to do annual reviews for yourself

  • and each other in the company.

  • So do the publications come into play for research scientists

  • when you guys do your work?

  • MICHIEL BACCHIANI: It's an important issue, I guess.

  • OKSANA YAKHNENKO: Take the blue pill.

  • OK.

  • So when you do your performance evaluation,

  • do the publications-- how much publications actually

  • weight for research scientist?

  • GIDEON MANN: So this is, again, my biased assessment,

  • but I think Google as a company, really judges you on impact.

  • And impact is a lot of different things.

  • Publications, launches, standards, talks,

  • contacts that you develop.

  • So I think that it really, again,

  • is you want to self define your impact

  • and how much, according to the way that you've defined

  • your impact, you've achieved it.

  • Eugene, you feel like you want to add something?

  • EUGENE WEINSTEIN: I definitely see it

  • as, let's say you come to Google and you just

  • do engineering projects, right?

  • And don't write any papers and don't

  • work on what people outside would

  • view as quote unquote "research," right?

  • That's OK.

  • It's an engineering organization, right?

  • But let's say you come here and you

  • expect that you'll be able to just, like, work on papers

  • and do theoretical stuff and not take part

  • in contributing to any products, like some of these more

  • traditional research organizations

  • that Michiel alluded to, that's not really OK.

  • So I think that it can definitely

  • be a boost for researchers when it comes to performance, when

  • it comes to evaluating your impact.

  • But that can't be the totality of the contribution

  • that you bring to the company.

  • MICHIEL BACCHIANI: So can any of you sort of comment on projects

  • that you're doing which you think

  • are essential to have this integration between engineering

  • and research?

  • That you don't think could ever get

  • done in the sort of traditional model, where

  • it's completely distinct silos doing either research

  • or engineering?

  • Or where it would be a much longer process

  • or much more involved to actually get that

  • off the ground?

  • You all work in different areas, so I

  • guess there should be a wealth of examples of things

  • you get out there that you think are

  • good examples of mixing the two.

  • UMESH SHANKAR: This those kind of

  • works if you hold it really, really close to your face.

  • OK.

  • Cool.

  • So maybe my group isn't the best example

  • of this sort of inextricable link,

  • but we're definitely forced to confront

  • kind of the edges of standard practice very, very often.

  • So in cryptography, one of sort of the first things that you

  • learn is, don't make up our own, right?

  • It's like very, very hard to get right and wherever possible,

  • you should use a standard thing.

  • But once in awhile, we come across situations

  • where the standard thing doesn't quite

  • have the properties that we need, and it's

  • times like that I'm really, really glad that we have

  • absolutely world class cryptographers who

  • work at Google, that we talked to

  • and say, OK, what are our options here?

  • And sometimes, they'll actually come up

  • with something different.

  • In other cases they'll say, well, there's

  • this new encryption mode that's sort of experimental.

  • And we believe it has these very specific weaknesses

  • and it either is or isn't appropriate

  • to the particular thing that you're doing.

  • Absent that expertise, I just don't

  • see how we could possibly make a good decision of what to do.

  • Conversely, because you're confronted

  • with such big challenges, and because the culture

  • of the company is one that really encourages people

  • to take big bets, that already tends

  • to blur the line between research and engineering.

  • Because I think a lot of people's mind,

  • engineering is something where, it's sort of really clear

  • what you have to do and you just kind of do it

  • on a quarter by quarter basis.

  • And research is this thing where you really

  • don't know what you are supposed to do,

  • and you get a really long time to do it.

  • In reality, I don't think either of those is really true.

  • I certainly know as a graduate student,

  • like, I had to publish stuff every year

  • to kind of keep my name out there and do a good job, right?

  • and I think for new faculty, it's probably much, much worse

  • before they have tenure.

  • Conversely, here, as I was saying before,

  • there really isn't a playbook.

  • It's very rare that we know in advance

  • what is the right thing to do.

  • And a big part of the challenge is figuring out that thing,

  • trying really bold and innovative things,

  • and finding out what the right thing is.

  • Whether it's published at the end or not,

  • I mean, that sounds like research to me, right?

  • The difference as I was saying to some folks

  • earlier is that it has to work.

  • You don't get to make assumptions that aren't true.

  • That's a really good forcing function, right?

  • And then I certainly know that in my previous time

  • in academia, like, yeah.

  • A lot of people would publish stuff and at a conference,

  • someone would ask a question like, but is that really true?

  • And really, the person brushes it off,

  • and that's kind of the end of it, right?

  • And the difference is that here, you

  • can't do that, because it has to work.

  • And I think that constraint of it having to work

  • is incredibly positive for innovation.

  • It forces you to kind of stop doing things

  • that don't work and try new avenues,

  • and I think the end result is much,

  • much better as a result of it.

  • And I think you're saying also about how

  • you want your impact to be real.

  • And the net result is, that's what everyone wants here.

  • We want to make a real difference,

  • and that's part of that same function.

  • I personally find it extremely satisfied to do that.

  • OMAR BENJELLOUN: Just one last comment.

  • I think one difference that I see between doing research

  • or just innovative work at Google versus in academia

  • is that the constraints that we have here are the real ones.

  • They're the real, most challenging constraints that,

  • if we solve them, then we have the potential

  • to have an impact on-- to solve the problem for real.

  • And I find that in academia, sometimes,

  • at least in my experience, we tend

  • to solve, sometimes, problems within constraints that

  • sometimes can be artificial, or we

  • try to define a set of constraints that

  • make the problem tractable, without knowing if these are

  • the real, most realistic constraints.

  • While here, we have users and they use our services.

  • And if we don't come up with something that works,

  • then it doesn't work.

  • So it has to work

  • GIDEON MANN: Here, I'll give a concrete example of an area.

  • So, distributed optimization.

  • When you have a particular optimization problem

  • with very many variables, larger than it's

  • going to fit on one machine, you want to solve it very quickly.

  • It's important to the business.

  • How do you do that efficiently?

  • How do you do that with low loss?

  • People have been looking at this problem for decades.

  • In the early '80s, there was work out of MIT

  • for tickets to [INAUDIBLE] looked at this problem.

  • But the networks have changed and the problems have changed.

  • And so, there are a lot of people

  • looking at this problem or variants.

  • One of the key parts of the Google Brain project,

  • if you read the paper carefully, is

  • they had a very clever implementation

  • of a large scale distributed optimizer.

  • I mean, as Omar and Umesh have talked about,

  • you're not forced to look at this problem,

  • you're not forced to solve this problem in every context.

  • Here, you're really forced to.

  • You really need both a theoretical perspective

  • and applied perspective to solve it.

  • And then the end result is something pretty spectacular.

  • EUGENE WEINSTEIN: So, we talked a bit

  • about how the constraints are real, right,

  • of what you have to-- oh sure we talked a lot about that.

  • That's an area where we're constrained,

  • but I think it's also worthwhile to bring up

  • the areas where the constraints, the-- how shall I say it,

  • the leeway that we have seems almost so real,

  • which is in the computation infrastructure

  • that we have access to hear Google.

  • Sometimes, you have to solve a research problem just

  • to get the thing to parallelize at the scale

  • that you're allowed to run it at.

  • The fact that we have to answer to Google's users

  • introduces the constraint in that sense

  • that we can't do stuff that's wrong, right?

  • But the situation with the infrastructure

  • that we have here at Google, it makes things that much more,

  • I think, appealing to a computer scientist from just

  • coolness standpoint and from the standpoint of how much you

  • can get done.

  • Sort of like the amount of leverage

  • you can sort of bring to the effort

  • that you are able to make as an individual

  • by using this infrastructure.

  • OKSANA YAKHNENKO: So I think this

  • is to Omar's comment on constraints.

  • So in my perspective, it's almost

  • like your definition of research changes a little bit.

  • So in academia, you have, like, either a set of known problems

  • that you're working on, or you try to make up a problem,

  • and you want to convince everybody else

  • that this is important.

  • Whereas research here is finding-- sometimes actually

  • the problems jump you immediately.

  • Because you have an algorithm that works amazingly well

  • on paper and it works well on, like, a million examples,

  • but then if you try to round it over the entire web,

  • it just doesn't work.

  • So how do you scale it up?

  • So this is something that Eugene brought up.

  • And then there's another way of looking at it is, like, how

  • do I find a problem that I want to solve

  • that's worth pursuing, right?

  • And even answering a question like that

  • is a research in itself.

  • So it's like, what are users want, right?

  • And if I want to get from A to B,

  • what are the resources that I need?

  • What are the kinds of people that I need to talk to?

  • How do I evaluate the results that I have?

  • How do I scale it up?

  • How do I put it into production?

  • Things like that.

  • I think that's a lot cooler and has

  • a lot more-- you learn a lot more, I think,

  • by actually doing research that way than just, like, hey,

  • here's a cool problem.

  • Let me try to come up with another kernel for a support

  • [INAUDIBLE] machine.

  • MICHIEL BACCHIANI: So there's been a fair amount,

  • I think, of academia bashing.

  • But I think all of us have a warm and fuzzy feeling when

  • it comes to academia, in the sense

  • that we engage with the community quite a bit.

  • And I think in various ways, we try to be supportive.

  • So in contrast to all the telling them

  • how they do everything wrong, maybe we

  • should tell them a bit sort of, how we are in favor of them,

  • and try to play nice with them.

  • UMESH SHANKAR: We love you guys.

  • No, seriously.

  • OKSANA YAKHNENKO: Are there people who are in academia?

  • Can you raise your hands please?

  • OK, so we're not offending anybody.

  • MICHIEL BACCHIANI: So I would like

  • to bring up some other things we do, in terms of, you know,

  • we do publish quite a bit.

  • I think it's encouraged, which I think is a bit of a misnomer

  • that we don't care.

  • I think we do give a lot of monetary funds

  • to people that are in academia.

  • We have a giant program, in terms of visiting researchers.

  • I think we try to be collaborative outside as well

  • as inside, and maybe you could share

  • some of you experience with that.

  • GIDEON MANN: So I think, in addition to everything

  • Michiel said, one of the interactions that I personally

  • have enjoyed most is the intern program.

  • Because it's a way not only of getting

  • kind of fresh blood and fresh ideas,

  • but it's also a way of strengthening connections

  • to the academic community.

  • So friends of mine that have gone into academia

  • will send interns my way.

  • And this is a way of kind of keeping

  • those connections strong and also transmitting knowledge

  • back and forth.

  • So I'm thinking of a friend of mine, Noah Smith, who

  • works at CMU/LTI, and he is like super,

  • super into these advanced latent variable models

  • and graphical models, and it's great, actually,

  • to have his-- he has a student here, now.

  • It's great to have his student here and get

  • that learning here.

  • And then I think also, it changes

  • his student's perspective about how to run experiments

  • and what experiments might be like.

  • To say that, you know, can simply

  • run 1,000 iterations or 1,000 different models at once,

  • and what does that mean about the search space?

  • What does that mean about the techniques that you might try

  • and you might want to investigate?

  • And so, I'm hoping that it really--

  • and I'm expecting that it's not going to be a one way street.

  • It really is kind of like an exchange.

  • I had a student from another friend of mine

  • who was working on systems related problems.

  • And I think when he came here, he

  • saw that the depth of kinds of things

  • was very different than the problems he had looked at.

  • And so, it was also a lot of insight

  • that he could bring back.

  • And so that kind of relationship I find especially valuable.

  • And that intern program is vast So

  • among our panel, who has current interns in their team?

  • So I think that gives you a sense.

  • I don't think that's uncommon.

  • UMESH SHANKAR: Just to kind of add

  • to that, as I was saying before, and it sounds kind of funny

  • but its actually also true.

  • You know, as a proof that academic research really

  • is valuable is that I really did end up

  • using so much of what I learned in operating

  • systems, databases, networking, of course security and privacy,

  • all these areas.

  • The concepts that were developed in academia, things

  • that kind of maybe seem far fetched

  • at the time, that only made sense

  • to use an algorithm at an extremely large scale.

  • Well, that time arrive.

  • We're very grateful that kind of work had been done.

  • And I think I wholeheartedly agree

  • about the kind of interchange between a place like Google

  • and academia.

  • I mean, we go to conferences.

  • That's a great example of that, where, not just Google, right,

  • but many other companies in the industry

  • have people who actively work on research topics, who publish.

  • You go to the conferences, you learn from each other, right?

  • And in some sense, the conferences, like,

  • shows you how many similarities there

  • are in the kinds of problems that people work.

  • There is also something that, the flip side of my comment

  • earlier that the things you do have to work,

  • is that it takes time to keep them working.

  • And there's a certain tax that you pay,

  • often substantial, for that.

  • And in academia, it is a little bit freer

  • of those sorts of constraints, and there's

  • something very nice about that too.

  • It lets you go in perhaps more different directions

  • than you would otherwise.

  • If I were to draw another sort of,

  • like, compare and contrast between academia and a place

  • like Google is that I think Google actually really rewards

  • taking long term big bets in maybe a smaller number of areas

  • as an individual, which can have a really big payoff.

  • Of course, the flip side is, maybe it doesn't work out

  • and now you're like, man, I just a couple of years on this thing

  • and it didn't end up really working out.

  • Academia, I think, in general, it's quite common for people

  • to work on more different projects at once,

  • and kind of, OK, the idea captures your fancy,

  • you can kind of start working on it, even

  • more so than you could at a place like Google.

  • And I think having both those models out

  • there probably is a really good thing, in total, for when

  • you look at what we, together, can produce.

  • MICHIEL BACCHIANI: Gideon disagrees

  • GIDEON MANN: You know, I'm not sure I disagree,

  • but I don't think I've ever thought about that way.

  • It's really interesting.

  • I think that you're right.

  • I mean, there's like a historical baggage

  • you start to accumulate of, this is

  • what the world, meaning Google, ecosystem looks like,

  • and that's what you have to operate in.

  • I don't think I've ever thought about it that way.

  • Interesting

  • MICHIEL BACCHIANI: Sometimes with publishing,

  • this is something that I think I frequently hear,

  • is that we are not allowed to published.

  • There's, I think a rumor that I've-- not universally,

  • but that's something I've heard frequently that--

  • or it's discouraged, or whatever.

  • And I think it's very unfair, but again, this

  • is one person saying it.

  • But you guys want to comment on whether you think

  • this is a fair statement, or where does this perception

  • come from, if you had to guess?

  • OKSANA YAKHNENKO: I think the perception comes

  • from that a lot of people who actually go to Google sort of,

  • like, disappear for a couple of years, and they don't publish.

  • The reason they don't publish, I think, is actually twofold.

  • Not enough time and not enough time to write a paper.

  • Because you get so interested in what it is that you're doing

  • and you become so involved in actually getting the thing

  • to work and seeing the impact of this thing.

  • Sometimes, you just don't really care to write it up,

  • or sometimes you just don't really

  • have enough time to write it up.

  • With that said, when I joined I still

  • had a couple of leftover projects for my grad-work,

  • for my graduate work.

  • And I took like two weeks or three weeks

  • that I was allowed to use Google's resources,

  • and like even like Google [INAUDIBLE]

  • actually wrap it up and submit a publication that got accepted.

  • They sent me to the conference.

  • And this was, like, for the work that I've

  • done that was still the ideas that where

  • the academic ideas when I was in school.

  • I'm an engineer, but I've done a couple of collaborations

  • with people in research where it's sort of like a two way

  • street, like where I tell them the types of problems

  • that we're really interested in, and I simply just don't really

  • have the time to look into it and where somebody

  • from research actually was like, oh yeah, cool, yeah.

  • This is like my area of expertise.

  • I have an algorithm that could be well suitable for this.

  • I can run some experiments and see

  • what will come will come out of this.

  • And then basically from something like this,

  • a paper evolved, and we actually submitted a paper

  • to conference.

  • In terms of submitting a paper it's like super easy.

  • This person worked for Corina Cortez,

  • and it was basically like telling Corina about the paper,

  • and she was like, yep, cool, you can submit.

  • Like, this is the level of approval

  • that it had to go through, pretty much.

  • EUGENE WEINSTEIN: I think with respect to publishing,

  • the things that we're saying here,

  • you guys don't have to take our word for it.

  • Just go to research.google.com, right,

  • and you'll see that there are hundreds

  • of papers coming out of Google.

  • These aren't only people that are

  • in the research organization.

  • So I think, for most people here,

  • I think it's pretty much a personal choice

  • about whether you care, like Oksana said,

  • whether you're motivated by getting your work out there

  • in front of people outside of Google,

  • or whether you want to focus more internally

  • about presenting your work to your colleagues

  • and getting buy-in from them and getting collaboration

  • opportunities there.

  • It's very much a personal choice,

  • but if that's really your passion,

  • I would say that the vast majority of the people

  • are able to publish, and publishing

  • is encouraged by managers, by VPs.

  • MICHIEL BACCHIANI: We have a question.

  • AUDIENCE: So I wonder, talk a little bit about the origin

  • of problems.

  • Obviously Google has lots of products that do things,

  • and clearly problems that are directly related

  • to those things are important and valuable to the company.

  • Is it possible to have an idea that maybe doesn't

  • have such an obvious connection to something for the company,

  • and to what extent would you be able to actually run with that?

  • A month?

  • A year?

  • Two years?

  • Whatever.

  • Does that make sense?

  • EUGENE WEINSTEIN: So I don't know if you're familiar

  • with the Chauffeur project, the self-driving car out

  • in Mountain View, so that's one example of something that's

  • totally not connected to the mission of the company,

  • as most people would see it.

  • So there are things like that.

  • Like, would I be able to, right now, go to my manager

  • and say, can I go and spend 50% of my time

  • building a self driving car?

  • Maybe not.

  • That's not always the case, but there

  • are these things that happen in that kind of organic way

  • and that aren't always directly related

  • to what your job role is.

  • MICHIEL BACCHIANI: So it seems to be the perfect time

  • to bring up the 20% project, which isn't quite 50%,

  • but, you know.

  • In contrast to what you said about the freedom

  • to do off-the-wall things, I think

  • there is some provision that provides a lot of people

  • to do so.

  • I think a lot of us, because we're so ingrained what we do

  • and like what we do, we don't really make use of it.

  • But it's a reality that exists, I think.

  • Maybe can you guys comment on whether you

  • have people doing 20% things?

  • OMAR BENJELLOUN: So I considered as a manager,

  • I've had many of the people on my team

  • who are interested in doing 20% projects,

  • and I've always encouraged them to do that,

  • especially if it's something that's

  • going to help them learn a new skill

  • or advance in their career or establish new connections

  • or find new, potentially interesting areas to develop.

  • And it's generally been a very positive experience.

  • Some people do it for a long time,

  • some people do it short period of time,

  • and then they stop doing it and then they come back to it.

  • It varies.

  • It varies a lot.

  • One other comment I wanted to make is, also, in the projects

  • that we work on, there's sort of a mix of projects.

  • So I think that many product areas at Google

  • have some established sort of short term projects

  • that they know they want to get done and launch quickly,

  • but they also have some sort of fraction of projects that

  • are more sort of exploratory and are trying to do explore sort

  • of longer term opportunities, and those

  • can be more researchy.

  • And it's always possible to go and propose an idea

  • and say, hey, I think there's something really interesting

  • here, and that's an interesting challenge,

  • and if we solve this problem, this

  • is going to have a big impact on what we're doing.

  • And if you convince your director,

  • your VP, then you might even get some headcount and engineers

  • that can work on this project.

  • GIDEON MANN: So Tom, to address your question,

  • I think one thing that I didn't realize about this place,

  • before I started is the autonomy and self

  • direction that each individual engineer has.

  • They have it in the choice of project,

  • but they also have it in their interest of time.

  • And so, you are so valuable as an engineer to the company

  • that you are actually a prized resource,

  • and the company's goal to some degree

  • is to keep you as excited as possible.

  • And so, to some degree, that means

  • that you can kind of move and your place of interest.

  • Now, I think what people usually find

  • is that after a certain point there,

  • they start thinking about aligning themselves

  • to the company's larger interest.

  • Now how can I support this larger area

  • that the company is going in?

  • It means that your projects will move faster,

  • it means that you'll move faster,

  • it means you'll get more support,

  • more other people will be interested in joining

  • your torch-bearing cause.

  • But it really is kind of your own discretion.

  • I don't think I understood how much that would

  • be true about being here, but it certainly is.

  • UMESH SHANKAR: Just to kind of follow up

  • on that very briefly, I think like the usual process by which

  • this kind of thing happens is, you start small,

  • like you build a prototype.

  • And the real process by which things get kind of sorted out

  • and what ends up making it and what doesn't is,

  • like, there's a marketplace of people's time

  • because people's time is really valuable.

  • And if your idea is really good, and you convince

  • people to work on it, then your project will grow.

  • And if you can convince people that your idea's good,

  • maybe it's not that good.

  • Because maybe it is and we're missing an opportunity,

  • but I think, on the whole, the system works pretty well.

  • And it works very organically, that means

  • that you have to probably invest more

  • time in learn skills of persuasion and things

  • that you might otherwise to get it off the ground.

  • That's not necessarily a bad thing, either.

  • So yeah, small things can become big.

  • If anything, I think maybe one mistake

  • that people sometimes make in trying

  • to start new projects is thinking too small.

  • It's hard to get people to want to sign up to something which,

  • even if it all works out great, still ends up being small.

  • That's just not that exciting.

  • So having a big vision for even something that

  • starts out small really ends up being key.

  • And that's really, I think, a lot of Google's all about.

  • GIDEON MANN: I just want to piggyback on

  • that I totally agree.

  • And I think one feedback I get from-- I

  • met with Jennifer Rexburg, who's a prof at Princeton,

  • and she was like, it's amazing, I meet all these engineers

  • at Google and they're all so able to communicate.

  • And I think, partly, that's not the interview process.

  • I think, partly, it's the work process.

  • You're forced in kind of your daily process

  • to communicate with other people.

  • That is kind of what will propel you and communicate

  • about your ideas.

  • And so, just as Umesh was saying,

  • it's like, that communication, persuasion, building

  • the vision, that is kind of the part that

  • will drive you on your project.

  • So agenda-wise, I think we're getting

  • close to starting the Q&A session.

  • I've noticed that we have more and more questions,

  • so I would like to encourage it even more.

  • AUDIENCE: This is a segue [INAUDIBLE].

  • I understand that you are all PhDs in computer science

  • [INAUDIBLE] as far as I can tell,

  • seem to work on areas within Google that

  • follow from graduate research context

  • in a specific area for your dissertations.

  • And so I was wondering, in part because I am a math PhD, not

  • a computer science PhD, about the adjustment process

  • that some people have [INAUDIBLE] Google, as friends

  • of mine have PhDs in biophysics and then

  • they start to work on video chats.

  • One example that is true.

  • Because I think it's kind of an adjustment process

  • to be a database researcher in academia,

  • and then a database researcher at Google,

  • and it's another kind adjustment process to be a mathematician

  • and then to work on [INAUDIBLE].

  • What kinds of [INAUDIBLE]?

  • Just the kinds of resources and learning

  • curve that people face in that?

  • OKSANA YAKHNENKO: So, most likely, if you are,

  • what was your example again?

  • If you're a mathematician, well, if you're a mathematician, most

  • likely, you will not be hired as a front end developer.

  • Mostly because, as far as I know,

  • and maybe actually, people who work in the HR

  • maybe will be able to comment a little bit more on that,

  • but that is as far as I know, they

  • try to put you on a project where you really

  • can contribute.

  • Because, yeah.

  • STU: May I try answering that one?

  • I'm not on the panel, I apologize.

  • [INAUDIBLE]

  • We have, [INAUDIBLE] one of the earlier comments.

  • Last time I asked, we had some 3000 people [INAUDIBLE],

  • some of them are none of the above. [INAUDIBLE].

  • A whole bunch of psychology.

  • [INAUDIBLE] a few medical doctors, Juris doctors,

  • [INAUDIBLE].

  • But we have over 2500 in the computing cluster, which

  • I will define as math, science, and engineering.

  • And then we have another 300 or 400 in biophysics, [INAUDIBLE]

  • which is as far out as you can get.

  • And [INAUDIBLE] hiring outside research.

  • Yes, we do take a look at skills.

  • If you're going to come in as an engineer,

  • you'd probably [INAUDIBLE] real programmer.

  • And then we actually do a sorting hat

  • with a serious effort to find a plausible place for what

  • you know.

  • We're assuming that you're a Ph.D. [INAUDIBLE].

  • And yes, of course there are courses

  • on this, that, and the other. [INAUDIBLE].

  • And your colleagues will contribute [INAUDIBLE]

  • but the goal is we can be doing something useful about a month

  • after you show up [INAUDIBLE] in courses and learning

  • [INAUDIBLE] how to make the elevators work.

  • But after that, we expect you'll be

  • doing something kind of useful in a month,

  • and be valuably up to speed in [INAUDIBLE].

  • And by useful, I mean a team [INAUDIBLE]

  • depending on you to actually do something. [INAUDIBLE].

  • MICHIEL BACCHIANI: More questions.

  • AUDIENCE: The problem between academic research

  • and corporate research is that every very smart idea

  • you guys have [INAUDIBLE] is the competitive advantage.

  • So How does that conflict with publishing?

  • GIDEON MANN: I'll take that one.

  • I had in mind to directly address this

  • when Michiel was talking about publishing.

  • I think that it is very clear that the fruits of all

  • this labor is material advantage.

  • And that it is valuable work, it's

  • important work, and that it makes

  • a difference at the company.

  • And so I think that everything that gets published,

  • people think about, OK, well, how much are we actually giving

  • away?

  • And I would say the things that are, in particular, I would

  • say Urs is probably the gatekeeper.

  • And if he feels like we're giving too much away,

  • he'll say, let's maybe either delay it

  • until the competitors have got in close enough,

  • or let's hide some of the most key pieces,

  • not so that it's useless, but just so

  • that we don't give away the bank.

  • And so, I think that, yes, clearly it has value,

  • and clearly we don't want to give away.

  • And so I think that.

  • But in my experience, to give an example,

  • we published a paper on distributed optimization.

  • And they didn't want us to directly say

  • the number of cycles, or the amount of time things

  • had taken up on that racks or on the machines.

  • So we converted it to some arbitrary scale,

  • and that was enough to satisfy people

  • that we weren't giving details away.

  • So I think that there's always room for negotiation,

  • even within that space.

  • AUDIENCE: [INAUDIBLE].

  • GIDEON MANN: So, my impression is that Google's general

  • position on patents is that they wish there weren't.

  • They didn't exist.

  • Google is involved in so much patent litigation

  • that I think it would be beneficial to the company

  • if the whole patent system was more evolved.

  • Maybe Stu has a slightly different perspective.

  • STU: True, but it ain't going away.

  • So yes.

  • Somebody will look to the publication [INAUDIBLE]

  • might have been patented and submitted, in which case,

  • some [INAUDIBLE].

  • We would be delighted to disarm, we just can't.

  • GIDEON MANN: But I think to--

  • STU: Have that conversation.

  • GIDEON MANN: But, to that point, I think that it's not--

  • STU: It rarely delays you more than a couple weeks

  • to actually get the lawyers [INAUDIBLE]

  • and [INAUDIBLE] probably one paper in 100 [INAUDIBLE].

  • GIDEON MANN: Oh, but I think part of the question was,

  • how important to your career is patent filing,

  • and I think it's--

  • STU: Very little.

  • GIDEON MANN: Yes,

  • MICHIEL BACCHIANI: There's a question over here.

  • AUDIENCE: Obviously, [INAUDIBLE] products here [INAUDIBLE].

  • So, excuse me, so [INAUDIBLE].

  • Because people somehow appear to be [INAUDIBLE].

  • EUGENE WEINSTEIN: Yes definitely.

  • So you mentioned 3,000, right?

  • So Google is, what, 50,000 now?

  • STU: Has 2500 PhD's, and nobody ever

  • uses the title unless they're a psychologist

  • or a lawyer or a medical doctor.

  • So unless you actually ask, you probably

  • won't know [INAUDIBLE] to get a PhD [INAUDIBLE]

  • actually pretty damn good.

  • AUDIENCE: [INAUDIBLE].

  • MICHIEL BACCHIANI: So I think it's not so much that it'll

  • give you an edge, let's say, in title

  • or in how much people respect your value, your opinion.

  • It won't get you that.

  • It's more the ability that you'll

  • have to bring that research kind of flavor to the work

  • that you do.

  • And I think in turn, because you will

  • be able to show that kind of skill,

  • in turn, that will get you the respect

  • and buy in from your colleagues.

  • It won't be the fact that you have the Ph.D. There was also

  • the adjustment aspect of the questions

  • OKSANA YAKHNENKO: I think one thing

  • that you'll learn as a Ph.D. Student,

  • and it's mostly because you spend, like, five years working

  • on one problem, is you really learn how to problem solve.

  • So it's not necessarily the title that gives you the edge,

  • but it's the fact that you, in the process of getting the PhD,

  • you learn how to solve problems.

  • So think about it.

  • If you come to Google, you get thrown

  • into an environment where most likely,

  • you will be working on something that is or is not related

  • to your Ph.D. Or to your research project directly,

  • you still know how to approach a problem, how to solve it,

  • how to evaluate your results.

  • And I think that's like a great skill to have, for anybody.

  • And I think most of people who actually work at Google

  • have that skill already.

  • So in terms of adjustment, I don't know.

  • GIDEON MANN: I'll speak to adjustment.

  • I was a post-doc, and I think my biggest adjustment coming here

  • was that Google is not very hierarchical.

  • And so, it's like when you're a graduate student or a post-doc,

  • you have your adviser, and your adviser

  • has a final say, to some degree.

  • I mean, about what you do.

  • It's really not like that here.

  • It's a very big company, there are

  • a lot of very different opinions,

  • and as an individual engineer, you

  • have a huge amount of autonomy and a huge of ability

  • to persuade that is kind of separate from your manager

  • and from your team.

  • So one of the ways that the engineering groups work

  • is that there's this interesting kind

  • of a triad that decides how things happen.

  • It's the manager of the team, it's the tech lead of the team,

  • and the PM, potentially, of the team.

  • And that triad is also subject to the whims

  • of all the people on the team.

  • And so it's like a very complicated process

  • how things get done.

  • And I think that understanding that model, which

  • is different from the academic model,

  • I think that is the biggest adjustment,

  • and I think a healthy adjustment.

  • I think academia is very rigid.

  • But I think that's a big adjustment.

  • OMAR BENJELLOUN: Just one more comment

  • I want to make on that question.

  • I think there's a very strong computer science

  • culture at Google, and engineering culture, but also

  • computer science.

  • So if you come with a view of problems

  • which is sort of academic or researchy,

  • or wants to think about problems in a fairly rigorous way

  • and model them property and solve them property and prove

  • that you have the right solution,

  • this is something that's very respected.

  • And so, I think at Google more than anywhere else,

  • doing things right and solving the problems sort

  • of at a fundamental level is something

  • that is valued and respected.

  • AUDIENCE: I have a a comment first and then a question.

  • [INAUDIBLE] question.

  • The comment, well, you asked, or somebody [INAUDIBLE]

  • the idea that people [INAUDIBLE] publishing.

  • Well, the fact is, I'm in software engineering.

  • I go to software engineering conferences.

  • I publish in software engineering.

  • And you see [INAUDIBLE].

  • I mean, it's not a value judgement,

  • it's just a fact if I look at the program.

  • So people come away saying, OK, there's

  • lots of PhD's at Google, there are

  • lots of smart people at Google.

  • They're not [INAUDIBLE] in software engineering.

  • What they do in software engineering [INAUDIBLE] Well,

  • why is that?

  • And so, I think that's where people get the idea.

  • It's a very pragmatic [INAUDIBLE].

  • MICHIEL BACCHIANI: The comment I have, and this

  • is my personal view of it, but I think

  • the panel said it to some extent.

  • But people join here, and it's a very different environment.

  • So they have this engineering team mates

  • that really want to get something going.

  • And at some point, you have the system out there,

  • and it serves people, and there's traffic going in,

  • and you have the signals coming back

  • from that which you want to optimize.

  • So to some extent it's sort of like taking a break

  • and doing the paper publications,

  • and sort of formally write it up and doing the measurements

  • in some kind of controlled way, which isn't usually frequent

  • the way that an application is deployed, where it's just,

  • you get more and more traffic.

  • A lot of the things are not very well controlled in that sense.

  • It makes it harder to do the paper

  • and sort of becomes a distraction.

  • I think they get so roped up into it.

  • AUDIENCE: I was saying [INAUDIBLE]

  • I was responding to the question that was asked, where do people

  • get this idea from?

  • I'm saying, that's where people get the idea from.

  • MICHIEL BACCHIANI: I mean, to some extent, I bring it up

  • because I think the misperception is that there's

  • some kind of corporate shut down on any kind of publication

  • because that's not what we value.

  • I think that's very unfair to address us like that.

  • AUDIENCE: OK.

  • So, my question for you, though, which is completely [INAUDIBLE]

  • is you spend a lot of time comparing Google to academia.

  • Now, I'm somebody who's, my father's attended

  • [INAUDIBLE] which, the minute I moved there, became AT&T Labs.

  • And I spent a lot of time at Google Research [INAUDIBLE].

  • So as somebody who spent time in AT&T Labs, I

  • ask you how you compare [INAUDIBLE] now,

  • of course, you went there in a different life

  • stage. [INAUDIBLE] According to this is a brand new PhD.

  • [INAUDIBLE] How would you compare life

  • at AT&T Research versus life at Google?

  • MICHIEL BACCHIANI: So, when I joined AT&T Research,

  • I gave you the context of getting

  • hired at AT&T. I remember I was done with grad school

  • and had lined up five or six interviews.

  • And the first interview I did was AT&T

  • because whoever was my contact there was very proactive

  • and got that going, so it was becoming the first interview

  • that I did.

  • And I visited one day and had the interview process.

  • And it was actually so much fun that I came back the next day,

  • and this is '99, right?

  • So the market is hot, I guess.

  • And they're worried00 I didn't even factor in my mind,

  • but they were eager to get people.

  • So the next morning, I get a phone call saying,

  • like, I still have to do the paperwork and the approval

  • or whatever, but you're going to get an offer.

  • At that point, I literally canceled

  • all my other interviews because I liked it that much.

  • So I loved AT&T Research When I joined,

  • it was really a fun place to be.

  • But I also felt that in my years there, I mean,

  • there were nasty things because the company was going down

  • the drain.

  • But AT&T Research in itself was

  • AUDIENCE: You have no idea how far down the drain--

  • MICHIEL BACCHIANI: But I mean, AT&T research in itself,

  • I think, was the greatest place.

  • Because all my colleagues there were famous,

  • so there was no attitude.

  • They was sort of like, do good work.

  • There was a lot of interesting people to talk to.

  • There was a collaboration within there,

  • but the rest of the company was alien.

  • In fact, I remember making fun of the rest of the company

  • where you'd have these town hall meetings,

  • and we didn't really understand what people are talking about,

  • because it's like, the rest of the company, to some extent.

  • So I think the contrast is, like, all of this

  • is AT&T research.

  • The entire company.

  • It's very open, collegial, sort of collaborative,

  • many aspects to it.

  • So I think that's the best comparison.

  • AUDIENCE: [INAUDIBLE].

  • GIDEON MANN: I'll take that.

  • And so in 2009, I had this project I wanted to work on.

  • I found a guy, [INAUDIBLE] Silverman,

  • who was at NYU HD program.

  • I convinced him that we should work on this,

  • and as part of the project, [INAUDIBLE]

  • in spreadsheets, machine learning,

  • he checked out the spreadsheet's curve, he built it locally

  • and he embedded our software into the spreadsheets.

  • I thought it was the coolest crap I'd ever seen.

  • And then he produced this demo which

  • showed what it would be like to have this code in actually

  • the spreadsheet.

  • It was like a different group, a group that

  • sat-- I think they may have at that point

  • sat on the same floor as we did.

  • But we didn't know them at all.

  • And so we built the prototype, he and I [INAUDIBLE]

  • mostly his work.

  • And then he showed the demo to the group,

  • and it was a very exciting moment,

  • because we're able to show them their software doing something

  • that they hadn't expected before.

  • And so that particular group kind of

  • laid dormant for a very long time.

  • And I think recently, we went back

  • and built a lot of those pieces in a different direction.

  • But I think that pathway is a very common pathway.

  • You see someone else's code, you check it out, you build it,

  • you modify it, you talk to them, you try to get their feedback,

  • you try to plan a way forwards, and it kind of moves this.

  • And that's just saying it's organic.

  • It's like, you can see just about any code in that company.

  • You can send an email to just about anybody

  • and they will respond.

  • Actually, the more senior are, the quicker they respond.

  • And I don't know why it works that way, but it seems to.

  • As far as collaboration, internal collaboration,

  • I think it is a wonderland.

  • It's fantastic.

  • people are very eager, they're very interested,

  • and it's a nice place for that aspect of it.

  • AUDIENCE: This one small anecdote.

  • I worked in IBM research for quite a while,

  • and IBM Research and IBM Software Group

  • were two completely separate entities.

  • And we had a prototype we wanted to build into the product,

  • and it was a nightmare to get their hands on code.

  • And there was envy, and protective, and politics.

  • I mean, here, we have access to almost all of the code,

  • you can talk to almost everybody,

  • mean there is not the question of what forums are there

  • for communication.

  • The company is the forum for communication.

  • OKSANA YAKHNENKO: And just to add to that,

  • we have tons of resources to even like, you know,

  • finding who's working on related projects.

  • For example, everybody has like an internal page,

  • and anybody in the company can actually

  • access your internal page.

  • All the projects have internal websites

  • with, like, members, what they work on,

  • what they're interested in, there's

  • a mailing list for that.

  • Then there are mailing lists on pretty much anything

  • you can imagine, so there's like a mailing list for machine

  • learning, and there's a mailing list

  • for some internal specific tools,

  • or like, mailing lists for some of the products

  • that are being developed.

  • It also happens from, well, in my experience

  • anyway, is that, when I joined the company,

  • I already knew a lot of people, not necessarily in New York,

  • but in other offices.

  • And just like, people I've met at conferences, or like people

  • who were in the same field.

  • And remote communication is like super easy.

  • You literally go to a conference room, you press the button,

  • and that person is on the screen in front of you.

  • Magic.

  • And then, as Gideon said, people are really, really willing,

  • and really supportive.

  • If you want to set up a meeting with somebody,

  • you don't even have to, like, sometimes you can just put it

  • on the calendar, and they will show up.

  • It's like, it's magic.

  • EUGENE WEINSTEIN: Also just to add one note about what venues

  • there are for fostering collaborations.

  • So we have a thing called beer and demos,

  • which is, it's on Friday, right?

  • Right, so you can show up to this room,

  • It's very much like this, and you

  • can have beer and watch people demoing their half working,

  • or quarter working, or it might work, kind of demos.

  • And if you have interest, if you're interested in the work,

  • then you can come up to them and create

  • a collaboration that way.

  • We also have millions of tech talks, people, both internal

  • and external speakers coming to present their work.

  • And you know, there are visiting faculty,

  • there are other people that are around

  • and want to share their ideas with you

  • and are interested in creating collaboration.

  • So there's many venues that we haven't thought of.

  • It Sorry?

  • MICHIEL BACCHIANI: So you could be needing for 10 minutes

  • already, so any questions that you think

  • are really, really important to be answered by the panel rather

  • than the person that will probably

  • be sitting at your table?

  • AUDIENCE: Ask a question?

  • MICHIEL BACCHIANI: Yes absolutely

  • AUDIENCE: [INAUDIBLE].

  • STU: I suspect that some of our offices

  • in other countries [INAUDIBLE].

  • MICHIEL BACCHIANI: One of my favorite quotes

  • is, some people in other places, probably for good reasons.

  • Final question?

  • AUDIENCE: So, [INAUDIBLE] it's most important, in response

  • to the [INAUDIBLE] I was wondering

  • what the extent of that was and how common [INAUDIBLE].

  • UMESH SHANKAR: Well, in some sense,

  • it's part of the price of success.

  • You build something and it's not being used,

  • you don't pay the tax, but that's not really

  • where people want to end up either.

  • And part of it, too, is I think you sort of build

  • in the cost of maintenance to your utility function

  • of how good your thing is.

  • Where that just becomes part of how

  • you decide if a system is good.

  • Right?

  • It's not too good, even if it just works,

  • or even if it works well.

  • It's how easy is it to modify over time?

  • How easy is it to maintain over time?

  • How much of your resources or your team's resources

  • is it taking?

  • And so, you get better at it.

  • Sort of naturally, things often evolves to a state

  • where you spend your time either on doing new stuff

  • or making the things that you already have more efficient

  • and taking less maintenance burden so you

  • get to a good state again.

  • So yeah, it definitely various, I think.

  • Working on infrastructure, your definition of success

  • is making people that use your stuff successful in a sense,

  • and that's what my team's work on

  • is largely in the infrastructure area.

  • So we probably pay a larger tax that most,

  • but some of what-- I sort of alternate

  • between calling it a tax and learning.

  • Because a lot of the times, the things

  • that we spend a lot of time with are basically customer service.

  • So a new team is integrating with some technologies

  • that we've built.

  • We've built security and privacy infrastructure,

  • and they'll have a novel scenario, right?

  • They have different data access patterns from ones

  • we've seen before, they're doing some mash up

  • of kind of more internal facing systems

  • with externally facing systems, and that's a new thing,

  • and they're querying data in a different way,

  • all this kind of stuff.

  • OK, we haven't seen it before, right?

  • And so it's like, ah man, we've got

  • to put down the code that we're writing

  • and help them figure out the right thing to do.

  • Maybe we have to change our code,

  • maybe they have to change their design,

  • maybe we just come up with something clever.

  • But the net effect of those things

  • is that, when you design the next version of your system,

  • you've already taken those into account.

  • So I think maybe I was a little harsh

  • in calling it always a tax, exactly.

  • There is certainly an element of that, right?

  • You end up doing some stuff kind of a lot, and it's like ugh.

  • Got to update something again, right?

  • Or some machine broke somewhere and now

  • you've got to fix something.

  • So that kind of stuff happens, right?

  • But I said over time, the trend is towards, OK, well,

  • if some machine broke for the 10th time

  • and I'm having to spend the 10th time to fix it,

  • maybe I need to engineer around that problem.

  • And now you have a much more robust system too.

  • And in terms of customer service,

  • maybe you build your system in a way

  • that people don't have to ask as many questions,

  • and it's just more natural and easy for them to use.

  • And that, too, is progress, so yeah, you

  • try to find a good equilibrium.

MICHIEL BACCHIANI: My name is Michiel Bacchiani,

Subtitles and vocabulary

Click the word to look it up Click the word to find further inforamtion about it