Subtitles section Play video
[MUSIC PLAYING]
DAN AHARON: Hi, everyone.
I'm pleased to be here with you.
I hope you're having an awesome Google Cloud Next.
I'm here to talk to you about something that I'm personally
very excited about, and we at Google
are very excited about, which is conversational applications,
which includes Bots, and I hope you're
as excited as me at the beginning of the session,
or if not, by the end of it.
Here we're going to introduce API.AI
to you guys, which is a tool probably many of you
are familiar with.
It's our Bot application development platform
that we acquired last year.
And we're also going to go beyond the basics
and show you guys a little bit more tips and tricks.
So you're going to hear from a couple people, Cornelius
and Petr, that have been using API.AI,
and they're going to share some of the lessons
that they've learned while building conversational Bots.
So, this is one of three different sessions
we have on this topic of conversational applications.
We had one yesterday which focused
on cloud functions and surveillance and architecture
to serve conversational applications.
We're now doing the middle session
that you see on the slide.
And then, following after me, Brad and Guilliame
are going to show you how to extend Google assistant
with actions on Google.
So, what is a conversational agent platform?
We'll just go through this quickly, probably a lot of you
already know, but, basically what it does
is it takes natural language and it turns it
into structured data that you can then
use in your applications.
So, for example, if a user orders
a pizza and says what ingredients it has,
the software will turn that into structured data which
says what's exactly inside the pizza, what type it is,
and then you can actually act on it.
What it doesn't do is, it doesn't really
understand your domain and your vertical
until you actually teach it and train it.
You can give training examples that
explain exactly what to do.
And, it also doesn't do any fulfillment.
It's not actually going to bake a pizza
and deliver it for you, unfortunately.
That would be nice.
But, the good thing is you're here at Google Cloud Next,
you've probably been to a bunch of sessions.
Most of this conference is about how
to fulfill the back-end of your application,
and we think Google Cloud Platform is a great way
to do that.
So, it's a very good complement to API.AI
And then, specifically, we're very proud of API.AI,
we think it's pretty distinctive in the space.
A few of the important benefits that API.AI
has that it is an end-to-end suite,
it really combines a bunch of different elements
that you need for building compositional applications.
It doesn't just do natural language understanding, or just
one component.
It can get pretty smart very quickly.
You're going to see that later on.
I'm going to attempt to do something that's probably very
risky, and not very wise of me.
I'm going to try and build in front of you,
from scratch, a whole new conversational application,
in less than 20 minutes, that handles an imaginary scenario.
Hopefully, it will work, we'll see.
But, you can see the power of API.AI
with a very small data set, what it's able to do.
It's multi-lingual, it already supports 40 languages,
and we're investing in growing that.
We want it to be as global as Google is.
There was a lot of speculation last year
when we bought API.AI that we will turn it
into sort of a closed system that only
works with Google software.
The truth is, it's far from it.
API.AI works with all of the different messaging platforms
that are out there.
And we're very proud of that.
We're a very open company at Google,
we actually want you to build the best
conversational applications that work across all the platforms
that you're interested in.
And we want to keep making API.AI the best
cross-platform tool.
So we're going to continue to support
all of the platforms that are out there.
I'm just going to go through this very quickly,
but this is just an example architecture
of how you can build a conversational application.
So, on the left-hand side, you can
see all of the different channels
where conversations can come in.
So, it starts from owned and operated websites
or web apps or mobile apps.
And you could have Google Assistant properties
like Pixel, Aloe, and Home, or you could have integrations
with the different applications that I just showed you,
all of the different messaging applications.
The other thing is you could have voice calls come in
and be transcribed with Cloud Speech API, and flown in.
All of those can get processed by API.AI,
which handles the conversational aspect, including context,
natural language understanding, and figures
out how to involve the back-end.
And then, through web hook integration,
you can connect it to Google Cloud Platform
to handle all of your back-end needs, or any other platform
that you want.
You can connect to other external systems like ERP,
like CRM, or whatever you need on the back-end.
You can get there through cloud functions or any other Google
top-form application cloud service.
OK, so now to the demo.
So what I'm going to do now, we're
going to look at an imaginary Google hardware store.
And we're going to try and create
a conversational app that processes requests
for that imaginary store, including service
requests and commerce requests.
So we're going to just quickly go through defining an entity,
defining the intent, adding an ability to buy other items
through a WebHook, and then we're
going to connect it to a messaging platform.
OK, so let's switch to the demo now.
OK, can everyone see my screen?
How's the font size?
Is it OK?
Yes.
People are saying yes.
OK, great.
OK, so let's start by defining an entity.
And we'll list all of the products
that we want to sell in this store.
So let's say we want to sell a Pixel.
And sometimes that is called Google Pixel.
And it could also be in plural, so let's say
Pixels and Google Pixels.
And let's say you also want to sell a home, which could also
be Google Home.
And let's add the plural.
And let's add a Chromecast to that.
Chromecasts.
And what else?
Let's add a Chromebook.
OK, and that's enough.
Let's save our entity.
And now let's define our first intent.
So let's make the first intent about service.
And let's just add examples of how
users would ask for service.
So maybe one guy will say, I'd like to fix my Pixel.
And you can see that API.AI automatically
identified that Pixel is one of the products
that I identified earlier and it labeled it as a parameter
name called Products.
I'm just going to change it to Product, because you just
want one product.
I'm going to save this.
And let's add a few more examples.
Let's say, can I please buy a Chromecast?
I'd like to get a Chrome.
Can I please-- oh, sorry.
I'm mixing commerce and service.
Let me redo this.
I'd like to fix my Pixel.
Can I please fix my Chromecast?
My Chromebook is not working.
And let's start with that.
And let's test it now.
So I'm going to take this example,
but use a different product.
So instead of I'd like to fix my Pixel,
let's try I'd like to fix my Chromebook.
Oh, OK, so that's great.
So what you see is it identified an intent that is service.
And it identified a parameter name Product
that is Chromebook, which is exactly what we wanted.
Now let's add a response.
Let's say, sure, no problem.
Dispatching service for product right away.
Save.
And let's test it.
Can I please fix my Chromecast?
And you can see, it says, sure, no problem.
Dispatching service from Chromecast right away.
And if I click on Show JSON, you can
see there's this very rich JSON that you
can send to your back end, to your application that
can actually process everything that you need to do to actually
dispatch someone out there.
It's all structured information that you can act on right away.
So this is great.
And we have something that is doing service right now.
Let's also add a Commerce intent.
And now let's say, I'd like to buy three Chromecasts, please.
OK, so it identifies quantity and product.
So let's name this one as Quantity.
And instead of Products, let's just call it Product.
And let's show a few more examples.
So can I please get five Pixels?
OK, so this one, it didn't auto recognize,
but we can easily fix that.
I'm going to remove this.
And instead, I'm going to mark the 5 as Quantity
and pixels as Product.
And let's give it a couple more examples.
Can I buy seven Chromebooks?
Please get me two Pixels.
OK, so it looks like it's working.
Let's also add a shipping address.
So let's say, can you please send me four cool homes
to 5 Fifth Street, New York, New York?
OK, so it recognized that we have an address here.
It marked it.
Let's just show it the full thing.
And then let's just mark quantity.
And I'm going to save this.
And let's add a response.
Adding quantity, product, and sending to address.
I think this one we don't need.
Let's delete it.
And let's save.
And let's test it out.
Let's say, can I buy five Pixels?
OK, let me try, I'd like to buy five Pixels, please.
OK, you can see the Intent, it recognizes commerce.
It got that it's a Pixel.
It got that it's five.
And the address is missing, obviously,
because I didn't give an address.
And so it can't really respond, because it
doesn't have an address.
But everything is working well.
Now, the previous sentence I gave
should have been recognized, but it wasn't recognized.
So this is where API.AI really shines.
You can actually go in real time and apply new training
based on the data, the experiments you just made.
So this is what we tried earlier.
Service for Chromebook.
That was correct.
So I can mark it as correct.
And please fix my Chromecast.
That also worked well earlier.
And this one did not work.
Can I buy five Pixels?
Oh, there was a spelling mistake here.
Maybe that's why it didn't work.
But I can go back and actually assign this training data
to what should have happened.
And I can say, this should have been 5 quantity, not
unit length quantity.
And Pixel should have been a product.
I can mark it with a checkmark.
And then from now on, it's going to add that to the examples
it knows how to recognize.
And then this one should be Commerce.
Quantity.
But I want to track this.
And Pixels is the product.
And checked.
I'm going to approve it.
And now let's try another one.
Let's try, can I buy nine Chromecasts?
Great.
Recognize commerce.
Recognizes Chromecast.
Recognizes nine.
And now we see that it's the same problem again,
that it doesn't have an address, so it can't answer.
So this is something really cool about API.AI.
You can very easily turn this into a conversation
where if I mark these three parameters as required,
I can now talk to the user and ask
it to fill the required fields.
So if it's missing a quantity, I can say,
how many would you like to buy?
And then for product, I can ask, which product are you
interested in?
Notice that there's room for multiple prompts.
This is because we found that sometimes you
want to ask the same question with multiple variations just
to make it more interesting for users.
So let's also ask, where would you like me to ship it to?
And let's save this.
And then let's try this again.
Can I buy seven Chromecasts?
OK, and so now you can see it's responding.
It says, where would you like me to ship it to?
And now I can tell it, 1 First Street, San Francisco,
California.
And then it responds, adding seven Chromecasts
and sending it to 1 First Street,
San Francisco, California.
OK, this is all good and pretty awesome.
Let's try one last one.
And then make this a little bit more complex.
Let's try one that's missing quantity.
So I'd like to buy a few Pixels.
Now, I never actually trained it on a sentence
that looks like that.
If you look at the other ones, there's
no place where I say a few.
But it already understands that I want to do commerce.
Actually, it didn't get the product this time.
We'll fix that in a second.
But I guess it's because of the few.
How many would you like to buy?
Let's give it eight Pixel And where would you
like me to ship it to?
7 Seventh Street, San Francisco, California.
And now you can see it finishes the full thing.
So we can go back on training and just tell it
that if you have something with few,
that still is a legitimate ask.
OK, so now let's make it a little bit more interesting.
Now, let's say that you want to be
able to feed this into your back end application
into some sort of a shopping cart
and be able to add sort of multiple items
from the same kind.
So what I want to do here is to have some sort of counter,
where I send this counter to my back end, my back end
increments by one, and then sends it back to API.AI
with an answer.
So the way I'm going to do this is I'm
going to create what we call a context.
So I'm going to create a context called Shopping Cart.
And now every time someone does a commerce operation,
it's going to save it in the shopping cart.
And I'm going to create another intent that requires
Shopping Cart as an input.
So this will now only happen if someone has a shopping cart.
And let's say someone says, please add one more
or I'd like to buy another one or another one, please.
Let's call this Buyanother.
Buyanother.
And now let's add from Shopping Cart, the product.
And let's add the quantity and let's add the address.
So number and address and shopping cart quantity
and shopping cart address.
OK, let's save.
Now, what I'm going to do is I'm going
to connect it to my WebHook.
And if it doesn't work, I'm just going
to put error, so that we know that it didn't actually
go to the WebHook.
And I just want to show you guys how that looks like.
So you can see, this is a WebHook
I created in Cloud Functions earlier.
That basically increments one to a context
that comes from API.AI.
I can show you guys the actual code.
It's basically four or five lines of code.
That's all.
And no JS, by the way, if someone's interested.
So let's try it out.
Let's see if it works.
So let's start by, I'd like to buy a Chromebook, please.
Let's make it two Chromebooks, please.
Where would you like me to ship it to?
Let's say, 7 Seventh Street, San Francisco.
So it's great.
It responds that it's adding it to my shopping cart.
I get a context of a shopping cart.
And now let's ask it for another one.
And so you can see the response.
Adding one more Chromebook, you now
have three in your shopping cart.
And remember the context of a conversation
that you have a Chromebook.
It's sent it to our back end, the back end incremented
by one, sent it back, and now you're
seeing the response at three.
If I try and trigger this again, please add one more,
you'll see it says you now have four in your shopping
cart of the Chromebooks.
So what you see here is I didn't measure time,
but I think it's something like 10,
15 minutes, we basically built this whole conversation
application.
It does commerce.
It does service.
It'll get better the more we add more examples to it,
the more it gets usage data, and the more that we train it.
But it can already handle a lot of queries pretty easily.
Now, what I would like to do is I'm going to connect it here
to Facebook Messenger.
And you can see we have one-click integrations.
Like, very easy to start and stop.
Let's just start it.
And I'm going to go to Facebook Messenger.
And now let's say, I'd like to buy a Pixel, please.
Obviously, it's not working.
OK, perfect.
How many would you like to buy?
[APPLAUSE]
Go ahead and tell it six.
Thank you.
Let's give it an address.
6 6th Street, San Francisco, California.
And now let's try the last thing,
another one where it goes directly to our WebHook.
OK, now you can say it says seven.
So that's it, guys.
[APPLAUSE]
Thank you very much.
We got through this together.
You can see just how easy it is to create a conversation
application that's really powerful.
And with ease, you can connect it automatically
to all of those cross-platform applications you saw earlier.
You can put it on your own website,
in your own mobile app, or work through it
with the different messengers.
Very, very exciting.
So there'll be Q&A later.
But now I'd like to invite to the stage Cornelius that
is going to show us a little bit more
about what they've done at Bosch.
CORNELIUS MUNZ: Thanks.
[APPLAUSE]
So thanks, Dan.
My name is Cornelius.
I'm working at Bosch Thermotechnology.
We are Europe's leading OEM for heating appliances, boilers,
appliances who prepare hot water for you.
So can we switch to the slide deck?
Slide disk.
OK, I go on.
OK, thanks.
OK, and our devices become more and more
smarter the last years.
So we have now out of the box IT connectivity
are built in in our devices.
Are if you sell a Bosch appliance, a Bosch [INAUDIBLE],
you will get out of the box AIPI support.
You can connect your app to it and remote control it.
So that's fine.
But we have a problem.
We lose the contact more and more to the customer.
So how get the interface back to the customer
if the device becomes more and more smarter?
And our approach was that we tried
to figure out how our conversational bots can
be used to keep in contact to our devices.
And how does it work?
So we have had the answer from a software architecture
perspective, many for layers.
On the left side, you can see the interaction
of the user integrated in Skype, Slack, Facebook Messenger,
whatever, your own home page.
And the user can send the message to the API.AI chatbot.
That chatbot processes the intent, as shown by Dan before,
and the intent detection fires then a HTTP call
to the WebHook.
And the WebHook is now a piece of software we have developed,
which maps the API.AI intent to a domain-specific API call,
so which will call to our boiler.
And the boiler can, in that case,
set a new room temperature setpoint to a new value,
respond with a HTTP, OK.
And then we can prepare a fulfillment message
in our WebHook to respond to the message of the user.
API.AI then routes that message back
to the originated messaging application.
And then it's done.
The boiler has set a new temperature.
And hopefully, the user [INAUDIBLE].
So let's have a look on the demo.
So back to the demo.
So here I have a demo set up on the left side.
You can see our internet portal, which is already available.
So you can control your heating appliance.
Can set some values, for example, the current room
temperature setpoint, the hot water setpoint
for your shower and bath.
And you can, for example, prepare a holiday mode.
If you are leaving the home, the device is going to stand by.
And you can save a lot of money if the heating
device is switched off.
On the right side, I have connected here my iPhone.
And Skype is opened.
And you see, like in the demo before with the Facebook
Messenger, you have a bot us a chat contact.
And then you can say, OK, for example I will be on holiday.
And I have used Siri to transfer speech to a chat message.
And then a required parameter is missing.
To call the API of the boiler, I need the date period.
And the bot asks me for that.
Next week.
I will send this next week.
And the magic in API.AI transfers from then next week
in a date period, which could be processed by our API.
And next week, hopefully, yeah.
it's transformed in a date.
And if our portal works fine, that we have to refresh it.
Holiday mode is switched on with the desired date period.
And we are done.
So use conversational--
[APPLAUSE]
--UX-- thanks-- to keep in contact with your device.
And that's a good possibility for us.
So let's dig a little bit more into development issues.
How do we have done this?
It's the same setup we have seen by [INAUDIBLE] before.
We have different use cases, so we
can activate the holiday mode.
You can set or get the auto temperature.
You can ask the boiler, if you want to take a shower,
if the hot water is prepared, and so on.
And you can set your schedule of the boiler with buttons.
And yeah, that's fine.
But let's stick to the activate holiday mode.
I have prepared some [INAUDIBLE].
And yeah, then I show this already.
And I want to require a parameter.
It's the date period.
And then I have switched on the WebHook for the [INAUDIBLE]
presentation.
And I want to use it here.
And now I take the JSON here, which
is processed, and copy it out.
And then keep this in mind.
And now I switch to our back end implementation.
Oh, my VM is switched off.
So let's be patient till it's up and running.
So we are developing our back ends .NET.
And we're using Visual Studio for that.
And it was really easy for us to use API.AI.
Oh, sorry for that.
Yeah, let's keep on talking.
Corporate networks are not a good thing for developers.
OK, how to integrate within a .NET solution.
I have a Web API project up and running.
And the first thing I have to do is I use the NuGet Package
Manager.
And I add the NuGet package from API.AI.
So they provide many integration SDKs for node, for JavaScript,
for everything.
So that's the first thing.
I added that NuGet package.
Then I prepare my controller.
Here you have the root prefix of that controller,
which receives them.
The POST message here, it's the POST annotation.
And you get a JSON body out of that POST message.
And here, it's the model from API SDK, which is
provided by the NuGet package.
And that model is then in a form,
which you can process easily.
And here I have added some if/else switches.
You can also use a switch statement.
Doesn't matter.
And how to test it now locally on your machine
before deploying it in the cloud and connecting it directly
to the chatbot.
So I use here now the debug mode locally on my laptop.
And use Postman or you can use any other REST client.
Paste the JSON in you have formerly copied out
of the API.AI console.
And then fire that POST to the local host.
You can see it here.
It's now my local WebHook.
And [INAUDIBLE] back to Visual Studio.
Hopefully, yeah, it holds on the line where the activate holiday
mode intent is processed.
And the action name was called activate holiday mode.
And now I have to read out the parameter, so the date period.
And you have your [INAUDIBLE] structure.
It's well-formed.
And you have the parameters.
And you see, OK, yeah it's very small.
But you have one parameter.
It's named date period.
And it has that value.
And you can fetch out that key value
pair out of that structure.
And then I call--
I have a API Manager class, which
separates all our domain-specific API away
from the controller.
Then I call the set holiday mode,
activate it, and activate the date period.
So that's the domain-specific API call.
And then I prepare the fulfillment.
And the fulfillment is the response
of the chatbot to the user.
And here I do it mostly like then before in the console.
I replace a placeholder key with the date period
the user was expected.
And then I return OK.
So HTTP 200.
And if I go back to the Postman, then you
can see here the fulfillment, which is also JSON,
and which is sent back later on if you deploy it to the cloud
through the API.AI chatbot.
And that's, overall, the thing we have prepared.
We try to launch this in the next two months
with the MBP approach.
So test it with a few users.
We are record what the users say to the bot
to learn how we should improve our utterances, which
fires the intent.
And then we are, yeah, excited to see
the response of the users.
And we start to deploying it not to Facebook and to Skype.
We start to adding that chatbot to our internet portal,
so we have an interface, HTTP interface through API.AI.
And then you can use it, also, in your web portal.
So API.AI was really seamless.
It takes two or three days to set it up
and running with our real boilers.
I have to prepare that.
I switched off the holiday mode later on.
Because my wife says, if you come back and it's cold,
you're fired.
So we have out of the box language support.
And that was really a good language support,
even for European languages.
So we can use German as a language, and Italian, French,
out of the box with API.AI.
Other solutions, that's not fulfill that requirement.
And here we do not have to translate
the speech with Google Translate before sending it to API.AI.
We can directly use German or Italian or French
as a language.
So thanks.
I would like to welcome Petr from ING to show the next demo.
[APPLAUSE]
PETR NETUSIL: Thanks, Cornelius.
All right.
Excellent.
So my name is Petr Netusil.
I work in applied machine learning department at ING.
Let me figure out how this works.
Works good.
So there we get to experiment with all sorts of fun
new stuff, like AI machine learning NLP.
And I'm excited to show you today
what we have been building with API.AI.
And hopefully, give you some tips
on how you can get started yourself
and something on top of what has been shown here.
But well, first, ING, big global bank.
But actually, my boss has recently
realized we're not a bank anymore.
We are an IT company in a financial industry.
And as an engineer, I find it excellent.
I find that transition great.
And good thing they also didn't just said it like that,
but they invested almost a billion dollars,
if I calculate-ish correctly, into digital transformation
to make that happen and to make sure
that we are on the platform as our clients are as well.
But anyway, let's go to the deep, gory details
of the solution.
So this is our architecture.
And maybe good to know, this is also our bot.
It's Marie, our digital assistant.
So she can help you with everything about the card.
So that is the kind of domain we have selected for a pilot.
And what you see perhaps immediately
is that the architecture's different.
So normally, the traditional architectures
would talk to API.AI directly and then do some WebHook
fulfillments out of that.
But we said, no, no, no, we want control, we want security,
we want logging, we want to do it better.
So we actually place our conversation logic
in the middle.
So I'll fly you through how this works.
So you start from any channel, Facebook Messenger, Skype,
or some other channel.
And then you go into the conversation logic
that will basically route with all these different APIs.
So if it's just API.AI discussion, you talk to API.AI.
So everything, again, happens via some REST calls, HTTPS
[INAUDIBLE] requests, like you have seen before,
or you can route the sentiment analysis, which is something
I'm going to show you.
And we are very excited about too.
So that's based on the Google Natural Language API.
We also built a sentiment analysis and intervention
dashboard, which is based on WebSocket.
So you have a real time kind of connection between the server
and the client.
We have also Twilio for authentication.
And lastly, and for us at least more importantly,
we hooked it up all to our ING APIs
and we will get live data from this moving forward.
So if my corporate laptop will work
and we can very kindly switch to it.
[LAPTOP BELL]
We have even the sound.
Very good.
[INAUDIBLE] number.
Oh, we're perfect.
So Marie, our bot, well, let's just start with her.
So what she will ask you immediately is,
you can get started about questions and requests
about ING cards.
Now, typically what we see in Belgium a lot is
that a lot of people are just calling us, like, hey,
I don't know where is my card?
And we get lots of these calls.
So they would ask, hey, I ordered it last week.
And now it's not it's not there anymore.
And did you send it?
Did you not send it?
So one incidence we have programmed
is, where is my card?
And Marie, what she'll try to do now is to authenticate me.
Because we don't want just everyone to know these details.
So our demo profile Robert has born on 1st of May 1967.
Now, what happens is we're going to match the user uniquely
based on the first name, surname, and the birth date
and get a one-time verification token.
Whew, yes, we got it.
The demo gods are with us today.
And I can put something grand on [INAUDIBLE] it won't work.
But let's be nice to Marie and put the right one.
So it's 129335.
Right.
So what happens now, we get authenticated
for the next 10 minutes.
So you don't need to push these tokens back and forth
all the time.
And we get the right delivery address details.
So this is one of the use cases I wanted to show you.
But the thing is we are extremely
excited about bots ourselves, but maybe not everyone is.
They are still young and they're still learning.
And what will theoretical happen or more practically
is that someone dislikes it.
So saying, I know this bot is terrible.
My god!
[HORN BLOWING FAILURE]
What is that?
Oh, so it's our sentiment analysis dashboards
are triggering that.
Thank you.
[APPLAUSE]
If you have ideas for a better sound,
please talk to me after the session.
Right.
So this is basically the web circuit
I was talking to you about.
So there is this server to client integration,
which triggered that.
So let me deal with it.
So here you see transcripts or, well, a dashboard
of all the different sessions today or now is just the one.
And you see the different sentiment, which is happening.
But what is interesting here is that I can actually
go inside that full chat transcript.
I see everything here.
And I can boss the bot.
So what I'm going to do, I'm going
to stop Marie and just start the conversation as
if I would be someone controlling and managing
that online conversation.
So hi, this is someone alive.
I want to get out.
You know, it's going to be a little bit silly now,
because I'm going to have that conversation with myself.
But the point is API.AI is responding now.
And I have a full control of what is happening, OK?
So you see that point.
Now, to satisfy the development and coding people
amongst yourselves, let's actually
see how ridiculously simply you can implement this yourself.
So our conversation logic is built on node.js.
And this is how you can require the Google NPM package.
So hopefully, you will use this commented line and not
this hacked way, which we used over here.
And then you just call a method that is the detection method.
And you wait for a callback.
And you get a callback back after some time.
And then you do whatever you want with that sentiment value.
So it's extremely simple.
And I think the guys at Google done a great job with that one.
Right.
Let's go to the API.AI and how we implemented this.
So what I wanted to show you and how
we used to trigger the conversational logic
is like this.
So say, where is my card or something else.
The JSON line you are interested in is this one.
So there is an action.
And just using that action, you can program anything manually
for fulfillments for further on.
But what is maybe more interesting here,
and I'm going to take you to our development Marie,
is that once your bots are going to be scaled further
and you are going to build a lot of different intents
and conversations, it will get messy.
It will be back and forth.
And people will not just follow the preprogrammed line.
So you want to be able to still navigate
within that conversation.
Now, one of the ways we found, at least for ourselves,
how you can do that in API.AI is to create, first,
some kind of a naming convention,
because otherwise it's a big mess.
And understand that for one use case,
there is actually many intents which you can use.
So say there is a use case activate card,
but one kind of a use case is more intents
which you want to trigger.
So we're going to start the first intent
and then pass this context up until the last one.
So let's see how it looks.
So you see that the first intent has no input context, but just
the output context.
And then you want to pass those along the conversation.
Here are the [INAUDIBLE].
Typically, you would put hundreds of them, which we
have in our to-be-live version.
And then what is also interesting
here is managing the conversation.
So here we put the response without a parameter and here
with a parameter.
And only if you have the parameter, as Dan showed you
before, it will be triggered.
All right.
So I hope you guys liked our little demo.
I'm very happy to connect with you during the conference.
And yeah, thank you for watching.
[APPLAUSE]
So we're going to have a bit of a Q&A.
So I'd like to call Cornelius and Dan back to the stage.
And, well, fire away.
And the nice guys who were there asked us
if you can come back to the mic, so that everyone can hear you.
DAN AHARON: We can probably switch back to the slides,
I guess.
PETR NETUSIL: Yeah.
Questions.
DAN AHARON: If anyone has a question, please, yeah,
come up to the mic.
PETR NETUSIL: Yeah, or just shout very loud.
AUDIENCE: Hi.
Thanks for the great demo.
One question is, what do you do about analytics?
You say the API.AI is a complete system.
I didn't see any analytics link.
DAN AHARON: Yeah, that's a great question.
So we don't have any analytics built into API.AI right now.
There's a bunch of other vendors that we work
with that provide analytics.
If you connect with me offline, I'm
happy to recommend a few to you.
PETR NETUSIL: Well, the way we approach it, we lock everything
and we have our data scientists get crazy on it.
So maybe that's a way how you can do that too.
AUDIENCE: And sorry, question for you.
You saw you interrupted the Facebook conversation
and had a manual intercept.
How did you do that?
PETR NETUSIL: Manual intercept of what?
AUDIENCE: The conversation.
You sort of had a manual intercept
into the bot conversation.
How did that work?
PETR NETUSIL: So it's in node.js.
But I've said that there is a global parameter for all
the sessions you can set up, whether it's POST or not,
and then you manage manage it based on that.
So I can show you later on, if you want, in detail.
AUDIENCE: OK, thank you.
Thanks.
PETR NETUSIL: Anyone else?
CORNELIUS MUNZ: Hey, come on up.
PETR NETUSIL: There is actually a line.
That's great.
AUDIENCE: I have a couple of questions.
The first one is regardless [INAUDIBLE]
figure out the intent.
Can I have two intents [INAUDIBLE]
technically take the same number of entities?
And how does it figure it out?
Let's say that I have one intent that
is supposed to act on a negative set of words and the other one
acts on positive.
DAN AHARON: Yeah.
So it learns by language.
So if you set up the intent with examples of negative sentences
and a set of entities, and then a different intent that
has positive sentences and even the same entities,
API.AI knows how to figure out which intent to choose
according to your examples.
AUDIENCE: And can you load the entities from, let's say,
a transactional table that you are continuously
updating all day long?
DAN AHARON: Yes.
Yes, you can import or export from the UI.
And also, there's a full back end API
that you can access everything programmatically,
which we didn't show today.
But it's all documented on the website.
AUDIENCE: And does that require retraining every time there's
a new entity on that table?
DAN AHARON: Yes, but the training happens very quickly.
In fact, the model was trained multiple times during my demo.
It happened in the background and you didn't even notice.
AUDIENCE: OK, thank you.
DAN AHARON: Sure.
AUDIENCE: Awesome demo.
Thanks, guys.
PETR NETUSIL: Some [INAUDIBLE] here.
AUDIENCE: I had a question on the difference between the toy
applications and when you went to production.
It seems like you'd have common [INAUDIBLE] explosion
in Natural Language.
And how did you handle that?
You said you had hundreds of the templates up there.
PETR NETUSIL: So we are not live yet.
We will be live in a few months.
But the way we have approached it is we
have lots of user sessions.
And we basically had people coming out
and say, just tell me, how would you ask this question?
And we have, yeah, I think we have hundreds
and hundreds of-- you know, just per one small intent,
hundreds and hundreds of utterances
programmed over there.
Try to get a trend.
I think it's easy and fun to do a bot,
but it's difficult to do it right.
So this is one of the parts where, yeah, you
need to do it right.
DAN AHARON: If you're interested,
you should check out the third session, which
is about actions on Google and talking about how to build
conversational actions.
They're going to talk a lot about lessons
of how to build a right bot and right conversational interfaces
that think about all those different expressions.
One thing we see a lot of people do
is use cards and buttons, which sort of help
simplify the experience for some users.
But make that optional.
So give the users the option of whether they
want to use the cards or whether they want
to type in Natural Language.
AUDIENCE: Would the semantic analysis help in that regard?
DAN AHARON: Yeah, yeah, some users love to type.
And they want to express themselves that way,
while others just want the fastest way of getting
things done, right?
So you give them both.
And then they pick whatever they want.
AUDIENCE: Thank you.
DAN AHARON: Sure.
AUDIENCE: Hi.
I'm not sure I know how to ask this question right.
But every demo that I've ever seen on conversation
relies always one-on-one.
What about the bot being kind of a moderator of a conversation
between multiple people?
DAN AHARON: Good question.
So the way that it's structured is
it's built on a conversation.
And it has, as context, all of the prior conversation
that happened.
So if you have a conversation with multiple people and one
bot, then as long as your intention is
for it to remember what everyone else is saying,
then you're good to go.
Then it doesn't change anything in the logic.
AUDIENCE: Yeah, like if you were trying to figure out, hey,
we're going out for dinner and here are five different places
we want to go--
DAN AHARON: Yeah.
AUDIENCE: --that sort of thing.
DAN AHARON: Right, like Google Assistant--
AUDIENCE: Yeah.
DAN AHARON: --and Google [INAUDIBLE].
Yep.
AUDIENCE: Yeah, yeah.
All right, thank you.
DAN AHARON: Sure.
AUDIENCE: Hello.
Great talk.
Great demo.
DAN AHARON: Thanks.
AUDIENCE: I liked that question too.
I guess, two things.
One, are there push notifications with this yet?
Could you set something that eventually the chat will just
chime in when a counter goes off or you set a reminder?
DAN AHARON: So I don't think so.
But let me get back to you.
Are we back to the slides?
OK.
Send me an email.
And I'll get back to you.
PETR NETUSIL: The way you can solve this is actually
to trigger it--
at least in our case, to trigger it
from your conversational logic.
So that's why we put our thing in the middle to say,
hey, psshht, let's fire up a push notification.
AUDIENCE: So you would say like, show
me the status or something, and then you would get back?
Is that what you're saying?
PETR NETUSIL: It's from an application side, not from API
directly.
AUDIENCE: Oh, I got you.
Yeah.
And a quick question about deployment.
So you had a dev environment and production,
would you just edit the interface
or do you version control that?
PETR NETUSIL: Good question.
DAN AHARON: Cornelius?
PETR NETUSIL: Oh, Cornelius [INAUDIBLE].
CORNELIUS MUNZ: I didn't get it right.
So can you repeat it?
AUDIENCE: Oh, so you had a dev environment and prod,
do you version control the configurations?
How would you deploy your dev to prod?
CORNELIUS MUNZ: Yeah, we have version control or we use Git.
We have different environments, staging, testing, and stuff.
And yeah, it's a continuous integration environment.
AUDIENCE: OK, cool.
PETR NETUSIL: So I think there's version control many levels.
On the code, we use Git, like any other person in the world.
But for API.AI directly, we just created different bots.
And what you can do is just extract one bot.
Just export everything there is and just
import it in another bot.
CORNELIUS MUNZ: So it can export as [INAUDIBLE] file.
PETR NETUSIL: It's export/import.
CORNELIUS MUNZ: So all--
PETR NETUSIL: [INAUDIBLE]
CORNELIUS MUNZ: --yeah, configuration
of the bot itself, not your code, so I didn't get it.
So the configuration of the bot can be exported as a zip file.
And you can store it safe within--
and you can import it later on with intents, entities,
contexts all available.
AUDIENCE: Oh, perfect.
CORNELIUS MUNZ: And at the end, it's a JSON file.
AUDIENCE: Oh, cool.
All right, thank you.
AUDIENCE: [INAUDIBLE]
PETR NETUSIL: Manual.
Yeah.
CORNELIUS MUNZ: Manual.
AUDIENCE: So one of your early slides
showed Cortana and Alexa.
DAN AHARON: Right.
AUDIENCE: Is it trivial for me to create
a bot that then interacts with Google Home, Alexa, Cortana,
a trivial connection?
DAN AHARON: Yeah, so their interfaces
are slightly different, right?
And so the data that they send to other applications and back,
the format is a little different.
But we've built our connectors to try and minimize
that sort of interruption as much as possible,
so that our developers can focus on the actual conversational
logic.
So for the most part, you can build one logic
and connect it to multiple connectors.
AUDIENCE: It'll connect
DAN AHARON: Yeah.
Now, the caveat is when you're talking to Google Home
by voice versus if you're texting
on Google Allo or Facebook Messenger,
the users speak sometimes in a different way,
use different words.
And the way you want to talk back to them is also different.
Like on a chat, you might want to show them cards with a map
or maybe you want to write a long paragraph of text.
When you're reading back on Google Home,
you don't want to read back a long paragraph of text,
because users will get bored, right?
So you actually may want to build
different conversational experiences for each one.
But if you wanted to create a single experience, you could.
CORNELIUS MUNZ: And from a technical point of view,
you have to make a trust relation between Facebook
and API.AI.
And you would generate an API key on Facebook or Skype
or whatever and copy that API key into the API.AI console.
And that's all so a trust relation is there.
And then Skype can talk to API.AI
and respond to that message.
AUDIENCE: Thank you.
Thank you, both.
DAN AHARON: OK, two minutes left.
So maybe one or two questions.
AUDIENCE: Good afternoon.
I was trying to build a toy project the other day.
You each use your building address, street address entity.
I noticed that in our test data, there
is certain address that's a rather unusual construct.
Never get recognized.
So add those to the training data.
But it just doesn't seem correct that.
So what we recommend developer to do for a situation like this
is when the black box fails.
DAN AHARON: Yeah, so it might be a bug.
So I would go--
we have a community forum.
I would report it in the community forum.
And we can look into it and see if it's a bug.
AUDIENCE: Thank you.
DAN AHARON: Sure.
AUDIENCE: Yeah, so my first question
is, is there a way to resume a conversation
or it's all within one conversation?
DAN AHARON: So that was on yours, right?
PETR NETUSIL: Yeah, it could be.
So if you mean the bot boss, you can just
flip it back and forth.
So as an agent, you have full control.
You can just start it, stop it whenever you like.
And secondly, every session we have with a bot is in memory.
So we remember everything you did, you said,
and all the context you have had.
So you resume anytime you'd like.
AUDIENCE: OK, cool.
And then more about the development life cycle.
You said we can export or import the configuration.
What about, is the training data part of the configuration?
PETR NETUSIL: Yeah, all the utterances, like the hundreds
I was talking about, is that.
AUDIENCE: And OK, just to confirm,
the last question is, what level of certainty,
giving a certain training data will the response
will be always the same?
Or it's behind the scene we don't know?
DAN AHARON: So it's machine learning-based, right?
So the more training data you give it,
every time you give it more data,
it could change the potential answer.
But we do give a confidence score with the results,
so you can see sort of [INAUDIBLE].
CORNELIUS MUNZ: It's part of [INAUDIBLE].
And you can see the confidence level, it's 80%, 90%, or lower.
And you can decide by your own, if it is lower, to give a--
DAN AHARON: We're at time.
But thank you, everyone, so much for coming.
PETR NETUSIL: Thank you.
DAN AHARON: This has been great.
And have a great rest of Google Cloud Next.
Thanks, Petr, Cornelius.