Dan is a Developer Advocate for Dialogflow who has been building conversational interfaces since 2010. He's spent time as the CEO of an agricultural technology startup, a software engineer in Silicon Valley and the financial industry, and a lecturer in data capture technologies at Birmingham City University.View the profile
About the talk
This session will demonstrate how to take advantage of Dialogflow's powerful, easy-to-use features to build engaging conversational Actions for users on the Google Assistant.
All right. Hey everyone. My name is Dad and this is Matt and we both Google and see what kind of product called. I look tonight. Thank you so much for coming. We've been having amazing Iron and was super excited to see what's up with you a lot over the last few days about how important the Google assistant is so I can help people get things done. The assistant Works across dozens of device types and makes the most of the BET makes the most of the best speeches of each device that could mean the quick after glass display at a smartwatch and it could mean the
benefits of no screen at all devices. The assistant is really smart and we've made it to some super cool stuff. But the real power of platforms comes from that developers developers to extend the assistant and do things we never could have imagined. This is what I look like I'm saying we taking what we learn from building amazing conversational experiences that allows any developer to do the same thing about we going to tell you what does where I'm going to show you how to build a sophisticated action for the Google assistant in just a few minutes. We're going to talk about a few
upgrades be made based on feedback retired from the ocean community. So are you still in our development? Keynote we've had really great momentum and growth this year. We have half a million developers. You start looks like which is a 250% increase this time last year Google developers that obviously really excited to use IML second Network. We will stick up Brands all around the world using. Looks like and a building some really amazing stuff. So if you want to see some case studies, you can check our website to see some of that pot does experiences using the product. All right, let's
get technical. So before we go, so, let's go over some basic action is an experience. I never many ways a user can invoke your boxer. First they can just ask the assistant to your action by name might suggest you action based on this functionality. Once in vote your action is going to haunt. The assistant is going to hand conversation control over $2 today and it will hold a conversation with your uses based on the structure that you define. You can connect this
conversation with your business to make things happen and generate replies. City following this conversation you'll use Dollar close you eye and it's a web app that even known developers can use this no coding required to simple apps and it takes at least 30 minutes the first to tell you the conversation under the hood thistle uses Google's machine learning and natural language understanding Souls, but you won't need a PhD to make use of them. The most complex action connecting actions with your code is super easy and you can even use the embedded Cloud functions to
fly bass at if you just tell. Which points the conversation is should communicate with your code in HTTP request with Jason describing what do use of one and you can use this to construct a complex response or so look something up and generally make stuff happen. So we want to show you how easy it is to get started with David play the next minute. We're going to build an action from scratch for an imaginary local business a business is going to be a bike repair shop. Our actions going to work like a help. Leg is going to allow
customers to check the shorts opening hours and book an appointment to service that bike. So here's what's going to happen during the deadline first. We going to walk you through how to respond to use a query with a powerful Concepts tool the intent. When I'm going to show you how to extract detailed information from uses query like times and dates. I'm going to demonstrate how to connect your action to business laws if it will Store appointments in a Google Calendar. Finally, we're going to integrate our action with the Google assistant and test it out.
Since you get started, I'm going to hand over to Matt and he's going to demonstrate how the finish action is going to work on. His shiny new pixelbook. Thanks, Dan. Can we go? There we go. So before we get started building, I'm just going to give you guys a quick demo of what we're going to build. So right now on the left side of the screen. We have the Google Assistant up on the pixelbook and will say talk to my bike shop today invocate are action. The right now the Google assistant is hanging off the conversation from the user from the Google Assistant to our action
the bike shop. So right now we're going to set up an appointment. And it's going to ask us what day we want to come in. So we're going to say next. Friday and I know that's what time it will say 3 p.m. And right now we're the cloud function is actually adding an event to our calendar. You can see there and asking us what kind of appointment so I'm going to say and then we can see the our action is confirmed the appointment and is end of the conversation. So now let's build it.
So this is dialog close console on the left. You'll see a panel where you have all the main Concepts that you need to use to build a talent agent. And in the middle, we have a panel where you do use all these main Concepts right now with the list of intense and on the right. We have a simulator where you can try out your dialogflow agent as you're building it. So let's go ahead and try it out will say hello. So we can see that the default welcome intent here was was matched and if we look open up for the default welcome and sent you can see that we have the response high. So
that's not quite what we want to say since we have a bike shop. So let's change that response up and say welcome to my bike shop. How can I help? So we'll try it again. and we need and so now we're adding a training phrase here. So that are high query is understood and the agent is training and using this High example as a training phrase for a machine learning model that's running in the background that will identify incoming request to this welcoming Center. So I'll try high again and now we can see that our response is
correctly identified and we send back the correct response to create our own intent now, so we'll go to the navigation pin on the left where it says intense and click this little plus sign to create our own in time and will type in hours. So this will be our intent that we're going to create for any intention of the user has to know the hours that are bike store is open. So for the first thing we need to do is add training phrases. So these training phrases are used as the basis for the machine learning models. So we need to think of things users might save when they want to know the
hours of our bike shop. So you do a few examples here. How late can I come in? I might say what are your hours directly or they might stay when do you open? So go ahead and say that I'm in dollars level starts this training process using these training faces. And while that's happening. We're going to add a response. Around here. We click add response and then we'll say we're open. From 9 a.m. To 5 p.m. every weekday looks like save and we'll try it out after I craving this out.
All right, are you open now? So now we can see the hours intent was matched down here and sent back the correct response. We're open from 9 to 5 everyday. Are you open now isn't really similar to any of the training phrases we have went. When do you open? What are your hours? How late can I come in? So this is the power of valak low and really shows off how powerful it is. If you can imagine trying to write a regular expression to match every instance of these four examples as well as the thousands of other ways that someone
can ask when is your store open? This is where I dial a flow just becomes really powerful and allows you used to talk to you in a natural conversational way. So we'll take a closer. Look at the response. Is it a little bit generic? So we're open from 9 to 5 every weekday and we're not actually answer the question. Are you open now? And in a normal conversation with someone you expect to have more contextual information so they might say we're open until 5 today. So for that we need to do some 30 calculation as to what the time is and when our bike store it and if
that falls within our bike store open hours ranks, so for any sort of calculation or API request or a database query are we need to do something called film it? So if you go to the left panel here, we see you down here. We have fulfillment sister told me this how you connect your Dalek location to code. In this case, we have an inline editor where I've already deployed some code so we can scroll down here and look at the hours function. We can see we check if the store is currently open and if it is we give a response that says we're open now and we will be open until 5 and if it's not
we told him when we're open next, so there's one more thing we need to do to connect this code to a dollar Clayton and let's go back to our hours and 10 and scroll all the way down and go to fulfillment and enable fulfillment for this incense. I want to do that if we try agree again, or are you open now? We can see you right now that this so right now the hours and scent is calling the Web book and it comes back with the correct response of we're open now and we closed at 5 p.m. Today. So that calculation was made and sent back the proper response. All right.
Now let's try something a little more complex. We're going to create an intense for making an appointment. So we'll call it the make appointment intense and for this intense will need a few more examples cuz this is cuz there's many different ways that user might say I want to make an appointment. So we'll go through those right now. I'd like to get a bike or no. I'd like to schedule an appointment next Thursday. Can I schedule service for noon? Can I set up an appointment?
or my bike tomorrow At 2 p.m. And last one I'd like to come in at 9 a.m. On May 11th So now you can see we have a few things going on here that are different from our previous example. So highlighted a few of the entities that are present in our user says examples. So entity entity has built-in so we can extract data from what users are saying and use it for in this case to schedule a bike appointment. So it all the red highlighted items are times and all of the yellow highlighted
items are dates so we can see that these are listed as parameters. So we have the date and time in the value that we can use to prevent this in a response is so in this case. We want to reflect back the time in the use of the time and date user-specified so we can confirm the right appointment time. So we'll go ahead and add a response here. and say great I've set up your appointment for the dollar sign dates are this takes the value from what the user says and put it on a response and then at the same time, so this
will add the same thing for the time value. See you then. I'll go ahead and see if that and then we'll try this out. So I want pointment. tomorrow at 3 p.m. So now you can see that are we Dollar close correctly match the make appointment intent and is extracted out that tomorrow memes May 11th, and that 3 p.m. Means 1500 hours. So if this is also it's easily parsable on our Web book and it correctly inserts those values back into our response. So I actually make an appointment. We need a date and a time. So we need to make sure that the user provides
both those pieces of data. So we're going to go to the parameter table and make both of these parameters required. Now when the perimeter is required we can to find a prompt so that if a user doesn't provide a date or time we can ask them for it. So in this case for day will add a prompt that says what day do you want to come in? And for our time lot of frogs that says what time works for you? Sports try that out will say I need an appointment. So I now that we've matched the make appointment intent, you can see that the response instead of our response to find here. We hit one of our prompts
that says what day do you want to come in? So we'll go ahead and answer that question saying next Friday. And what time works for you with a 11 a.m. And now you can see that this has been filled incorrectly by dialogflow and we get the ultimate response back there. Also know that we've set up an appointment. We might want to get some additional information from the user about the appointment. So we might want to know what kind of appointment type so we'll go ask them a question and say I do you need a repair or just a tune-up?
So now we need a way to capture the answer to this question if they want a repair or a tune-up, so I don't look so already has these built-in interviews for date and time, but you can also create your own entity. So we'll go ahead and do that now on the left panel again, you can see we have a channel for Edge to use another plus button. So it will create a Misty and we'll call it appointment type. And then we'll add entries for our appointment type entity than this case will have a service option and a fix option. So will entering the first value service and then
we can also interested in entering sending values for each of these entities. So that was someone might not actually say the word service but they might say another word. That means the same thing so we can to find synonyms that will map back to this canonical value. So so instead of saying service, I want might stay over Hall. They might say maintenance. They might say tune-up or they might say tune up with a space in it other Valley. We have six or someone might say repair men. Am I say their bike is broken? They might say they have a
flat tire. or they might say will say that. So now that we have an appointment events view we need to create an intent to capture it and in our in the case of our intense, we only really want to capture this appointment type entity after our make appointment intent has been matched. We want to make a new invent that only is matched after the make appointment instead for that guy. Look for something called intense. So if we hover over the make appointment and sent you can see there's the ad follow up in 10 option so will create a custom
follow up and said so this new intent that's listed here with that arrow means that this content will only be matched after the make appointment at 10. So it's like that and we'll add some music as examples over this week. Just ask the user if they want a service or if they want to fix their bike. So we'll enter in some examples of how abusers my answer that question. It might say I need a dare. Or they might say can you service my bike? So now you can see us before with the built-in entities Dollar close correctly
identified our custom entity that we just created. Since we need to know this information, we're going to make it required and we'll add a prompt for it as well. We can service or repair your bike. Which one would you like? So now for a response here, it would be nice to confirm our actual appointment time and date after they've said what appointment type they have. But if you'll see here in the perimeter table, we actually don't have the time or day care. So how do we get this value for that? We're going to need to
dig in a little deeper into how to follow up intense work so follow up intense work through something called contacts. So once we added this follow-up intent to make appointment intense a contact out put contacts was added to make appointment there two types of a contact out put contacts and import contacts Outlook contacts attach a contact to a session after the intent has been matched. So in this case after our make appointment in ten has been matched the make appointment follow up contacts will be added to this session. Next let's take a look at our follow-up
intent for a follow-up intent. We have the same value make appointment follow up contact except instead of the output contacts. It's in the input contacts. So this makes the make appointment custom or our follow-up intent only get matched once do you make appointment follow up on text is attached to the session. So in our case, the only time that the make appointment follow contacts attached to the session is our parent make appointment in ten. So this means that this make appointment custom follow-up intent will only be matched after our make appointment parent intense
eye in addition to controlling the flow and how intense her match contacts also store parameter value so we can use the value of make appointment - follow-up to reference the time and date that we previously gathered make appointment in ten. So we'll go ahead and do that now. Will say okay well schedule a dollar sign appointment. So this is grabbing whether they're fixing or servicing their bike just like last time at 4, and now we want to insert the date. So will do pounds make appointment follow up and then due. Date
ads and then we'll do the same context value. Time. This will grab our time and date values from the previous events. You been? All right. Now instead of trying this out on dialogflow simulator on the right. Why don't we try and see what this looks like when you deploy to the Google Assistant? After that, we'll look again on the left panel and we'll see a tag a page called Integrations. So this Integrations page shows all the Integrations a dialog Full House with other chat and voice providers. So you can use any of these to deploy agency or users on a
platform for now. We're going to start out with the Google assistant and you can see some information about how dialogflow works with the Google Assistant, but for now, we're just going to try it out. So cooking test opens up the actions on Google console zaksons on Google console is where you define all the information about your action like invitation name. So for right now, it's weird. We say talk to my test. My test app is a special name. That's just for you when you're starting to test, but you can try you can enter in any name for your brand. Once you get into actions console
for now. We'll try with just talk to my test app. Okay, here's the test version of my test app. Hi, welcome to my bike shop. How can I help are we to find earlier? So let's try this out and I book an appointment for tomorrow at noon. What time works for you? Great. I set up your appointment for the 11th of May 2018 at 12 hours 0 minutes and 0 seconds. Do you need a repair or just a tune-up? So now you see you tomorrow at the make appointment intense and now we're going to try out our file open chat. So
we'll say I need a bear. Okay wheel scheudle a fix for the 11th of May 2018 at 12 hours 0 minutes and 0 seconds will see you then word and Matthew to the word sex store in a response that says okay, we'll schedule a text and then also grab the time and date from the context to surface back to the user as well. So I now that we've got an action for working. I'm going to pass it back to Dan is going to talk about some of the new features that we built and I looked over the last year and some more features. I didn't get to go over again. All right,
you should take about it. We send you how quickly you can build a really powerful assistant action with. Looks like in a few minutes, we built an engaging conversation. We connected it to all card and we've been able to deploy and test it immediately and the really crazy thing is that off to review this action will be available on over 500 million devices with no install required people. Just have to say the name of your action. Conversational experiences are a pretty new idea. I'm building your first agent can be hot because there's a whole set of Concepts that may be
unfamiliar for you refill. A lot of tools that will help make it easier to get started. Donald flood comes with 45 free Bill ages that you can use as a starting point when you were developing we cover a load of different real world scenario is like setting alarms booking tickets and also to stop all you need to do is import the intense identities and customize them to support your action. In fact every single do I look for Asian comes equipped with over the 50 system and disease these cover Cumberland Concepts including numbers. They
sometimes amounts with unit. Don looks like an extract values based on these concepts with no additional work. So you can understand your customers and help them get things done without a lot of Ed. We also have a growing library of sample Legends to demonstrate how to use off beaches and he's a reference to no adults and I'll get Hub. We got it a new feature where you can open the sample with one click and try it out instantly. So this will deploy the agent and the Fulfillment so you can just customize it and be up and running straight away.
Speaking of hot dogs. We've also built in interactive dialogflow ages. So you can play with real life examples that help you learn important sulfate. We're regularly publishing new videos into Saudi riyals to help you develop and we just published another getting started video, too. I look so YouTube. Once you find the basics, we provide tools to help you do really powerful stuff. So it will help you extend your actions over time. So you can improve your customer experience and performance. Free example natural conversation brunch in
complex and unpredictable ways things that they discussed earlier in a conversation can have an impact later on. I need the post conversational platforms requires you to hard-code everything which was really bristle and difficult to do anything to improve over time. Fortunately Dollar close contacts allows you to shave the flow of a conversation without having to predict every possible Twist and Turn ahead of time. This can help you get beyond the idea as a menu based phone conversation and create genuine conversation between people and computers.
Once you actions up and running you'll have a ton of logs of conversation and you can use this data to improve your actions machine learning model. I'll training feature will show you all of your past interaction and it helps you assign them is training phrases for intense. You can also create new intense when customers ask for something that you didn't handle. Every seat should we added guarded by these three goals. We want to create an excellent. Experience for all developers. We want the best possible natural language understanding and machine learning the easy to integrate in
Schuyler NE conversational by 4. We use developer feedback to decide house with over platform and I wanted to show you some of the features that we got. It has lost his eye. First we've heard that you want Bassett insights into how conversations are going. So we didn't prove the history. I made it much easier for developers to see how the agent is being used. Now. You can fill sit on dates and platform and you can search through the aux to find particular interactions to the nobody. This will help improve your teeth bugging experience and help you get to a best of
Agents as I know the time. We both have been working really hard to support all the languages that you're used to speak and today we supported languages and now we recently added languages including Hindi Thai and Indonesian and we cover all of the languages the assistant supports plus more. We've also added multilingual ages so you can build one agent that works for all of your users across the world. Many of our customers have been asking us for an Enterprise solution. And so this year we introduced to Enterprise Edition this brings Dollar close to
Google Cloud, which means you get enterprise-grade compliance and customer support on a cloud sons of cervix is soft felt that personal gaming assistance app on dialogflow Enterprise Edition and Sam was able to understand an ounce of 88% of players requests in the first three months of that paper. So I hope you feeling excited to get started building with dialog tonight. Therapy tons of really amazing assistant sessions at this year's II and you can find links to recordings of the mole on the IRS website so you can enjoy them that your Ledger you should also
make sure you check out a whole lobster who called you three building of first action. Your feedback is superimposed into our team and we really hope you enjoy this session. So we don't sell a lot for you to head over to the website and let us know your thoughts. It's super easy to get started with David. But when we put together a page for live links that can help you get off and running and links on that to our online communities where a whole theme is hanging out and we can help answer your question. Also, don't be shy about spacing on energy does hashtag if you want to show you I have
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.