Thushan is the product manager for all things fun on the Google Assistant. He always wanted a talking robot of his own – now he gets to play with one of the biggest. Previously he created apps like Toontastic (which the New York Times called one of the best apps of the year, and included in Apple's App Store Hall of Fame) for kids at his startup, Launchpad Toys. Before that he was a multimedia journalist at The Boston Globe. He has a B.S. in computer science from N.C. State University.View the profile
Ralfi Nahmias product manager, Actions on Google Ralfi is a product manager focusing on "chatbots". He is responsible for DialogFlow, Google's bot building platform, and APIs for developing AoG's gaming ecosystem. Previously he worked on natural language question and answer systems over large data sets featuring on products like Google Analytics and Google Spreadsheets.View the profile
Andy is a Creative Lead at Google, where he infuses delight and play into the Google Assistant as part of the Personality team. Prior to joining Google, Andy worked as a creative director, designing award-winning digital products and experiences for brands like Steelcase, Cartoon Network, Crayola, Nickelodeon and the Smithsonian Institution. Andy earned a BFA in graphic design and a AAS in illustration from the Rochester Institute of Technology in Rochester, NY.View the profile
About the talk
Learn how to design and develop fun games for the Google Assistant on smart displays. By walking through the new APIs, you'll learn how to best take advantage of the capabilities of smart displays for immersive game play.
Good morning, everyone. Thanks. Thank you for coming so bright and early on this to 8:30. And and thank you for those who are watching a lot. I we are so excited to have you here because in the next 40 minutes, you will be experts in making games on the Google Assistant. We're going to be too and who is excited to play these games and they're going to get into tips of how with words pixel sounds and the code to make it happen. And then lastly what you can do to be found on the Google Assistant. But before
I go any further, let me introduce myself. I am to Sean I am a product manager for fun and Delight on the Google assistant and I'm joined by Andy. He's also fun. He's our team creative lead and Ralphie who is our product manager on dialogflow. And while he doesn't have fun in his title. I can guarantee he's fun. We are a really big team of writers engineer's designers Founders who drive the personality and Delight of the Google Assistant the steam crafts how we
respond to lots of parties. Like tell me a joke to how are you doing or is surprisingly popular one, which is hey Google. I love you. People say that. And the reason we do this is what when you think of the Google Assistant everyone thinks it's about getting things done. And that is absolutely true. We're here to make your life easier. But we're also here to make it a lot more fun to and that's our little slogan for it for our team. We build apps like a trivia game called. Are you feeling lucky? And
the reason we do this is it that we can learn how to make these games and make sure that they are is apis that you need exists so that you can go tell them we also build templates and we worked with BuzzFeed recently to bring them onboard onto the Google assistant and where you can find out what desert are you? You know all this fun quizzes that they have on their site now you can play through the Google Assistant. And we also work with awesome startups like Vale who made a game called King for a Day. That's one of the first that runs on the smart displays that you probably
seen around and we're excited to show you those those apps. For example, they run on the smart this way right here and there soon will be running on phones and TVs and we've tried to make it easy for you. Rest assured all those awesome Google actions that you've already built can get upgraded with he's with Technologies. You're very familiar with HTML5 to make that happen. I'm sure many of you are like us where when you were a kid. You want to the talking robot. I always wanted Optimus Prime
and now we have it all the Google Assistant. And that is quite an opportunity for us. This is one of my favorite charts. Each curve here is the adoption curve of new technologies overtime. And if you can see each one gets sharper and has hired option rates as we get closer to where we are now and then you look at us smart speakers. Now while it's small right now that curve is the sharpest of them all people are running to these devices and adding it to their homes around the world.
Now you probably think of the Google assistant and is primarily coming through these two Google homes and and tons made by Third parties and that's definitely the way to think of us, but we want you to think of it. It's more than just those devices. The assistant is everywhere. It's in your kitchen on those smart displays. It's in the living room on TVs and most importantly it's on the goats and millions and millions of people's pockets on their phone. We think that's an opportunity to reach users in a variety of modes in a variety of places and we believe it's
a new type of play we call it conversational play and why we feel like we're and I'm in the black and white film era of this like we're still learning what how to go build these games. We do know a couple things when we look at who are users are we're staying up a couple patterns of who they are there Millennials their families and their kids. perfect game players We also want to highlight this this is Angry Birds rise to Fame in what's important to note? Is it took a
while before Angry Birds that came on Seminole games on touch screens smartphones were released and it took about two years before Angry Birds even came out and and rice. And this is where we are right now smart speakers have been out for a while. But there's an inflection point that we believe is going to happen particularly around play. And we're down there and we believe that one of you many of you hopefully all of you will be on that curve coming out. So I'm going to hand this off to Andy who will talk about building blocks of play.
Alright, thank you to Sean. So I'm Andy and I am also officially fun. And what I would like to do is talk to you about the building blocks of great conversational gameplay. There's going to be unique challenges that makes designing conversational games very different than your console PC or mobile games and took here are some of the principles that our team uses to deliver engaging experiences that users love. First we need to create the universe think of some of your favorite games. What are the world that they transport you to some of my old school
favorites are things like Dark Castle and space class in Delta. And there's this orchestration of ailments do the background the characters in the story The Sound they all work together, but they often rely on visuals to bring this world to life. Now it's conversational games. We don't always have those elements to work with do we need to look for inspiration elsewhere and one of the places that we can look is the oral storytelling techniques that go back hundreds of years. So a simple story starting phrase like Once Upon a Time help
set the stage in fairy tales fables and miss and the origins date back past the 1400. They're all so pretty much Universal. You can find equivalents in dozens of languages. Do things like a long time ago or there once lived. So rustic re our universe, we need to set the stage just like once upon a time does for fairy tales. It doesn't need to be much but it does help users understand the world are an example if you ask the assistant, what should I be for Halloween? It's going to go through a series of questions and help you decide a
silly costume that you should wear when you go trick or treating here to approaches on how we can bring the player into the game. Let's listen to The First One. Welcome to the Halloween costume quiz answer these five questions to find out what you should be for Halloween this year. All right, so that technically does the job it brings the user in a little bit. It also at least gives feedback to let users know that they've entered the right game, but I think we can do more. So now let's listen to this second option. Wow, I'm so glad you asked I don't want to toot my own horn, but I'm
kind of an expert great you give me thank you. Okay run along and have fun. I need to help someone know why. Okay, let me assess your costume interest with a few questions. Are you ready really starts to set the stage and more importantly it kept users excited about playing the game. So the other thing that we really need to do is to kind of look on ways to layer in soundesign. It's such a critical element, especially since we have fewer elements to work with so we have to look at ways to kind of paint this audio world. Now this can be done in a couple ways one is coming through
the background. So music or atmosphere to kind of really help kind of trait that that whole effect or we can also use as a foreground element. So maybe want to use it as a way to give feedback to the user that they've done something right or wrong or we want to use it to emphasize key moments of the lights or dialog to illustrate this point the Google Assistant has a poem for you are loving audience. It's titled bike and kickstand. So let's listen to this. Thanks for humoring me just give me a few seconds to write it down. You're like a bike and I'm like the
kickstand Lean on Me friend when it's time to make plans whether you're searching for info trying to spell Chateau looking for recipes or other necessities. I'll always reply. Alright, so pretty good. Now, let's hear it with some sound layered in. Thanks for humoring me just give me a few seconds to write it down. You're like a bike and I'm like the kickstand. Lean on Me friend when it's time to make plans whether you're searching for info trying to spell Chateau looking for recipes or other necessities. If you have
any questions, I'll always reply. You feel that it's right here. It's deep inside. It really just help Elevate everything and so fortunately later on in our wonderful program. We are going to show you how to add the love into your dialogue. Mount another thing to consider is how to use sound to get users to focus. This can be especially helpful in multiplayer experiences. So in our game are you feeling lucky users are automatically given names and since you can have up to six players at a time we found through our research that uses
might forget when it's their turn to answer. So to help with your help get users to focus while we pair them with audio cues. So let's listen to how that works. Okay, just so everybody is straight. I'll give you your names again. They are Nibbler scooter. freestyle and puddles do this to be a great way just to kind of associate that audio cue with that name bring users back and it's something that you guys might want to consider as you're developing games to how can you use
audio cues to get users to focus? Now our next principle is to keep it simple. Conversations they're inherently linear and it's really easy to move forwards but it's much harder to move back. We can't ask users to hold too much in their heads. So here are a few ways that you can keep things simple. The first is use a consistent structure familiarity is going to make people feel more comfortable and the more we have deviations. It's going to increase the cognitive load of that player
Andres to a higher level of confusion and possibly errors. Imagine that's wearing a quiz and here are just two different questions pretty similar in structure. Now we restate that question. And then we follow up with the options. So we still kind of can vary the language a little bit but you can see the structure is very consistent. And as the player goes through the experience there now going to know what to expect allowing them to focus on the content and not the game mechanics as
we push the boundaries of conversational gameplay. You may want to change the structure. So maybe you want to have like a bonus round or something like that going to be okay, as long as you take the time and teach the users of this portion of the game is going to be different. So you're going to need some leeway, you know, is that worth it? Is it worth it to take that time in order to kind of explain it even though it may slow the pace down a bit. Now another simple thing to keep in mind is when you list out the choices for a player just make sure you follow up with
the exact order. He'll be looking that would you rather travel on a tiny plane or surfboard we follow up with plain and then surfboard sometimes it's easier just to kind of switch those around but keep it consistent. Our next principle is to keep players engaged. We don't want your games to feel like a lecture. It's very easy for players to lose focus and get distracted. One way to do. This is by analyzing two players experience over the course of the game. So some of you kind of coming from the game worlds, maybe Austin Ally with the player
interest her which is essentially a way to represent the highlights and lowlights of a player's experience. And this is very similar to kind of creating user Journey Maps. If you're familiar with product design and some some of that kind of a similar types of Concepts, you're just simply going to watch players playing a game and you're going to notice kind of like key Milestones watch the reactions, you know, where they cringing are they excited and you're going to start to plot this as you do this, you're going to generate these patterns and I sees patterns happen. You're going to
want to kind of make sure that the curve is up don't looking at a graph like this. This is a little bit of a concern you can see it starts off with a great intro and it ends with an outcome, but the middle kind of sad that and we don't we don't want that to happen. One way to keep players engaged is to make the player parts of the conversation. You know, your experience is going to be directed by the dialogue. So we want to make sure that we're always talking with the player and not at the player. You can do this by creating digestible chunks you find moments for your
player to participate. The other thing that we can do is response contextually whenever possible crafter dialogue based on that players response. So imagine care the player is asked if they would rather play with a unicorn or a dragon and here we have a very generic response. So this could be something for any of the questions. However, in this example we are able to respond something that's not only a bit more delightful. But it also makes the game feel smarter. It's providing simple feedback to the user that the game understands what they said.
Now he's being moved to kind of more visual experiences a smart displays. What we're going to want to do is avoid transcribing word for word all of the dialogue Tucson mentioned before the BuzzFeed quiz, and so here is kind of an example on the right here. We see this is the TTS. So this is what the output would be on the left is more of the experience itself. So you can see we're not just kind of using the screen just to come throw all that text and we want to simplify it. We want to move the user forward. We want to use the screen to create that Universe. We also want to just make
sure there's opportunities to kind of increase the brand and create more of an emotional connection with the user. So don't transcribe your text word-for-word. Another way to keep players engaged is to have a great pay off. Our users time is very valuable and there's a lot of competition for it. So we need to make sure that we were wore them for getting to the end. So if you remember that interest her but I'm showing before it ended on that high note you guys as game designers are going to figure out what that payoff is going to be.
Alright, our last principle is to always be playtesting now in the game industry. This is nothing new. We know that it's going to help increase quality. We know it's going to help find bugs and basic design flaws. We also know that engagement is going to drop off if the game feels broken and we should expect the errors are going to happen, especially with conversational games. We know that players can respond with anything the environment may be polluted just kind of full of background noise. We definitely got a bit of that as we are kind of in the sandbox and you can kind
of hear all the people around, you know, there's just to be situations like that. If you're at a party where it might be tough, so we have to anticipate some of that. We also know that players may forget what to do next a few common errors that we see are going to be things like no input errors where the assistant just hasn't detected a response at all. We also see things like no match errors where maybe the assistant can't understand the reply and we also get false except errors. This might be where the assistant misunderstands a user but just kind of moves along anyway, So
here's a few things that we can do to minimize those errors and maximize the fun. 1 start with your concept and you can do this in a very Lo-Fi way. So have a user's players read a script of kind of a happy path or essentially the ideal path of the gameplay. You need to make sure that the gameplay is actually fun if the happy passes and happy then it might be time to look at ways to iterate on it. Another thing we do is we like to test with her eyes closed. So these are conversational games and you start by designing with voice only you can
do this with a working prototypes or something like dialogflow or you can also do this as like a Wizard of Oz technique where team member May stimulate the response of the system and you kind of have it set up in this controlled environment. This is a great way to help test the content the pacing and the complexity of your game mechanics. The last suggestion here is to test with a range of audiences. Not just your primary demographic. So watch your parents play with it. Watch your kids play with it. You know, they
made Bubble Up issues with comprehension and complexity of your dialogue. They make kind of December that some words are just hard to say. Go to wrap up really quick some of the basic building blocks that our team uses are create the universe keep it simple. Keep users engaged and constantly playtest now. I'm going to hand it over to Ralphie. Hello everyone. This is Ralphie who's also fun from the lacrosse team? Now that you're equipped with the expert tips from Andy, let's dive into how we
can actually Implement office. The good news is starting is really simple. We are working really hard on developing tools and capabilities to make it easier for you to focus on what you do the best developing games. It's a high-level developing actions for Google has three main components this applies to any kind of action games or non games. The first of is you need to build your conversation of floss. Then you need to connect your conversation flow to your game logic and lastly you need to publish your actions to Google Assistant. I'm going to focus on dialogue for our
industry-leading conversation agent fluff on creation platform actions on Google console, which is the tool that you probably should actions to Google Assistant. Dialogflow is a tool that makes it super easy to build conversation flows from dialogues applying machine learning to the training examples that you provide. Let me tell you how this works. Imagine you said the following sentence send three dragons to England immediately attacked with green Flames for 5 hours. Guy LaFleur on the sin that understands that this
is about game turn. It's not about user asking for help or trying to exit the game about game turn it to identify important Concepts within this piece of dialogue. It on the Stars coming concerts, like numbers countries colors can also extract specific Concepts about your game like characters and weapons. It's smart enough to understand some more complex Concepts like time which is relative in this example why you say 5 hours it resulted from the time that the sentence was 1/3 to 5 hours later. And the cool thing about dialogflow is it comes with built-in in
Spanish translation support with just a few clicks. You can replicate the same conversation a flaw in another language. If you want to learn more about that looks so you can attend a dedicated session on it at 3:30 on stage 2. Now that you feel your actions, it's time to test up until now you can only test your actions using your own developer account on devices that are associated with that particular developer account new feature that allows that allows you to share your actions would set of alpha or beta users. The alpha Helix is really good for testing and
design and development case. They're not fully Seal reviews. And that's the quickest way to get to your testers and get feedback and it right. The better releases for games that are ready to launch it goes through full review process. The goal here is to find edge cases bugs and issues with final polish. Ultimately. We want you to produce the best game possible. How do you build test and publish affect any part of the Google echo system, so you should make the most out of that. The easiest way to do that is account linking. It allows your
users to use their own Google accounts to play your games, or they can link their Google account to your account management system. Set up account linking simply just go to the accountant can be on access console. There you can Define what permissions you plan to ask from user these permissions or Define a scope. For example, you can ask for access to use as contact calendar photos or other Google services. They might be using. Show me use his perspective deal go through this familiar Google sign in slow to use their own Google account or link the Google account today
or gaming account. Creating the expenses developer is fairly simple. Just right literally two lines of code and use our sign in helper function to request sign in. Once you do that you need to handle users response. If the user goes to the authentication flow successfully, you'll receive an access token what you can use to access the resources that you asked permission for if something goes wrong, you just need to acknowledge it and and Grace Lee active. One of the key differentiator of Google
assistant is the fact that it's available in various modalities and enables different attraction types and how to manage these different interaction types. Let me call you to go over what I mean by these different infection types. The first one is what we called voice only you can think of your smart speakers the whole in fractions voice-based input and output. The next contraction type is called voice forward. These are smart speakers you've seen in in the sandbox area here again. The primary motor contraction is voice what is enhanced with visuals when when
necessary? The turkey interaction model is Intermodal you can think of your phone here. You have conversation and visuals intermix, you can start with a conversation switch to a Visual and go back and forth. And the last one is visual. Only phone on mute is an example here. There's no audio input or output. It's all about visuals and text. So let's go to an example. Miss it at a user's playing your game on a smart speaker. And then you want to show them the the image of the new monster evolved.
We Define the image here. The first thing we need to do is we need to check if the device that if the device the players playing your game on has the capability to show an image. This capability is defined as screen output. If a device has this capability great, we just show the image but in our example of the players playing on a smart speaker, which doesn't have that we ask music tools to switch the device that has this capability we do that with the helper function called new Surface. We Define what capability we are looking for in
the sky screen off, but we give the contacts to the user on the smart speaker. There is acting with to see your evolve monster go to your your phone and I will send a notification to the device that we want to get the user in this example. We say see how your monster has the Vault and then they can talk to this application and continue their infraction on their phone. The final step is a developer to manage the slowest you need to you need to handle the response of the user for our new new Surface from its use it except the from move to the new capable device. Then
you simply show the show the image if they don't you just need to gracefully exit and acknowledge that that you won't be able to show the image. What's your game action? Start getting traction? You need to think about the engagement in our platform to help you to help you engage with your users. We literally have a section called username engagement on our actions, So there you can often to push notifications routines and data update suggestions. If you want them to push notifications you will you will be allowed to send notifications to to
your users only see two users needs to provide consent to treat needs to be used topically and sparingly. You can think of routines as actions came together. For example, we have something like I'm home routine where you can Bond a bunch of apps you want to trigger and if you say Hey Google I'm home. You might have Google turn on the lights adjust the thermostat tell you a joke, maybe and start your favorite playlist. Video of places also similar you set up a daily notification that reoccurs every day at the set
time. The user experience for daily routines and they are Cassandra teams' after your users engage with your app an exit. They'll see a suggestion ship prompting them to add your action to daily updates are all routines. Rookies may not be that suitable for gaming vertical but something to think about I'm just going to focus on date outfits for now. This is what you see. This is what your users will see if you opt into the update after the user and the infection that will be a chip that says do you want me to send in the sky is jokes daily
user option. Google will guide the user to setting the time of the day when they want to receive the daily update from this point to do two things. First you need to handle users response to the Date Update flow. If the user goes through the floor successfully unit acknowledge that if not again, you need to graceway exit. The second step is you need to Define what you actually want to do when the daily trigger arrived here with set to trigger the update intent. In the back best dad jokes
example, the update in turn would just send a new job. Ultimate this is what your users will see. Leslie I'd like to talk about money station options for your game. We just announced our digital Goods API. IO that allows you to go and talk to him using Google Play infrastructure early launch this year where you could use your existing play entitlements with assistant actions, for example, if you bought an audiobook or purchase a monster in your game, you could actually use it in your in your Google Assistant. What did you tell Goose API. It makes the universe Tulsa true. If you have a
transaction in your action the same the same entitlement can be used in your Android app from our partners in headspace to show you how this could work on the smart display. For those of you who don't know about that space that their company who built products on guided meditation. So let's say that the session is over and headspace offers the option to subscribe. BSA response with the intent to subscribe and the action shows available options in a very curated and visual way. User picks an option and its hand over to the checkout slow.
Here if you been able to come blinking and the user already has a confidential store with Google to check out would be very seamless you just with a single type of command you have your premiums contact that can be reused on on the assistant and also an Android apps with that. I'm going to hand it back to Tucson who's going to talk about implementing sound best practices in your games. So now we're going to Pump It Up. If you remember back when Andy was talking we showed the difference of what sound design can do these devices really
rely on music sound effects and voice and we have some technology that we wanted to Showcase that that can help you pump it up quite a bit you could for example record tons and tons of lines with a voice actor and get a sound engineer to add sound effects in and make some beautiful audio you could use this technology called SML which is speech synthesis markup language that allows you to do it in real time in code and what that also allows you to do is to put variables along the way now we have this static poem, but imagine if it said thank you for
humoring me to shun you could add personalization to it and yet still have that Dynamic sound effects and all so here's our script. And we're going to start pumping it up SML is just like HTML you just add tags. You wrap it in a speak tag, you put paragraph tags when you have each line and then you can start doing things like adding sound effects and it's a simple audio URL from your server. But there's more than just text and sound effects. There's a sound layering and that's what we talk about here. We have a tag called media. What media
does is allow you to such a make two or multiple tracks and then layer them together. This is our first track here. I'm going to call a poem and it's going to start one second in and you'll see why in a second. We had another sound effect that we even put a little dramatic pause. And then we worked on our music Loop. We call it music we can control how many times that Loops through set the sound level if we wanted to fade in / fade out. And then we use that name from earlier when the poem ends what's running for another second
and then we have our music Loop itself. So all together we wrap it in another tag called parallel or Parr and that runs both of those tracks together. Now what I showed you here is a truncated version of that poem The let's hear it. And here that example. Thanks for humoring me just give me a few seconds to write it down. Sound effects, you're like a bike and I'm like the kickstand got the music. On my friends when it's time to make plans whether you're searching for info trying to spell Chateau looking for recipes or other necessities. If you
have any questions, I'll always reply. And not fade out. That's all done in code. The other big thing we want to leave you with is if you make something and no one hears it only is that a punch in the gut though, you're wondering what you can do to make that happen happen. So we want to tell you that very very soon. We're going to be opening up Hey Google, let's play a game to all Developers. This is called a built-in intent. There are other talks about it, but register your app for it consider it the App Store
of games on the Google assistant and when someone says play a game which is what a lot of our users are doing don't get a list of of the games available in this works on those smart displays phones and smart speakers. But to make that happen with a little homework for you go back into your actions on Google console and fill out all the metadata. That's what we used to look at your rap know what what it's about and feature it with user. So make sure you're beautiful icons and your descriptions are great there.
And I'm going to tell you a little secret you don't usually talk about how we rank things, but I wanted highlight this this is how we look for equality. Straight up Martini. It sounds great. The gameplay is great and users a loving it and rating it. We're looking for long sessions and games that have people coming back for more and more. We're looking for games that map to those core demographics of Millennials families and kids and we're looking for games that aren't like the others around there. We want to offer our users a wide variety of games. And then
if you can update your apps regularly and feature the new tech like jumping onto those smart displays that will give you a great shot of getting in front of users. So we want to leave you with one last note. I think many of you have been on the touchscreen revolution of making apps. And we think there's an opportunity very similar to here with these smart speakers and digital assistants. And on behalf of the dialogflow the actions on Google team assistant. We know that it's hard but we're here for you. We want to see you succeed
and we would love to hear from you. I'm going to show you our email addresses later, but we think there's an opportunity to become the household name of conversation and play and games and we want it to be you all of you. Thank you so much. Please leave us feedback and take a snapshot of our email addresses. We would love to hear from you be at new tech or questions about how to use it a nap. So you're making would love to play with them. Thanks so much, and we can't wait to play your game.
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.