Duration 35:52
16+
Play
Video

Build an AR app with the Poly Toolkit for Unity

Brittany Mennuti
Product Manager at Google
+ 1 speaker
  • Video
  • Table of contents
  • Video
2018 Google I/O
May 10, 2018, Mountain View, USA
2018 Google I/O
Video
Build an AR app with the Poly Toolkit for Unity
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
26.91 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speakers

Brittany Mennuti
Product Manager at Google
Bruno Oliveira
Software Engineer at Google

Brittany is a product manager on Daydream. She oversees apps that make creation of AR and VR content easier for creators and developers, such as Blocks and Poly. Prior to Google, Brittany developed tools for Vine creators at Twitter and led the shipping team at Etsy, building products to help Etsy sellers easily fulfill their orders. Brittany has her BSE with a concentration in marketing and a minor in fine arts from the Wharton School at the University of Pennsylvania.

View the profile

Bruno is a software engineer working on virtual reality and augmented reality. He is part of the Poly team focusing on APIs and toolkits to help developers use Poly in their applications. He has previously worked on the Daydream platform team, the Google Blocks team, Google Play Games and Android.

View the profile

About the talk

Poly provides all the resources necessary to build an AR application - even if you have no prior 3D design or programming experience. With its repository of thousands of free Creative Commons 3D assets, Poly is a platform to bring AR ideas to life. In this session, a live demo will show you how to use the Poly toolkit for Unity to quickly create an AR app.

Share

Hi everyone. Thanks so much for joining us today. My name is Britt minuti. I'm a product manager on the Google arvr team. I'm joined here today by my colleague Bruno software engineer on the team. We have an awesome agenda ahead for you today. So by the end of today session has helped us to demo and a RF, but before we get into that, I'll share a little bit of history as to how and why we built poly. I'll share what poly is and how it can be useful to you as a developer and I'll pass it off to Bruno to dive into the poly API before we do the demo. I may be wondering what is this poly thing

and why should I care about it? So I think the best way to answer that question is to take a quick trip down poly developer a Memory Lane. The team that built Polly was actually originally devoted the idea of productivity in VR. It was our goal to make people more efficient and effective at work using immersive technology with a whole bunch of hypotheses about how we could do that which figure the best way to validate those hypotheses was to build a whole bunch of your prototypes and test them with real people. We both everything like organizing your docs in VR to training people on how to

use the espresso machine and are micro kitchen, but regardless of the Prototype that we built one thing was always the same. We needed 3D models and finding the right 3D model was really really hard specialist. We wanted a lot of 3D models that look like they belong together. We said to ourselves. Okay, no big deal. We'll make our own 3D models turns out making your own 3D models. It's also really hard. And in fact what you see here on this screen is a model of a hand created by a software engineer named Chris on our team for one of our virtual meeting prototype. Now how many of you

actually knew that was a hands before I told you I don't believe thinking that maybe creating models could actually be a lot easier and perhaps the are was the answer to that. So we pivoted and we built blocks blocks is an app for the HTC Vive and Oculus Rift that lets you create beautiful 3D models in VR and what makes block special compared to other my applications is that you actually inhabit the same space as a thing that you're modeling. So instead of training your brain to think about a 3D object on a 2d surface. You can

literally just walk around it. Thanks to the launch of blocks that same engineer Chris that I mentioned earlier was able to make this beautiful fish tank model in less than 15 minutes. He also made this amazing robot workstation all unblocked, and I don't know about you, but I think that that robotham looks a little bit better than the hand. I showed you earlier. You might have also seen Chris and some of our launch videos. That's right folks one day you too can become famous. And that's a reflecting unblocks. You started to realize there were a whole bunch of other resources for 3D

assets. That could be used in AR & VR development. For example still fresh is another VR app that lets you paint in 3D space. You can create everything from beautiful portraits to entire 3D environments replete with both audio and visual effects. And we also knew that there was this vast community of 3D model is out there already making tons of assets that could be used in AR & VR development. So we talk to ourselves. Maybe the problem wasn't that there wasn't an easy way to make models. It was also that there wasn't a single resource to sign all those assets to use for

development. And that's why we built Poli Poli is a library of 3D assets and experiences and our mission is to organize the world 3D information and make it accessible to creators and developers like yourself. When you visit Polly you'll see thousands of ss. Some of which have been contributed by Google but many more of which have been uploaded by a vast community of 3D modeler is using block still fresh and other modeling a and when you upload your assets to poly, you have the option to make them what we call remix of all what this means is that you give other creators

and developers the right to use your assets under a Creative Commons by license. This means they can download those assets use them in their app as long as they give you a tribution. Well, the good news is that over 82% of public assets on poly today or actually remix of all so what this means is there are tens of thousands of assets available for all of you to use today for free under Creative Commons license. And when you go to poly you can Browse by category. You can also search for exactly what you need. We recently added object recognition to start so you'll find

what you need. Even if it isn't manually tagged. And when you find the perfect ass that you can download the file we offered downloads in the form of obj and in some cases fbx files as well. Now you might be thinking that sounds great. But can I actually find useful assets on poly? Well, let's look through some examples. Maybe you needed a sandcastle model for your vacation Mode app. Or perhaps you want to recreate Noah's Ark in your living room. Or maybe you've always wanted to include a realistic version of your BFF Carl and your

next game fun facts Carl's actually a real human. He's actually one of The Talented designers behind blocks and Polly. Or maybe you're building an archaeology app. I need a whole bunch of real 3D scans of historic artifacts. Well, thanks to a recent partnership between Google arts and culture and Sai are there are a dozen of these actual artifacts hosted on poly for you to download today. And finally, perhaps all you ever wanted was a politically correct cheeseburger. That's right folks. This burger has a cheese and both locations. No matter what you're looking for and you'll find

everything you need on Polly's pretty unique VR or ar experience like this VR Scavenger Hunt game created by artist and developer Harland Perez. He built this using blocks poly and Unreal Engine. Or these ARS on the left still see an app from developer answered white retakes beautiful flower models from Polly and put some in the otherwise dismal New York City subway platform or on the right and a half from developer benford's that uses object recognition and the poly API to retrieve low-poly version of Real World objects in real time.

Now she stopped back to Pawley's Mission. It occurred to us that we might be falling a little bit short on the accessible portion having to go to a website and download an object. Every time you needed it for your development was perhaps not the most efficient solution. So that's why we built the Polly API and with that I'm going to hand it off to Bruno to share more. right So we're going to talk about the palladian vale to get there. So many ways that I could begin a presentation about an API someone

with an inspirational quote something. Somebody famous said that's relevant to the topic. So this is the one that I chose. I go to the eye creates more problems than it solves. Does anybody know who famously said this and when? That's right. nobody ever and that's why any time you're working with an API. There's a simple but fundamental question. They have to ask that question is why? Why would you use this API? How is my life easier or better if I use the poly API? Transfer that

lets go back a couple of decades. So I've always liked building games. So my friends and I would get together and it would try to build a bunch of games in the old computer stuff the time. We kind of knew how to do the code. But the problem that we had was that none of us really knew how to make art with no skills whatsoever expected years with Oreo challenge. But somehow we have very high expectations about what it what are Graphics we're going to look like. In reality, however, our Graphics look more like this. Let me know what that is.

It's a dragon. Of course. But come on. If this is a valid hand them out to valid dragon. Are you going to think that my skills would have improved over time? However, you would be wrong. I'm the kind of developer who uses cubes for everything. So with my skill level I could theoretically make a game about a cube that shoots other cubes. Protect the larger cubes from being hit by the smaller cubes all against the backdrop of moving cubes. You think that

there's no way that that game is viable you think that there's no way that's an actual thing. It's because you haven't played cubic violet. Not only did I launch this but Google in some official capacity actually endorse this because this is a sample for the Google Play games API. Now my point with all this is that we all have this book as developers in our Shelf. It's in my instant. It's at the top of my shelf. Everyone has it. It's the best seller. It's all apologizing for the lack of a graphics. Sometimes. I know we can we can get away with it. Sometimes I can do clever tricks in our

games to make graphics are necessary. My favorite one is Magic the dragon is invisible. But it doesn't work every time user stand up noticing that we're doing this. Mount something for when I show them this. What they suggested me is why don't you just buy a collection of three assets? You don't just buy a collection of light like nice 3D assets and then you don't have to use cubes or ugly dragons anymore. And then I say well this is the traditional model for asset acquisition. This model has a couple of problems.

What are the problems with this model? biscotts obviously Maybe I should go into more details. All right, if you ever actually try to buy a 3D model of a cat online, you know what I mean? You can just buy one cats. You have to buy super cat back Deluxe special edition with 740 regular nanometers gas hard to find with separate assets. The second problem is that sometimes you're still in the prototyping phase of your amp at this point in my app development cycle? I'm not even sure if I'm making a game about cats or about spaceships or even about cats that

fly spaceships. It's about a thing. So I need that play with the flexibility to try it out with different acids. And the third problem is users. Well, as you know, your thoughts are fickle and they don't know what they want. So what happens if you ship your amp and then I know it's Unthinkable but suppose that you never actually thought about cats at all in the development cycle of your amp. And now when it's in the user's hand at one time that you just want to have a cat in the amp, how do you do that? Well, you would need to somehow figure out a way to pull out a cat from a

run time API. so probably by contrast has the flexibility that you need. Do I tryout with many different assets from poly? It has a run time API which means that you can go and search for cats and then presents cats to the user or anything else if they want. And also it's not a static election. It's not something that you buy at once and then you only have the same acids. It's a growing collection. Because users are contributing to that collection all the time. If you see they're going to see that even I with my limited skills made my

contribution. Can you guess which one is my 3D model over there? any guesses It's actually none of them. I uploaded this Cube. So whenever you're making a game and you need a cube, you know, we're looking. So we have this week collection of assets on poly and then on the other side, we have our amps and you want to be able to consume those houses. So what's the missing piece in the puzzle? Of course, this is an API. Wpi's how to get taxes through these Assets in Foley Al

naming is difficult. So we put a bunch of people together to come up with a name for this API for poly and after several hours we came up with all the API. if I precise What about a high-level? How does the safety I work? So I just want to make a search quarry. In the simplest case that's going to be a keyword. For example, the active say I like cats I would like some cats please. And then the AP I would return some interesting cats have to consume. What happens if you're not a cat person? Can you still use this API the answer is yes. Or what happens if you want to be much more specific about

the kinds of animals that you want in your amp in that case. You can make a more complex request. example you could say I like low complexity assets from the animals category medium toothbrush, and I want to be ordered from newest to oldest so I can make very complex request like that. You maybe you'll probably will not get any cats on this one because we know cats are very complex animals. And if a user signed in you also have the option of getting the user's own assets. So this means that you can get the answers that the user has

uploaded to Folly but hasn't published yet so I can Assets in draft State and he could do it use that for example, so that these are add those to your scene. so we talked about the Watts and we talked about the Y now, let's move on to the next question, which is when when do we use the poly VI? Fortunately the answer to this one is much easier. How about now? Let's do a live demo. You know, what's the first rule of life live demos on stage? If you never do a live demo on stage, I'm going to try and see if this will work.

How many people actually think of his name was going to work? How many people think the same was going to fail? How many people think it's going to fail in some like African-American way like something's going to catch fire on stage. That's not bad. So this is using arcore and put the Pollito kit for Unity. So first of all, this is just a r for showing me the planes that it's tracking. So we have your that's very nice carpet. And now when I tap my finger on the screen, I'm going to place an object and this object came from poly but it's free

compiled into the app like this little intersection down to the ground. I was just I have this hack, but I can increase the height of this. Right here with me. So we have those little intersection to make it look more like a real City. What's one thing that we could add to make it look more real? CarMax in cars or cars I guess search I made this keyboard myself and I'm going to do these are all search results and papers for a little bit and I'm going to

streets. So actually put some put some scripts in this so that because I should do something so if it would run then the car. Bull Run and Alice had some other cars so go back to his house. That's at this one. I'm going to place it over here hiding games. But you know in some games your goal is to like save the kingdom save the world in this game. Your goal is just to make traffic as bad as possible. The other mini game in business how many cars can you put in the steam without

them crashing through each other? My high score is 1 let's see if I beat my high score. Well, my house far is not to. Mess up something else. What's something else you would see in a typical city after all this is a r searching for boats boats we have All right. Can I have this Cruise liner with a little bigger? I'm going to put this Cruise liner here on the road that we go. Show me driving on the wrong side of the road. I think that's not the worst traffic violation that's happening here.

Oh, my high scores is Dude Perfect. Let's have something else we could do all forms of transportation or something like that. I'm just making this up as I go. Dora oh, that's what this year. giant jar of jam and I got this traffic jam exactly the terrible pain. So this is how you can look at objects from Pauline to this demo and I think pretty much nothing else surprised. He's going to happen if this demo, of course a dragon and set fire to everything

for those of you who thought that something is going to catch fire you all right away. All right. I'm grabbing customer work. I know that's what the how do I actually use the API? Play Nepali Bhai on one side. And you have are you up on the other side and it's naturally cross-platform because it's the rest API. This means they can get access to it from any device that has access to the internet that we support several platforms and masculinity and real web turn on the police took it for you to be so probably took it for

you in the last few used for UTI. So when you download and install poly took it it becomes fully integrated into your Unity editor. So you're going to see that window. I know that window we can browse for all the outfits so I can see that's going to show you by the fault is going to show you the featured ones on the website where you can change the filters. So you could feel to buy a object type you could feel her by category and you could say if they all let me look at the stuff in the transport category the food category you could also do a search. So

there's a little search button there. So if you are you need a dragon for African search for Dragon, if you need a cube, if you need a cat anything you want this place searched and then you find the answer that Union. Then once you find once you find some good acid that you want to import into your scene is on my team. I noticed that this is already this is already in your senior project. And this is already a perfectly reasonable app. If you could play it's going to run and it's already like a fool

app. Of course, if you actually want the garden to do something you have to write some code. Otherwise, it's going to be more than more like one of those contemplatively Gardens. But you know, what's going to happen. You're going to ship this app on top of the Garden on Google Play. And then the first thing that users are going to do is they're going to say sure that's Gardens. Very nice. however I want a cat. That's what that's what was going to happen. So to get the users there cats, you have to use the poly API. Infuse runtime part of the Polly API

I have no idea what that blue robot is doing. It's an actual picture from what's going on in the Datacenter right now. If it's an overview of how God works a request object example would be the request. This request operating system codes your search Quarry. Then you pass a few of the object to one of the API methods for example list assets that match that's search request. Then you're going to wait for the call back on the cob is going to have the results for

your search and then bazan. Search. You have to figure out which ones you want. If there's more if there are several matches you have to figure out which one you actually want to import and then lastly you call Paula at the other import report That Into You scene. Bumble bees example of the list. I requested featured. This is a default feature flooring. This means you want to get exactly what Folly is currently showing of the website. So these are assets no specific category. No specific keyword. Does anything good that always showing probably show him the website. Then an

executor, of course you pass your request and then once I got a call back, you're going to get a list of houses that you want. Then you going to choose among those assets in some way. For example, you could just let the user pick one of them and then wants to know which one you want to report. You just call Polly apis of import and passing the asset and that's going to get imported into you. Don't suppose they want to do a keyword search, you know example we want guys to know that you're simply the set the keywords field of the request to

cap. And that's it. Then the exit through West and then I get gas. But the fault you're going to get answers from u.s. To latest, which is normally a good sorting order. But what if you want what if I don't want new cards, I want the best gas and that is accepted sorting order to pull the order. Fast and then I get from from the best cats to the worst cat. What happens if you are more picky about the cats that you want while they can fill in order of the fields. For example, you could specify the maximum by Citi Field say I want me to come back for the cats at most

Steve the polygon complexity of the model. So if you want if you're running an amp in a performance sensitive environments, you might just want low complexity models. If you don't if you're not perfectly straight about 4 months to grass for high prefer for high complexity models that are realistic. You can also set the for my filter and you can request how many cans per page. Do you actually want? Forgot it's getting close to lunchtime. We're getting hungry. So of course he has a solution for that. You can request anything from our food category are using an API call. If

you start the category 2 food, then I got to get going to get everything from the food category of food at a time is newest to oldest. So chances are if we run this for you right now. What happens if you're worried about the quality of the assets, maybe I don't want any cookies that anybody has uploaded into the side. I want to have some pretty area. so You could just hire a team and have a team whose only job it is to search for assets and mark the ones that are good that you went up is that you use it and it was or not, but we already had to do work because that's what we do for the

body. What website so we decided to expose that in the FBI to make things easier. So we have a flag status and you can use that in your search. This means that whatever acid has that said to chew, it means that we there are human reviewers actually Mark that as being a good answer. So if then you can use that flag if you're 30 are assets as well. Thumbs up. All they have to do is circulated equals to and that will filter only by the review acids. Now we're going to have a good problem to solve which as you run the query and I have a bunch of choices for the user have a

bunch of cats that they can choose from. So, how do you actually let them pick a cat or a dragon or a car or anything after the search for? 40 killed in fiery just download all the models important to you in the objects and then show them to the user right but this might be wasteful because you might actually spend a lot of memory and a lot of time downloading those and then these are just going to pick one and then never look at the other cats. So what's a more economical way to do this while we have some nails so dep. I allows you to download something else for any assets in our API.

So for example, if you have an asset from Melissa, Texas results, and you want a thumbnail for it to show to the user. All they have to do is call the patch thumbnail API call. And then you're going to get a call back with a with a Unity texture 2D that represents a thumbnail for that asset. You could do anything that could map it to an object or you could just converted into a Sprite and then displayed on the screen. This is what we do in our demo when were showing the possible cars. So these are not the actual models. What I'm doing is I'm showing the thumbnails for the models and

Wednesdays are actually one of the men are we going to be sure that the other ones that car? That's when I actually import tomorrow. Picking up placing objects what happens if you just place an object in the world that you need to make sure there's an object is anchored to some real-world surface to do this. Airport hangars. Yes exactly. How did you know? So that's when the actors in handy. So when are for the Texas table service is going to report the surface to you through the API. If you want to add an

object so that it's from a place on that surface you use what is called an anchor object. But what you do is lie, and then you said it's parent to the Anchor object. So that means that the object will remain stable on that table. Even if you move the phone around if you change your edition of the phone, it's going to be from the place of the tabletop, even if it's going to remain there. How do you accomplish this in code or after you put an asset? What you do is just that the active member to false. This is going to make it so that the object is invisible before

the user places it and then presumably you going to show some your wife for the user was going to say well that the screen somewhere to place is object. What's the user touches the screen to place the object then you do a reicast to figure out where these are head. And when you do a reicast you can figure out exactly what the foes of the rake as hit is and that's what you used with the anchor. Once you have that you create the anchor. And then finally you're going to set the object visibility property to so that the object begin surrender and that's all there is to it

set the anchor and now we have this objects in your yard. And a lot of other features that we're not going to cover so it does automatically skating of objects for you if they can convert units. So you're seeing is in centimeters of the object is the meters is going to convert and you can get the picture that says you can get the user's own assets we can get the even the assets that the user has liked. You can change the material assignments. If you have a different look for your amp and he needs a specific

materials do automatic attribution generation. So you don't have to keep track of each Which object to download it from where we just automatically generate a copyright file they can display in your amp. Book a little bit about what goes on behind the scenes that you don't have to worry about it does for you. So what's the difference between the two foxes The Fox and the last in the Fox on bed light? Well clearly the fox in there by 2 sitting down. right The other difference between us is that one is a model and one is data. So model is a

collection of polygons that you want to render on your C and data is just characters. the data represents tomorrow no Vale API, of course only outputs data and what it does is convert the data into Unity models. For that we need to establish the format. This is one of those things that could be easy. But in reality is not cuz apparently nobody reason how to do this. This is a partial list of 3D formats from Wikipedia. Now we have to start somewhere. So we decided to start with the most basic one.

Obj a great format and it has everything that you're ever going to need for 3D object if it's 1995. Or today's standards. It's a little under specified. It has a bunch of non-standard extensions limited material support. How to be a little more modern we have gltf it has it has an apple express Savannah so I can display how many Canon killed scenes nose and materials and best of all it's a format that gets everything right? It's stable. It's future-proof and it's

unlikely to change in the near future. Just kidding. It's changed. However, Joseph you got everything right and is unlikely to change in the near future. Free practice in the polypeptide we offer objects both in for a minute and then in the DLC of to format, so if you're consuming the body of a directly you can choose which one you want so I can choose obj Joseph to employee toolkit uses y'all TF2 and imports dance object everything automatically. All right. So we talked about Polly to

prepare uni, there's no real and if you're developing a native app on Android iOS web site showing how to integrate with apple API on those platforms example example of the same Principle as the environments where I can download and then use for the Assets in your seat. Developing for Android there's a sample that shows you how to include as if somebody into a nail or just tap the screen and then you can include specific while the object on your scene. If you're using web, there's also sample of a simple users to BJs.

This is actually the easiest one because our website is also built using 3DS as a fun fact the guy who created 3D ass is the one who wrote the sample. So this just shows the wide variety of experiences so you can build with with the body of the eye. For example, you can also be no sweetie. I listen example of a simple Escape the Room app of your after we made in the office a couple of months ago just before the object that you know, this isn't even though different people created the different objects in that scene. They somehow it'll look nice together. They look

like they belong to the same thing and also added some waiting for the next respect and we got this bad like cheesy spooky atmosphere that we wanted the both of us. And I think I just spoil the game for everybody because now you know that the exits go to 714. Fun fact when we actually try to play this game the office the first person that gave it to me just put the headset on and beside you have to manage to find a way to escape the room through the door and what they did just walk through the door and want the game. Because wants to stop them

and then we have to be more specific for the second user. So you actually have to solve her puzzles before you walk through that door. It's just shows you some some of the design problems. Well for me, of course what it means is that I can stop using assets like that and maybe I feel great my ass has to look more like that. I'm not exactly a dragon expert but I can sense that. There was some improvement in Dragon quality there. But you're interested in trying out the file API and building applications with poly, please visit our site.

So probably. Google.com is the 3D Gallery site where I can browse acids develop quality is a developer sites where you going to find everything you need to start developing with Paula today, including all the 80s. Thank you very much. And we want to hear from you. So if you are if you like the session this feeling the survey if you didn't like the session the survey is optional. And I said, thank you very much.

Cackle comments for the website

Buy this talk

Access to the talk “Build an AR app with the Poly Toolkit for Unity”
Available
In cart
Free
Free
Free
Free
Free
Free

Access to all the recordings of the event

Get access to all videos “2018 Google I/O”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Software development”?

You might be interested in videos from this event

September 28, 2018
Moscow
16
159
app store, apps, development, google play, mobile, soft

Similar talks

Kat Fang
Software Engineer at Google
+ 1 speaker
Kiana McNellis
Engineer at Google
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Ludo Antonov
Head of Growth Engineering at Pinterest
+ 2 speakers
Ben Galbraith
Product director for the Web Platform at Google
+ 2 speakers
Malte Ubl
Engineering lead of the AMP Project at Google
+ 2 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Dave Burke
Software Engineer at Google
+ 6 speakers
Romain Guy
Senior Staff Software Engineer at Google
+ 6 speakers
Chet Haase
Leads the Android Toolkit team at Google
+ 6 speakers
Dianne Hackborn
Product Manager at Google
+ 6 speakers
Aurash Mahbod
Director of Engineering at Google
+ 6 speakers
Tor Norbye
Tech lead for Android Studio at Google
+ 6 speakers
Stephanie Saad
Software Engineer at Google
+ 6 speakers
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “Build an AR app with the Poly Toolkit for Unity”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
558 conferences
22059 speakers
8245 hours of content