Duration 34:56
16+
Play
Video

AR apps: build, iterate, and launch

Tom Salter
Tech Lead at Google
  • Video
  • Table of contents
  • Video
2018 Google I/O
May 9, 2018, Mountain View, USA
2018 Google I/O
Video
AR apps: build, iterate, and launch
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
14.77 K
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speaker

Tom Salter
Tech Lead at Google

Tom Salter is a technical lead working on Google's ARCore platform. He spends his time optimizing the underlying ARCore algorithms, making sure as many Android devices as possible can understand our world. Previously, Tom led the Augmented and Virtual Reality developer tools team, and has been working on AR and VR platforms for nearly a decade.

View the profile

About the talk

Developing in 3D for the first time? This session covers the tools that will help you build, iterate, debug, and launch your AR experiences. Learn how Sceneform SDK enables you to construct and render 3D scenes with ease, and how to integrate Poly assets and get familiar with GAPID, helping you understand your apps rendering performance in detail.

Share

Welcome my to the third day of Ayo, it's great to be here talking to you this morning. My name is Tim psyche and I'm a software engineer on the AR & VR team today. Tom and I are going to talk to you about the tools that we felt to make a our development easy and fast for every Android developer. Said you heard that the developer keynote arcore is Google's platform for building augmented reality experiences. You've been hearing a lot about arcore the last couple of days and I hope you guys are excited about building for a r I certainly am our goal is that

at the end of this session every single one of you would feel like you can build an AR app. Don't we're going to cover today. It's how you can get an asset. How you can build an app using that asset and finally how you can iterate in launch? Stop before I get into the details. I want to tell you a little bit about something that I like to do. So I really like to time myself doing ordinary things. I'll time myself doing things like iron in a shirt or even taking a shower

or read. My kids a book on its kind of quirky. I know but I like to optimize my time and do these things as fast as possible. Sometimes my kids don't appreciate it that much. I'm so what I want to do today is I wanted to do something and they are as fast as I possibly can but doing something that I call. AR done fast, that's right. I'm going to do a speed run through building an AR app and tell him I'm wondering if you can help time this Speed Run and see just how fast I can build a nap and they are

not to make this happen as fast as possible. I'm going to use the same form SDK, which is our new 3D framework that we just announced in launched on and this framework makes are development easy and Java so so that I can see what's going on. I did one take video recording of developing an AR app last night and we'll see how I did let's jump the next slide and Tom. Are you ready ready? We'll see. All right. So let's go start. All right, so I'm getting my ass it from Paulie. Google.com to make this as Illuminating as possible. I'm going to search for a lamp post will see

what I find. Okay. So this first lamp post from Naomi Chen looks pretty good. Let's have a closer look and see if I like that this is pretty much exactly what I'm looking for. So I'm going to download the model file and as you can see here, I've got it downloaded and I've got all the files here in my downloads folder. Know what I'm going to do is I'm going to open the Holocene form sample app, and I'm going to see if I can get this lamp post being used instead of Andy know what I'm going to do is I'm going to drag all of the model files into my sample data folder and then using our

just-released seen form tools Android Studio plugin. I'm going to import that asset into my project as you can see as soon as I click finish. We've got a brittle build going that's doing the import of our asset into our project in order to see it pop up in the viewer in just a second to see if we like what we got. There it is. So that's pretty much exactly like it did on poly. And so let's see if we can get that added in place of Andy. So we're going to jump into the code here. And as you can see Andy is loaded using a resource ID. Now, you might have noticed. I actually

put this file into the assets folder. So I'm going to change a little bit how I load it. I'm going to load the file from the assets folder going to drop that in on and I'm loading the file now and I press play almost play. Okay done. Alright, hang on with the time. Let's see. Let's see what this looks like back to the slides. So let's let me go. Okay, so here's our house. So we're done. We have an app that's running in AR we've got a lamp-post we can place our Lamppost. We

can place a lot of Lamp Post in our office and make our office is decorated with lamp post as we would like it to be honest looking great. And so that was our Speed Run. And so let's see Tom how how fat how did I do there 1 minute 45 seconds 1 minute 45 seconds that's minutes. Not hours. Okay, so that's how fast we can build with a r. But now we're going to slow down and we're going to talk through all the tools that we feel that make this possible. Send it. First thing I'm going to do is talk to

you about assets. I'm you can get your assets that you use in an AR app from many different places. You can Source them online. You could have an artist create custom assets for you on but what I'm going to do for this talk is that noise? We're going to get our asset from Polly now Polly is Google library for 3D assets. I'm poly you'll find thousands of 3D assets that you can use in your apps as you thought I was able to easily search for a Lamppost and find exactly what I was looking for. But if you want something else like a tree you can

search for that and there are lots of trees available in all sorts of colors shapes sizes. There's some maybe some dangerous looking ones whatever you want. You'll find what you're looking for. But if you don't know exactly what you want and you just like to be inspired you can Browse by category maybe you'll find an asset there that will inspire you to create a whole new AR app. So have a look around and see if you can find something that you like and I think you will on most of these assets or many of these assets are licensed under the the the CCB why and so

that means that you can download model files for these and use them directly in your ass. All you have to do is credit the author and then you can use them. So here I'm downloading the model file and look at us. I have an asset as you saw on the speedrun. Puritas Now if you look closely you'll see that this asset has three different files. Now you want to make sure that you keep all of these files together. Cuz each one of these files is important to actually be able to render this asset the way we want to but what do we do with these files

now that we have them? Just for fun. What's crack one of them open in Android studio and see what it looks like. So this doesn't really look that much like a Lamppost maybe a little bit but you probably were guessing that this wasn't going to look like a Lamppost if we just open up the raw files on as you can see, we've got an image. We've got some a text which is showing what all the vertices are in this object. You might you might love a file like this if you're a graphics developer button in and if you were using a game engine like Unity, it would know

exactly what to do with this should be able to run to this you'd be able to speak it and be great. And I actually want to stop for a second and let you know that at 11:30 right here and Stage 7 or of course streaming on YouTube. There's another set session building for Unity and you can learn a lot more about Paulie and more about using his assets in unity. But for now because we're not building a unit that we're going to build a native Java app. We want to see how we can use these Assets in Android Studio. So,

let me talk to you about seeing form. So we can work with 3D models in Android Studio using team form or using the scene Form Tools plug-in that we just announced and is available now on we announce it at the developer keynote on Tuesday. So scene Farm is our high-level 3D framework and it makes building AR apps and Java easy. Now seeing form includes the runtime API and that handles all of your rendering at handles on doing things with arcore on and making your app work on the Android side. But seeing for him also has a set of tools that interact with Android studio and

let you import View Inn Suites your assets right in your ID. So, where do you get this plugin? Well, like most other Android Studio plugins, you can get your plug-in right from the Android Studio plugins page you go to preferences plugins browser plugins and just search for seeing for him and you'll find it. So once we have the plug-in, it's super easy to get your assets imported. The first thing we need to do is place our source files in our project. We're going to drop them in the sample data folder. Now. This is a little bit important because

the sample data folder is a folder that does not get bundled into your APK. Now you might wonder why would I want these windows are these these these files not bundled into my APK? Well, as you can see here what actually hang on I missed one thing on his use Kenzie. Are we support obj fbx and gltf files until we're going to focus more on obj files right now because that's what I got from Polly. But this exact same flow works with FX ngltf what you might find other places online or on or or Source from

things like that. But anyway, so we drop these files into our sample data folder. I like I said because we don't want them bundled into our app. Why don't we want them bundled into our app? It's because we build runtime optimized binaries that scene form will be able to use to render your assets in Android. Now these binaries are off my slave load fast, and they look great the include all of the data that you need in order to render are in order for seeing form to render your models. I'm so that's why I'll actually can you go back one

slide? So is that is the SSC file until you're in here a little bit more about SSB files later, but just suffice it to say sfv files are the files that we want bundled into RAR APK. Okay. So now that we have our file in our set of files in our sample data folder. We're going to right click and using the same Form Tools plug-in look like import scene for Masa. This will trigger our wizard our import wizard flow. So this is the import wizard as you can see this is fairly simple on

we have three fields to fill out. The first is a file path door model and that's already filled out for us. The second two are outputs in both of these files are created by the import wizard. The first file you'll see is an SFA file and I'm going to go into that in detail and just a couple of minutes but this for now you just need to know this should be placed right next to your model because this is going to become one of the source files that will Duvall show how your model is rendered. Your SFP file is what you want bundled into your APK. And so you're going to want to put that

somewhere that gets bundled you can I have two options here are your options are your assets folder and your restaurant folder? I'm you can put in in either of these the loading is slightly different when you get to the run time, but other than that, they both get bundled and it doesn't matter you can pick whichever one you choose. So that's how we have to do in the import wizard. We click finish and we're done. Let's see what happens next. So you can see as soon as we click finish. We're going to have a Gradle build tickenoff. We're compiling hear the

sfb that we just imported. Now what's going to happen is we'll get these battle pass added were picking these two faults that we're going to fill in in the SFA, but I'm going to tell you about in a second and we're building this run time optimize binary. But as soon as we finish this build our sorry, as soon as we finish this field, we're going to see these as Fass the files added to our project and so you can see we've got our SFA next to our model in R. Sfv right here. I think I have it in restaurant right here. But that's not all we also have a viewer and so

the viewer is going to pop open as soon as you get your ass at fully imported. And as you can see we see exactly what our model is going to look like. You're here. It looks just like it did on poly which makes sense cuz that's what we implemented. That's what we we imported and you don't have to deploy to your phone. You can just see it right away. You don't even you don't even have to have an app built yet. You could just import it right into your project in View files without having your app built on and this uses the exact same render as a run time. And so what you

see is what you get this is going to look just the same on your phone. Okay, so this looks just like it did Nepali but I was thinking I wanted it may be a little bit different for my app on in poly. It sort of looks maybe like it's plastic maybe on and I was hoping it was left maybe a little bit shinier. I'm so let's see how we could do that. So I talked to you about the SSA a couple of times. So there was a text file that you saw in the viewer. And so that file is your SSI file. Beyonce's file defines housing form

will render your asset. You can see it has a bunch of different parameters. And if you look closely you'll see that these might affect the the way your asset is going to look on. So we're going to look at a couple of these one of them here is metallic and one of them is roughness and so by default when we import from poly we get something or the roughness has one in the metallic is zero and that's going to make it look like it's sort of made out of plastic. And what I like to do is turn the Metallica up maybe to one so it looks really metallic on and turn the

roughness down. Maybe not all the way to zero. I don't want it to shiny, but I'm going to turn it down and then I'll see what what happens. Does soon as I save the SFA file we're going to rebuild the runtime bundle. And as soon as that is complete, we're going to see that reflected in the viewer. So here you can say I guess this GIF is sort of looping a little bit but you can see once it completes. The viewer is looking a little bit different. It's looking pretty much talent and this is basically what I'm doing for. So I want to I want to point you to

this documentation online. So I mentioned a couple of the parameters and there are actually a lot more parameters that you can edit in your SSI file. There are a lot of foreigners that affect the look and feel of your ass over there also parameters that affect things like it's scale or the way, it's Collision Works for when you're tapping on your on your object. And if you check out this page, you'll see all of the details on all of those parameters now one thing I'll mention I mentioned how we support gltf fbx in obj files. When you import each of these files the

parameters will be slightly different that you can edit in your SSA and all of the details are in the on this site. So I would strongly recommend you go see which things you can tweak in each of these types of files. Okay, so we've got our ass at look in the way we wanted to let's see what we need to do to get it in a are now you remember I talked about placing the the runtime optimize binary the sfv in the assets or restaurant folder so let's look at how we would load that in our run time out of each of these as you can see here this

is an activity that you guys are all very familiar with on we haven't on create method and what we want to do is our model renderable is an object that in the run time is going to describe all the details of how to render our lamp post You can think of this as the runtime representation of our file that has our optimized on our optimize model and see what we need to do is we need to load our file see the run time here and we're using that using the model renderable builder now loading to take a little time. So if this API is an asynchronous API

is create a builder Source setting it to a restaurant or a resource ID in restaurant that points the lamp post in build. Now the model render will Builder returns a completablefuture. And so that completablefuture we're using this done except method so that after that's complete we can we can go and store the renderable. That's right. That's loaded into our lamp post Runner Bowl so that it can be used later. I'm so this was for loading out of restaurant on if we were to load out of assets. It's just slightly different. This

code is almost exactly the same except for one line when we set the source ID. We just need to set it to a URI and parse out the actual file name instead of a resource ID. You'll notice this is just the lamp post. Sfb directly in the assets folder. This is how we would load that you can define a directory hierarchy. However, you please and just a reference the pass all the way from the assets folder to your SSD right here. So there's a lot more to learn about seeing form in about how to actually display these on in your app on when I was doing my

speed run. I used the sample. And so that's a great way to just try this out and see how your assets will look. But if you like to learn more about how to do more things with these assets and how to display them in the run time. You should check out this session. It was yesterday a rendering for Android AR apps and it's available on YouTube. Of course, you can learn a lot more there. So I'm going to jump right into our actual app we built and let's take it for a spin see what it looks like. Now that we've reached the parameters in the SFA. All right. So there is our lamp post. It's lit

nicely. It's got everything that you would expect of a giant Lamp Post in the middle of your office and on and it's it's looking great at that's just what I was going for. So let's jump back to the slides now. and on So I want to talk about one other thing, which is I'm asking you to install this Android Studio plugin. You might wonder what it what is this plug-in going to do to my project to build. Do I want to install this until let me tell you a little bit about what's going on under the hood. So we've built

a steam form Gradle plug-in and this is what it actually doing the asset bills that I showed you. When you import on what we're going to do is we're going to add this scene Forum Gradle plug-in to your Gradle build on. The first thing we do is we just add the classpass dependencies to your project build file. You'll know that we load this from the Google Maven repository. So you want to make sure that that's in your project build file. Of course. This is the default for all new Android apps, so that shouldn't be a problem. And then in your app

build great Bill. Gradle file. We apply the plug-in just once we won't do that more than once but then for every asset that you import you'll see one of these asset definitions in your file. Now you'll see there a few past there. And if you remember the import flow, you'll remember that those look very familiar on an impact to write. Those are the exact same time that we had in the important wizard. This is all the import wizard. Does it writes these files into great also, could you edit these files manually if you wanted sure that would be fine. Now, we think that the Android Studio plugin

is the quickest and easiest way to get these in but of course they can be edited and changed here as well. I'm in finally once you build and you've got these asset rules in your project. You'll see that here. I am building an APK using the assemble debug Rule and Gretel. And what you see is we're getting this compiled asset task and we have a compiled a set task for each of the assets that we've added to our Gradle build and these are added to the dependency change that will run before the task that merge in assets are resources into your APK. And so

you can always be sure that these bundles will be updated and optimize binaries will be updated and ready to go at every time you build an APK and you'll always get the latest and greatest version of the optimize binary that has all your updated parameters and that should be that should Yes, sorry. I forgot about Source control. So no one thing you might ask. Since we're building these files and part of cradle and do we need to check them into Source control. Where should they live on in the truth is you could

you you want don't have to check in the Optimus Prime Time Rentals. This could just be built as part of your Gradle build, but we would actually highly recommend that you do text them in the sense. These are tracked in Gradle and we understand all of the dependencies and the the input files that were used to generate are optimized binary. We can always understand when it needs to be rebuilt until he just checking your SF. He's checking your sfa's and on and then pull down the latest Source from your Source control and build

will know which we have to rebuild in which we can leave unchanged. And so you always have the latest stuff to run. Okay. So now we have our basic half and I'm going to turn it over to Tom and he's going to tell us how after we feel a nap. We would test it irate and launch that app. Cool. Thanks him. Hi everyone. I'm home. I'm engineering manager on the arnp all team. I'm going to talk about some of the tools that we fail to make its writing and building in D. C Venezia Festival earlier this year. We released alcohol supports the Android emulator.

So no matter where you are always able to build for alcohol. So if you'd like to develop your vocal app without having to deploy to your device get up and walk around jewelry room place or object. Then you can do that. If you would like to develop coffee shelf or on the bus or on the plane then we thought you come in. The virtual environment that we let you walk around then it's got multiple rooms and stuff tables at stop Chas and if you want to see what it looks like with a cat in your room, then you can do that, too. Third to

get started you need to do is use Android Studio 3.1 overbuff. Use the latest on the way to Oreo system image, make sure you use an image that has access to the Play Store. You can download a real cool or there are restrictions on a website to sideload if you'd like and then it should be for the back facing camera once you settle out Goods guard. So walking around your room. It's not as easy as holding your phone open looking around. But if you play Love video games like me than you'll find it

really really natural. So you use wasd on your roll on the mouse or trackpad and you can navigate like a video game sale call tracking stock. We don't say anything on that rent them itch we just run the full tracking stack, which is really cool because as new features real cool come online, then it will just look at me like that. So this week in the keynote we announced to pull for augmented images and wassall that stuff works. We don't actually have any images

in the environment just now if you get the canary build of Android Studio with the emulates of right now, then we bought at the sex with you. Why which lets you place an image in your world? You can change the size of it. You can put on a horizontal surface. Is vertical stripes is an issue can see we didn't have to do anything to make it work. So what helps you with the right one thing that works really hard on his helping you Depot control application. So I want to show you a tool that we've

been working on the helps building and Depot getting these complex seems really easy and unity and Unreal Engine Inspirations. We made building a really easy, but sometimes you want to go and see what's happening. So we don't get it, which is the graphics API tebogo. So it's a really powerful tool all of the graphics API that your application is lets you replay them lights are on searching to see exactly how your frame is being built draw cool, but your uncle or see

what resource is your app is being used or investigate why you ask doesn't look quite as you expect then you can help you understand exactly what your GPU is being asked to do as well. So here's a really really quick example of which of stepping through Djokovic will call you can see the Auntie modal the full resources of this is using and you can see every single GL drawer element school which contributes steel final image and you can see it built up Djokovic Oracle. So we can I go for a quick

will went all through Cafe it. There are two main stages. The first stage is tracing which is capturing every single API cool that your wife is making his replay which is on your desktop. So that you can step through and reap what you sow said she was at your house. Surf Festival to get going. So file capture Trace. You have to choose your device choose your package making sure your packages Depot and then it will connect to your device start your package and then it will

start streaming this information back. This is the you why that you got and you can replay each frame with indefinite and step to ridgewick electrical. Full opengl es applications which most Meralco applications. All right. Now, we actually capture from the start the application all the way through until you pick still use the film the top to choose between the frame that you'd like to take a deeper. Look out. Once you found an interesting frame, you can really dig into the details on the left with you. So that shows every single come out

the door frame. It's neatly separated by frame and watch Roku on any of the promises to those functions can be changed and you can replay it just to see how that might make a difference in the sense of we've got a friend before pain. So that's up to the current command and that way you can step through every single commands to specifically talk. You can bring it more into the light and you can change those promises. I don't have right there various types for your resources. So you can see the textures

that come to be a seal application and you can see every man an investigative polygon count. You can see Ashley Gibson number of seats in triangles so you can take a look that if you also want to do all the shade has the building meals to eat. Opengl State you can go and look at that too. Let's take a look up the concrete issue the kind of thing. You might see within Gap it. So this is like sample that I created from all Seattle office. This is just an empty hanging out on a big chess set. It looks like just one or two Androids just hanging out there but really common issue on

mobile applications is open troll. So ovitrol is when you shade a pixel multiple-time stop become become a performance bond neck and with limited mobile devices. So you generally 100 pixels on a saucer. How many times do we think the those pixels left run time for me to actually get renders well. 1 2 3 4 5 6 7 keep stirring the young like check your assumptions on how your frame is actually being built up with unity and unreal exactly what that doing. Football to my site you could rent it out your objects front to back instead of Back to Front

drop location and going to the hood poke around and see just check your assumptions. I'm always surprised every time I go in so I could capture. So there is a lot more features within cap is going to take a look at the get home to IR link and we have a lot of tutorials and guides on there and how you can do extra things Daydream and vanilla Android apps as well as well as the vulkan API so can become popular. So what's your opinion be booked and it's looking exactly as you want your next step is to get out into the world. So we'll talk a little bit about that now. So is a

are there are two types of applications that you should think about when you're developing your I are out this quiet so you don't support arcore, but you have to go and test for alcohol support if you want to create an elf call session. sir For those devices for applications. The onus is on used to going right that codes that checks to see if they are poor is installed and also check to see if the device is alcohol compatible looks like right now. So we have a very simple

availability aqi you go in cool check availability and you wait until that if possible. Toyoco, then you can go enable UI which it takes you into you lay off modafinil session. But before you go to check if you have support, you should check with a l core is actually installed on up there. We basically have the simple API that is a request in a request that can tell you whether its installed if it's not installed, it can go in and stole in the background for you.

Let's look for a are required applications that really really simple. There's really nothing to do a we going to the whitelist thing on the Play Store for you. So it will only be available on applicant on devices that have a local support. These devices will just be whitelisted automatically. So we just before we finish up we would love to get feedback from you on how this on this session. What does a recap we've shown you how to build your AR applications really really quickly with synfilm using assets from poly. We've shown you how to write

quickly and even if you don't have a device you can go and build a outdoor applications right now. We've shown you how to do an ollie graphical issues in Cafe.

Cackle comments for the website

Buy this talk

Access to the talk “AR apps: build, iterate, and launch”
Available
In cart
Free
Free
Free
Free
Free
Free

Access to all the recordings of the event

Get access to all videos “2018 Google I/O”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “Software development”?

You might be interested in videos from this event

September 28, 2018
Moscow
16
159
app store, apps, development, google play, mobile, soft

Similar talks

James Birney
Product Manager at Google
+ 1 speaker
Eitan Marder-Eppstein
Engineering Manager at Google
+ 1 speaker
Available
In cart
Free
Free
Free
Free
Free
Free
Ibrahim Karahan
Partner developer advocate at Google
+ 2 speakers
Pierre Lecesne
Senior Software Engineer at Google
+ 2 speakers
Anthony Morris
Software Engineer at Google
+ 2 speakers
Available
In cart
Free
Free
Free
Free
Free
Free
Fred Chung
Developer Platform Manager at Google
+ 2 speakers
Dan Galpin
Team Lead at Google
+ 2 speakers
Eric Kuxhausen
Software Engineer at Google
+ 2 speakers
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “AR apps: build, iterate, and launch”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
558 conferences
22059 speakers
8190 hours of content