Vaas is an embedded software engineer specializing in 802.15.4. She has enabled OpenThread on several Nest products. Her key focus is on RF performance, vendor engagement and radio communication on Nest products such as Protect, Thermostat, Secure and Cams. Prior to this, she has worked at IoT startups spanning Zigbee Smart Energy, lighting control and solar panels. Vaas earned a Masters degree in EE at UC Boulder.View the profile
Rachel is a technical lead and manager on the Google Cloud Search team which delivers intelligent search to enterprise customers. She also worked on the Chrome frontend for many years and prior to Google, did visual effects simulation for Industrial Light & Magic. Rachel holds a B.S. from Brown University and a Ph.D. from Stanford University in computer science.View the profile
Brittany helps solve privacy challenges related to data access in the Security and Privacy organization. Prior to Google, Brittany worked as an IT security analyst and consultant for companies including Ernst and Young (EY) and Alert Logic, Inc.. Brittany earned her bachelor's degree in computer science with an information assurance emphasis, and her master's degree in information assurance and security from Sam Houston State University.View the profile
About the talk
A team of female Google software engineers developed two interactive demos using Android Things for the Women Techmakers event at the 2017 Grace Hopper Conference: Rosie the Robot - a fully controllable robot, that can take photos and wheel around to interact with participants; and Steve the Tree - a 20x20 foot installment that reacts to sensor input, displays light arrays, and is controlled using Firebase. This discussion panel will showcase lessons learned, how experts in software approached hardware development, and how Android Things enabled them to create the demos.
Hi all welcome cui women techmakers fanola experiences developing on Android things. My name is Melissa. I'm a program manager on here in Google and I work on the Android things team and fell in a moment. I'm going to let the wonderful and amazing women to my side introduce themselves and we're going to talk about how they're all Google Earth and how they all work together on a series of projects actually face on Android things. So over the next 40 minutes will actually walk you through their prototype to production story what they learned what they love and of course with a loads along the way.
So let's meet our lovely ladies. Let's start with Rachel Rachel Patterson, and I am obviously cool fashion how long you've been here and also fun facts to 10 years. I work on Google Cloud search, which is our Enterprise search solution and a fun fact is that before coming to Google? I used to work on visual effects for movies, and I've got a credit in Transformers. I am Carson Holgate. I am the tech lead on one of Google's internal communication tools. I'm going to Google for 5 years and a fun fact about myself is that I wrote the
software for a bartending robot. Yeah, that's really cool. I'm boss krishnamoorthy Eminem. Software engineer. It must have been there five years and fun fact about me. I've been part of this dance grooves called Bay Area flash mob even be able to answer it hundreds of places and in the San Francisco from Pier 39 to Union Square to even the mayor's Ballroom in the city hall by just mingling with the crowd and then just breaking out into a dance. So that's awesome. And then my name is pretty Wilbur. I am a privacy engineer at Google and once a fun fact about myself is that I'm a builder and
a tinker and most recently had built three 3D printers. For one up for those of you who aren't familiar at all. These projects were billed for something from Grace Hopper. So Grace Hopper is the largest world's largest gathering of women technologist that every year and each year googlers participate in this past year my team Android things printers with the women techmakers team, which is a global program for women in technology. And so I'm actually helps Works provide visibility community and
resources for women in technology to drive Innovation and participation in the field. So they do this through things like hosting events launching resources piloting new initiatives in organization and obviously teaming up with fun spins like mine. I have a little bias cuz it is my team supported in their industry us to work with him and then find a whole bunch of amazing volunteers across Google to build things. We jumped at the chance. So how did you guys get involved so I know obviously how I got involved because I
got excited and was involved with the women around sometime in July. So Grace Hopper is October around July Melissa sent out an email basically asking for volunteers to work on something for Google involving Android things. I personally have been involved with Grace Hopper at Google for many years. And so this sounded like something different ninja saying, so I was really excited about that and plus I have really been wanting to get involved and internet of things and this seems like a great way to blend those two interest and
I was trying to get tickets to Grace Hopper, but I couldn't so I want to get involved with Grace Hopper and I saw this email come from Alyssa about Android things and I have some it Android experience and some hobby Hardware experience, so I figured I'd be able to contribute. And I saw Melissa's email and I'd heard about Android things never used it. So. This is my chance to use it. And also this is my chance to meet other great women of Google which I did and so I signed up and for me, I was actually noodle Irazu Google or at
the time at so I wanted to find something that would be interesting where I can meet us at the company and just learned more. So unfortunately this came in my mailbox and I was able to take a part of it and not as well as we do not instantly knew what you wanted and everything got it going to do this. I know exactly what's going to happen, right? Come up with her. I ate and go through things. What did you build tree? Which Steve is a Giants 20-foot by 20-foot Footprints and 17 ft. Tall tree. So takes up a lot of space there Steve is a
Joshua tree. That's what he was based on and we build everything is so the tree itself got a tree bark or foam completely flame retardant Fire Marshal all good with that and then he's covered in about 5,000 LEDs and 20 some-odd sensors including Force resists of sensors microphones light sensors gesture sensors. So that attendees to the event could interact with Steve and he waved him and he reacts through his LEDs through the frequency and the color and How many events are showing
but when he started as sure we didn't start with the idea of stupid tree, he was there supposed to be a smaller part of like a larger open world interact with the things kind of game and it all converged into Steve and we incorporate a lot of the ideas with different kind of sensors and the light show in to the main tree calling him Steve. Why where did the name Stephen Hawking his name Steve? My Roomba is amazing. Is there a real steel there is a real Steve. He was a friend of mine who works at verily he's the nicest guy you'll ever meet. He
also a bartender or a tree and it's so based off of this, you know, it sounds like you were building with tons of sensors. And what does it look like? We were you guys doing here with a bunch of different sensors and Prejudice. I hadn't worked with Andrea thinks I haven't really done much with Hardware. So this actually decide that you're saying here is Mika connecting two different for different Force resists of sensors. He's an analog to digital converter. There are easier ways to do this that I
now know, but there was a lot of learning that took place in building. Steve found in the knees are all the lights that just laid out all 16 strands of 5m LEDs and roach different values so we can show the light show and make sure it all works. So it seems you're at I owe can we find him soon would have loved to make it we would have loved to have him here but he put down roots in Houston. So this is basically a bar in all this substance is basically the
construction he's pretty big. I mean that's pretty impressive in mess up. Right. So I've never felt so good about you to the Android she is a walking and talking Android. So she is controllable using a phone app using a nearby connection and on the top she has a camera that can capture a picture and it can do some image processing and it can detect facial expressions and it can detect actress in and speak to the audience. She can also tell them hey, you know, nice hat, you know, I like your smile nice
to meet you. So she would raise her right arm and offer a handshake. So how big was she she was huge ft tall including the rubberized wheels that were at the bottom. She was actually made out of the first version out of Styrofoam wood pallets and a trash can so we initially wanted to use other methods which but this was a solution that we can make in such a short amount of time. And so I heard there's a rumor that they're Rosie's here on site today. Is that correct? If it's the same version that you guys still tours is a different
version or what is that process look like from going from a base like this. I believe it's a trash can when did I just a little bit trash can. It was a lot of work. So the initial version of Rosie was going to be you know, something similar maybe not as similar design. But today it has evolved to a 5-foot tall Android that can run around that uses machine learning techniques as well as an impatient. Voss worked on as well. So it's really it's really awesome.
And then pick your Adoration of her and I ended up with a sort of the production version that we see out in the world right now. Right? I'd like to interact with her pretty awesome at her buying the trash can and yeah, it was that always happen. If it wasn't she's a trash can in to use foam. How did you how did your design on your Evolution? The first version of Rosie was actually created using a cad file. So I took about two weeks to create the cad file and then we went around to different shops to ask them. Hey, can you build a can you fabricate a 5-foot tall
Android in about a week? So and unfortunately, they said no and they said that it was a week and a half left. We decided we still want to do this. So we went to the hardware store and we went to a fabric stores and got kicked a cake box and other supplies for the head. We use PVC pipes for the arms and be used to in this giant trash can for the body. I was quite a bit of work where that come from. It's from Rosie the Riveter. So the world war individual who inspired a lot of women to decide
to go out into the working place. So we wanted to have something that a lot of visitors of Grace Hopper would be interested in understanding and have a good experience with her ml text right now right now. So you guys have all kind of touched a little bit of fun this but all these demos are built using Android things. So before you started working on Android thing, did you guys have any hardware experience Rachel? You said no or little bit not really with microcontrollers Brittany build a 3D
printer for the South but the scale of experience, but I've never built anything from scab scratch or to descale. And you guys have never used Android things. No, not at all because you also deciding that, you know, it's make roses are move or Steve's lights and all the sensors connect to an Android things card and the LED strips also connect two cards. But what we did was we had each of the cards would write to a Firebase database and they would write the
wrong values there and part of the reason we did that was because we didn't know exactly what the situation was going to be when we got to the women techmakers event. So for example, the four sensors, we knew that those were going to have grass over the top of them and we didn't know how much that would affect the resistance in for the sound we had no idea how loud people if you were going to shouting at him or if they're Going to be background music. So we accounted for that but having everything right to the database so we could then modulate those values once we got there. So the hunters
would write to the Firebase database when we had Cloud functions that would translate those Rob values into an advocate that the tree the individual strands of LED is Madrid from and used to determine like how fast they're going to want color spectrum to run through that way during the events if it got louder or like something happened with the four sensors on the floor. We could change the code on the fly on the server and not have to go around to every board and we flash them. Great time for Rosie. I started just by downloading Android studio trying out a sample Ave. And then I started
looking on GitHub for samples that I could potentially use on Rosie and I found this perfect one for the movement of the robot, which was it called the robot car. It's on GitHub. If you look for it, you'll find it. So this is basically if he gives you instructions on how to compile how to load the app and build a small race car kit. So I started with just building this run trophy race car kit and while debugging it and what story was great because once you plug in your phone and your iMac 70 development board, as you can see in Tupelo tabs, you can see the log. So if you can leave the
connection that way so once I got the small robot car working, I just filled up too much bigger wheels and much bigger tire torque Motors that we needed to carry the weight of the robot and similarly for the abortion with the image recognition did the same thing I am Look for things that I could use with a kid that it comes with an LCD screen and a camera and watch it was so I went to get out and look for a sample that would let me use all this and again, I found this has attacked her a sample that was out there and I was great and I replicated the same thing and pulled it in and
changed a bunch of things to get it working on a Rosie and for the are moving decimals. There's a simple pwm sample. That's an Android things and yeah put all these different pieces together their were obviously a lot of talents just for all of them to work together for example, and I'm going to stand up to the bigger Motors Hyde Park Motors now, they needed that much more bigger battery and then that was operating on a different voltage it was operating on How much higher current so I have to buy a huge LiPo battery and buttons explorer was to the power of a 5 x so a lot of learning
they're definitely a bunch of burns Motors in the court in the end. Everything kind of came together boss did all the hard part. I just know I don't think so. Yeah that's like standard was on software engineer. Not at all. Yes, so I sent any of this actually translate over into the work that you do every day. I hope it does not So Much Anymore. Like I said my privacy Engineers, I work in security and privacy. So most of my time is set in the screen. So having the opportunity to have
be able to do something on the side that is I guess word spacing is really awesome. The film was actually just getting started with everything isn't it pokes around a little bit and they have a with media Raspberry Pi or a microcontroller or literally just have an idea I dream in a prayer right now. How do they get started? What it was you recommend a jump on me and they walk you through setting up the whole Android things. So if you get a kid and I think there's a way to get a kit while you're here at the
labs will walk you through the entire setup process and by the end of them. I know there's one that uses tensorflow to take pictures and identify dogs. And that one's really cool. But the important thing is if I the end of it you've you've gone through the experience and basically done the hard part of learning how to get stuff on Android things and make it Things and so once you jump over that hurdle actually making changes and doing stuff and I just needs for what you want gets really easy and I would say you
don't even really need Raspberry Pi or microcontroller experience if you know a little Android if no little Java it's easier to get up and started and the kids come with a rainbow hat which has enough sensors to build something cool right out of the box and you don't need Hardware or inverted experience either all the drivers for Android is hidden behind the screen apis. So if you want to just control the motor all you say is you set the direction and speed and that's it. So you're not rewriting all the device drivers from scratch to get the various companies working.
A lot of people when they look at, you know, programming opportunities. There can be a concern. Hey, I don't know what I'm doing, but the Android things Community is a great resource and then also, you know just learning about it and you know taking part and getting Hands-On is a great white. Get started and so that you didn't need any help building your change into any problems at all. Right. That went wrong. Like I said, we were planning on building Rosie version 1 as a 3D are from 3D printers
using fabrication techniques to do. So, unfortunately, it was really big I didn't realize that you don't have you know, something like two foot diameter. You don't waste and all this other components could cause such a concern so about two weeks before after exhausting every single off. And made available to us and we just said, you know, hey, we're just going to just be Scrappy and get stuff done. So that's where the funeral going around a million hardware stores.
Pretty I feel like that's an accomplishment. That's pretty impressive right to like at moonshot the printers to move forward you immediately your next steps were to think about the same structure and redesign and then think about how the materiality came into play with the software. Is that correct? That's right b a start off with something so we should for example, we got PVC pipes for the arms, but later when we put the motor on it be found that that was getting stuck and the
PVC pipes were just too heavy so we had to come up with an Definition of suspend Britney found these perfect forms that we could use instead and there a lot of other things to ask. For example, when did put Lucy together even on these bigger Wheels the wait was too much for the wheels to handle because they're These boots were designed for a fast race car. And so they couldn't take the weight off the trash can so we had to cut pieces of plastic out. So it can I carry the weight and a lot of things even with a camera that we have to have enough lighting to get the exact expression and
we have to go to the right angle to every single step. We have you ever heard a lot. We get rated and came up with some solution for everything. Sounds like a lot of a lot of time to get to it. So obviously you I'm guessing you had a ton of time to feel that. It's right. How long did it take for you guys to build Rosie? You know, we was we had a month initially but once the 3D printing option failed it was about a week and a half and I guess more accurately more like 3 days of late night trying to figure out what we
were going to do when solving it that's not much time and have it for Steve. CVS photo from the center site. I think you put most of that together in about 2 to 3 weeks. I mean hypothetically ideally we would have had more time. We would have started immediately when you send out the email but there's a lot of figuring out what we were actually going to do with Steve and Steve became Steve and you guys needed things out like that was a sufficient amount of time or you both needed a lot more time or I think we would have needed two
months. At least for time will we have like two weeks trying scramble, but the last three days were just insane like we were so the truck was waiting for us. He was telling parts. We were still working on it and then like talking things that were done and the head I had to carry the head with me in the flight was for the rest of it was already shipped. So I carried the head of the robot. It's crazy. And so when you're building Steve's a tree at 25th Street, there's nothing went wrong there. Right? So everything was perfect as perfect
designs the main Tree in while I was resigning the main tree. I made a lot of assumptions about how the hardware in the software would work. And of course most of them are wrong initially. I thought we were going to go to control Forest Ranger LED from each board and then I got some of these strands of LEDs and I got some of the boards and I realize that this is not going to work the way does this the LED strands had implemented the standard method I couldn't do the things that I thought I could do. So I looked at so the problem was that the board's only have one miso pin to do
that the data riding to the to the Strand so it looks at using a technique called bit banging where I pick the part that had Arisen the standard and I wrote just the individual bits out through gpio pin to light up the Strand and do the light show, but I got about 3 seconds per frame. Where is Doing it the right way to stay at a pen. I got 30 to 60 frames per second. So I reached out to the Android things community and I'm like, this is what I'm trying to do. This is what I'm expecting to happen in. This is actually happening and I hear back if there's a way of traction between a gpio pins in
the software for security reasons, of course, so it wasn't going to work the way that I wanted it to and this is like a common a common saying with operating system based boards fit banging right technique that I used to write. So then conveniently we were able to get a bunch more boards and the base of Steve a tree as you saw him. Pretty good photos has 16 deck boards. Just kind of like sitting around the base each Board of controlling one's kind of LEDs. It worked out in the end. Yeah, and then another thing that we ran into so I mentioned before that we're
writing everything to the Firebase database so that we could make sure that we can modulate when we got to Grace Hopper to account for the difference environment. Well, we didn't really account for the fact that we had 20 sensors writing about every second to the Firebase database and we had 16 strips of LED is reading from it was a lot of in and out to the Firebase database. It was fine for a while until the night before the events in the whole tree was set up in the sensors are ready to go or just testing things and then the DJ got there to test her set up to sling a
Bruno Mars song with a lot of bass and because the tree has microphones on it. It was reacting is like dancing along to the music then suddenly the whole tree stops working. So what we scramble to figure out what's going on, like we check connections and eventually I open the Firebase database logs and I see that we're out of free quota like some point. I'm just so happy. It happened the night before the event and not during the event of some of us and I start scrambling trying to figure out how to fix Kodi issues like going through official
channels and eventually reach out to someone on the side facing before because my my my real Project X is Firebase and he's able to text Dakota problems and we're up and running again and this whole laser light fireworks show until later in the evening. The fireworks are going off and Steve is going with the fireworks and it's kind of like a nice sign that okay going to be alright. What does this mean that you guys didn't account for noise or is it just because it's also mean that you would just more successful than you
anticipated for something at that size. Right? I think it's a good take away that it's like when you're starting out it's really hard to think about something at Giant magnitude and stuff. And so, you know, you're able to kind of building and iterate. So so it sounds like between all of you guys and your software experience and background is Firebase actually helps with learning a little bit about Android things and troubleshooting some of the problems that you ran into is that an accurate statement. You think it made it a little bit easier to get
experience helps especially with an android-based platform and the Firebase stuff was great because usually when you're working with the hardware used to figure out how the boys are going to communicate like is it going to be Bluetooth is going to be like being one of those things but because this is Android and all of the Android libraries in ecosystems come along with this we could just write to Firebase. We didn't have to worry about like the inner board communication mostly for the
first time and so Andrew was a very comfortable environment for me so I could focus on the internet of things and I wasn't learning 16 different things at once just focus on the one I needed to Had no experience with Android. So this was all in you everything everything was from scratch but having a hardware experience helped definitely. Like I said, this isn't our software engineering isn't directly my day-to-day job, but, you know looking up the Android community and then having you know co-workers who have experience has been really great and with his grace.
So what was something that you guys took away from this or is there anything that you felt like you've learned from this? Hopefully you learned something. And so I learned that overnight shipping is really awesome. Sarah definitely a lot of things that I ran into work and the sensor for example realizing that I need an analog-to-digital converters the night before we had to leave and doing some late-night soldering so getting those overnighted was awesome. So yeah, that's my favorite. I
learned just to ask for things when I started designing the tree. I looked for the open-source drivers that were available and the kinds of LED so we could use and I found these five strands of LED. Okay. We're building a tree we want you to get to the basement to the branch. You should probably start about 6 feet off the ground. So five meters is -6 feet is about 10 feet. So then suddenly we have this 20-foot by 20-foot footprint and unlike this is insane, but I kept going to so I'd have the design. I can be like look at this thing that we could have built that would have been awesome and
at the end I put in a couple minutes, but how we could like make it smaller. I cut it in half cuz I didn't think it would fit within our budget. Event space and this is fine. We can do this. When I signed up for this project, I had no idea what was coming. I didn't even know we were going to go there or what we had a lot of ideas. He was rolling around figuring out in school suicide up but everything seemed doable to a certain extent until the 3D printing fell through and at that point
we weren't sure whether we should give up or keep going and then just built it from scratch herself. So that was a really really scary and but we didn't give up we just kept going and going for stood and finally we said, okay, let's get one person working and then doesn't want the other one. But in the end everything came together every all the electrical circuits worked perfectly and she looked beautiful like a pretty Android and everything was going so might take away is that they all have the confidence in yourself believe in yourself and keep trying it will pay off for me. I know this
is cliche, but don't be afraid of failure on like bosses mentioning we Had a lot of iterations we were first going to use a brother are well-known license character from a large company, but then we evolved from that to Android and then the 3D printing failing and you know, just a lot of things in between that didn't come out as the way that we initially thought. But ultimately we use those failures as a way to learn and understand what was going on and improve and we were able to get Austin results after that
failures. I think they're just like growing pains constant iteration. Since then from that I can't let you guys go without asking. What's the next thing you're playing to build using Android thing. So when I'm not a software engineer or normally build Andre things I do a lot of dancing and so I'm really it's actually now that I know that the microphones worked really well with sound and hoping to incorporate some Android things pops into some of the dance performances that I do. And I'm looking at how
to add Android thinks my bartending robot so that when someone walks up with a glass it'll trigger the assistant integration and be like, hello. What are you drinking today? They spend I'm planning to build a Halloween project for my kids and the Kids on the Block to probably be up candy dispenser which has a bunch of pressure sensors and proximity sensor. So when kids walk up Twitter to dispense candy for them for me. I have recently learned how to use a sewing machine. I was know you had a hand in it, but I was always scared the machine. So my goal is to hopefully integrate Andre things
into some of the textile work then working on crate and then move on now will definitely do the scavenger hunt. Cool Anthony and how about those kids are alright. Thank you so much. And also thank you so much for building. The amazing projects do guys that and working with our team and player bio Pages for both Steve and Rosie. So if you want more details if you want to be able to season these pictures up close and personal on really know hopefully complements our work not to analyze it too much that you definitely take a look at those and check them out. It's also a few questions about it's a
great way to post inspections and hopefully we can answer them for you and none of us would even know each other. So yeah, thank you and she's always there for us. She's always there to support us and Rihanna. About to go crazy. So happy your hair.
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.