Jonathan designs and builds tools to make developers more productive with a focus on testing on the Android platform. Before that he worked on several Android applications such as Google Maps and Google Pay. He has a Masters degree in Computer Science.View the profile
Stefan has over 16 years of experience in software development. He holds a master's degree in computer science and a bachelor's degree in electrical engineering. Currently, Stefan is the tech lead of Google's internal Android build rules.View the profile
About the talk
There are many testing tools available for Android, and selecting the right tool can be confusing. This session will showcase the Android Testing Support Library (ATSL) — a new set of testing APIs that allow developers to write tests of all sizes across different execution environments. These new APIs will make testing easy, reducing the cognitive load for developers and keeping them in the zone while rapidly iterating.
My name is Jonathan and I work on the mobile ninjas wear team wouldn't Google passionate about testing contestability? If you have a written test for Android before it's likely you've used some of our products espresso robolectric these tools combine two billion testing locations both within and outside of Google. So today does a general consensus within the software development community of the virtues of writing test. Sure. Does it cost to writing test? What
is an accepted one throughout the life cycle of your project? Test provide fast feedback on failures like a bug caught early on in development a spot cheaper to fix then after you deploy your application. They give you a safety net for making changes geocode you free to refactor clean up and optimize safe in the knowledge that you're not going to break any of the existing functionality. I want small a suite of readable test provide the living breathing specification your applications Behavior.
Now in software development resist the concept of the testing pyramid. It's made up of three layers units integration and end-to-end test Fidelity. But this comes at the expense of test execution time. It also gets harder to maintain a debug these kinds of tests. But just like the rock bands will you need to Perfect Blend of musicians to create that great track each layer in the pyramid is equally important you leveraging the advantages of one layer to compensate for trade-offs in others to produce a holistic automated testing
environment. We recommend the 70-20-10 splits as a general healthy guideline. I know why the rules of these pyramids still apply some of the unique characteristics of Android development have introduced some difficulties along the way. So unit tests do 10 need to be fast run on the local Workstation. An integration management test juice. I need to run in a really faithful environment tend to run on a rail or a virtual device. I'm so separate tools revolved Beach
lever the pyramid. Robolectric all the multiple Android framework for your off device unit tests. espresso and Android testing support Library photos on device tests I'm sure it's got some really familiar call Concepts such as getting a handle to your application context or maybe driving your activity lifecycle. And each of these tools has its own distinct apis and ways of doing things for cheating these exact same tasks. Now this is like two something of a test writing crisis as a
developer. It's hard sometimes to know what tools are available for use. And which of those are recommended. Having multiple tools at each level has led to an explosion of styles each with their own distinct patents and apis. And this in turn leads to lack of Mobility between layers. Toscani's leave infected or reuse between Lesnar pyramid without being completely Rewritten from you tool. To discuss it further that's orientate ourselves with what constitutes a well-structured test. So there's some
following common patents that Define a well-structured test. And so we generally brake test on if it's replay sections given when and then and I like to separate them with a blank line to clearly demarcated them. So given some predetermined state of a system. when you execute an action that you wish to test been verified the new site to the system or some Behavior has occurred. The short name the test after both the condition you're testing. I'm the expected outcome. Keep a test focused on very specific behavior and
then test only if you behave independently. Unless it's always more with tests. These guidelines will help you keep each test understandable in isolation. Use cumin setup methods only for scaffolding. This is maybe creating the object to the sound a test and wiring up some expenses. But take a look at the problem of the explosion of styles. I'm going to highlight this using a simple test case. Will have a single activity with one button that responded to a click sending an intent to the Android system. And I'm going to walk you through
this test case comparing the different styles of Makita robolectric or espresso. The first let's consider mockito. No markings a really powerful tool but it's one that's often overused and sometimes used inappropriately. Making your own classes is great. But Walmart in the Android framework may seem like a great idea at first it consumed lie down a path of difficult problems. Minion short fast is a stateful or they have complex contracts and he's a really difficult or even impossible to satisfy with marks.
So even though we don't recommend it, let's just stop by taking a look at one of these tests and walk through it using the mockito framework to the Android framework. Tell him to give infection we can you up our activity on the test, but look responsible have to step out some of the Android framework behavior in the Supra class activity. so that a response as we expect and I'll test now this introduces some problems. Festival lapaki stopping the class on the test. And this means we're not testing
the true behavior of that object on the pest. Unfair tomorrow brings with it excessive stopping which introduced his only son desirable boilerplate and that quickly becomes distracting from the true intention of your test. For the wind section of a test to execute your coat on the test you the first have to need to register an automatic app to earlier to get a handle on that click listener, which you would vote manually to call the code that you're wishing to test. Now with this approach you can soon to send into a undesirable mess of
argument captives stubbing calls and answer invocations. Finally in the event section contains had intense sent the Android system. You going to have to use another argument capita? Unlock any Android Frameworks in this way tends to force you into a testing implementation details when you want to be testing Behavior instead. These drawbacks and it's late to developers to build their own abstractions to isolate Android. Distance from Leesburg sound set of problems. Presley You're introducing
another layer of crossed into your application and secondly, you're introducing texting apps where bugs can hide. And we believe that why you should Architects your application very thoughtfully, the limitations of the tools shouldn't dictate your application architecture either. So, let's see how this looks like with robolectric Rogue electric is the popular open-source testing framework that allows you to follow the best practices surrounding marks as you're able to use real on Fort Jackson your
tests rather than having to program your own stuffing behavior in each test. It runs on your Local Host, which means it's very fast making it ideal for unit tests. River Electric 10 scrape test that read a lot cleaner. So let's walk through each section in turn. In the given section, we can simply bring an activity instead resume state for a test just by calling Grove Electric set up activity API. In the wind section were able to use real Android SDK API such as find view by ID to get a hold of the view and then Rogue electrics. Click on a p i
safely click on that. Do you invoke the code that we wish to test? Finally in the bend section we can use regular tricks on the testing apis to check the intent was sent to the system. See how much cleanup is fashion is a folksy on the items that really matter in the test. I'm afraid of all these extra cases of distraction. Espresso is a UI testing framework and it runs on a real or virtual device that provides you with a really realistic environment. The trade-off hair is a much lower execution
speed you building up your entire IDK, Define it to device and in Spanish hating the test run waiting for the results and then collecting those back on your local workstation development Cycles. Now the exact same Android contacts exist are we just getting hold of an activity clicking a button and then verifying and intent was sent the Android system. Hey, there was you'll see the apis a very different. best app for this one together turn the given section will use the activity Tesoro
comes from the Android testing support Library activity bring it to the state and provide us with a handle to a knot tests. For the wind section we can use the espresso view Metra apis will find it even question and then safety check on it to invoke the code that we wish to test. I'm finally in the den section. Would you say espresso intense library to capture that 10 and verify that it was the one that we wanted send to the system. Notice hair, but I want to test of many similar structures to robolectric test
example of Esau earlier. The syntax is very very different. so while each of these testing framework Hopkins contrasting strengths and weaknesses is his explosion of styles. It's really become the big problem for writing test. Who in the audience has been using robolectric to write test raise your hands? And what about espresso? And who's using both? We often hear develop is talking about I need to write a Rover electric test or I need an espresso test,
but would much rather you be thinking about writing an Android test instead. Resale as a developer that no matter what kind of test your writing. You shouldn't first have to think about environment and tools and libras that you'll need. We believe that you shouldn't have to suffer the mental load of having to learn multiple sets of apis for doing exactly the same thing. so how to cost you shut the freedom to reflect her and reuse your code no matter what you choose to run it. So what if there was only one set of apis that
you needed to learn? And now we might should also be free to focus on writing your test rather than considering those tools and Libras and environments. Wow to make this a reality today. We're launching Android test if part of Jetpack. Which app hack testing is now first class citizen or the Android tool chain. What unifying the development experience around the canonical high-quality set of apis it will reduce the boilerplate and eliminate the number of tools that you need to
learn. Naturally call in and support is included allow you to write beautifully concise tests. And of course all of this will be open source wheel of contributions from a community members. What kinds of satisfy developers needs in each of the four key sections of a test remember scaffolding given when and then? Scaffolding encompasses the configuration and control apis think getting a hold of that application context. Pete Escovedo and Ray J unit for runners used to execute your
tests. You can use the instrumentation registry to get a handles that application context. Well today we're excited to announce for the very first time you can now use he's a p I suppose you're on and off device tests. The next section of a test is a given section and Halo going to provide two key categories of apis for you will become part of Jetpack adding more apis to help Drive the component lifecycle for you in test. Have you seen previously activity testrol is used to stop your activity and make it available. Sit test in the
resume State you probably use this API running on a device many times before all today. This API 2 will be available to test that run off device as well. Secondly will be providing you with a set of Android test day to build his these will help you construct Android objects that you'll code and the test will interact with. Menifee Android framework classes that you need to have for setting up your test date a difficult to create often does no public construction protest San
Marcos out of the question either and sometimes they just plain clumsy to instantiate with any degree of brevity atoll. So we're including Android test build is within jetpack give you a concise way to set up your test environment. They produce readable code that Froome weight straight the Android components that you need to interact with on a course that portable Android test day to build his work for both your on and off device use cases. The third section of a well-structured test is a when section exercising the code on the test.
Usually this is just simply a case of calling your own code directly. But when you're writing a UI test, it's likely that you're right for the reach for the espresso View-Master apis. Today what happy to announce that espresso to is joining Android test jetpack. Espresso view ipi's effluent and they read beautifully give use them for your device test for a while now. And today we're providing preliminary support to these tests in the case also. The final part of a well-structured test is it then section.
This is why you make a sessions on the state of the system in response to an action. The first lady espresso intense to is going to be joining jetpack as Android test intense. That was intense apis that you've been using to your own device. Testing great news today. They too will run in your off device tests. I'm finally what else are releasing on his sessions library to help reduce the boilerplate in your tests. Using traditional Jaden if it's actions can lead to test the
not immediately readable see here how easy it is to get the actual and expected arguments mixed up. And Android uses a lot of integer constants for efficiency. But it makes it difficult to comprehend the error messages in tests. A Google we love to use truth. It's our own open-source fluent testing a sessions Library. Using affluent assertions Libras a great step to producing more readable code. I'm right in the test become much more easy to because you can lean on the built-in support of your IDs auto-completion feature.
Set to help you write concisely tests against Android code will be releasing a set of Truth extensions for Android which reduces the boilerplate reads. Beautifully. I give the meat really meaningful error messages. Of course these assertions work across all environments phoseon and off device tests. So with jetpack Android test will be bringing you the tools that you need so that you can concentrate on rice and beautifully can size easy to read test without first worrying
about Libras. All tools are environments a single set of canonical apis for common tasks. It will reduce the boilerplate leaving a test clear and readable. I don't be environmental agnostic allowing you to run the test data on device on your local workstation or perhaps in a cloud test lab I know that I'll show you how to use this unified set of apis that decouple the active actually writing a test from where it's going to run you over to my colleague Stefan. He's going to show you how you can run these tests and then use simplified weld stuff on.
Thank you, Jonathan. Welcome everyone. Can you use Android quite amazing to everybody who is joining us at the live stream today at Google We Believe? The testing should be a fundamental part of your app development strategy. Let's bring back the pyramid. As we can see our friend with the jetpack has solved the API dilemma. Now that we got one API to rule them all. It's really easy to start writing test for Android no more excuses. We can start with a simple test
for our business logic usually a unit test and overtime we can add more and more tests. Once we start implementing the UI for application. It might be worth adding an integration test. This just became affluent experience. Since both layers support the same apis. No more context switching great. But that's not the only use case burden uapi become really handy. Let's imagine we have an integration test. I got two large and complicated. I have seen many of those tests. It's just too convenient to test business logic and your wife though
in one single test. At Google we don't like those tests. They are hard to read and they tend to become flaky. Small ball Focus tests are much better. Let's reflector this large complicated test. Refactoring a large test can be a very painful task. Who in the audience has experienced this recently raise your hand? Here's some good news for you is the original test was written with the new API this test become much simpler. We still have to test our UI. And we'll keep this part in the integration test.
This layer provides High Fidelity and we don't want to lose that for all you lightest. The rest mainly business watching can directly go into the unit test layer here. We can't speed. Since we can run off device on the local television. Let's verify the refectory together. We still have high fidelity for DUI test. That's good. We gain speed even better. And the tests are decoupled and less complex. Nice job your co-workers full. Thank you. So let's run those tests, but wait. We still have to choose a runtime environment. It's not obvious trivial
to pick the right one. We also have to work in multiple sources. The combination of random environments Plus or sets Philippe to an explosion of test configurations. Just imagine how hard it would get to choose the right configuration. Should I run on a device or offer device? And don't we need to run the entire test sheet on our continuous integration server before we submit and you all know the feeling. After we kicked off the test run waiting eagerly for the results. Oh, no hold this admit. One of the tests like it. I'm pretty sure
many of you have been in this situation. Flakiness is one of the biggest pain points for developer productivity. So what if That was a better way. To set up your test harness. Execute your test in a reliable environment with unified test results. Today we are proud to announce project nitrogen. Then you single entry point for all Android test. Is fuel for your jet packs. With our expertise of running billions of tests a year. We had to bolt the fastest and most reliable test executor.
Covers the entire testing life cycle from setup to test execution and Reporting. nitrogen provides deterministic Behavior across different Protestant It will be fully integrated in Android Studio. At Google we already use nitrogen to test our own apps such as Gmail Google Maps. photos YouTube and many more Is highly extensible? He provides a PS4 test others to customize. the test invocation at any point will be fully open source later this year. But let me give you an overview.
Starting to set up an connect to test to any execution environment of device or in the cloud. It installs require test artifacts and if necessary, it can run custom fixture scripts for you. Let me show you how this works nitrogen fines and Provisions a device. This can either be assimilated device a virtual device or real device is ready and install the application under test and the test. Nitrogen stages any tests are dependencies and sets device properties. If additional setup
is necessary, it can all be done here. Let me give you examples setting up an extra panel starting an emetic server granting permissions and much more. And finally It prepares the device for the next step test execution. There are many ways how we can run an Android test. It's a little bit like running in the real world. kiwi have sprints mid-distance and long distance runs different environments Shrek running Road running cross-country running running a unit test station is like a Sprint. As fast as possible
to the finish line. The tests we on your continuous integration server is the marathon the goal is to get to the finish line and not to fail along the way. Today these tests are executed in different ways. from the command line from Android Studio or trick it automatically when you submit with project nitrogen, this will become consistent. We provide eval Define protocol. And are unifying all the entry test offerings. Nitrogen uses our own device infrastructure to run your tests.
Android orchestrator and enrich ayunas Runner you can Oregon Naval the only by Sandra strepto today. It has been available since Android Studio 3.0. the orchestrator collect UPS and kicks off test execution play running each test in a separate process shared state is minimized. And crisis are isolated. moreover your pastor executed in a Familia Che unit environment. provided by Enrique on his Runner the orchestrator collect all test results additional artifacts and streams it back to nitrogen.
Nitrogen provides a unified reporting for Med for these test results. In addition, he provides a huge selection of test output data such as logcat screenshots video profiling data battery performance and much more. all artifacts of scope protest this means that for example, a locket snippet is reduced to the test method no more digging through hundreds of lions of Lockett. Let me show you how nitrogen improves the entire testing slow on Android. First nitrogen find the device and come take a seat for the test run.
Second it runs test in isolation using the orchestrator. and finally Well tester running. Nitrogen host site infrastructure will be streaming the test results from the device. Pulling down altice updater and feeding it back to you. Which nitrogen all this complexity is hitting from you. Whether you're running a test on Android Studio. Or on a CI server. Nitrogen is the single entry point for all Android test. It works from a Sprint to a marathon. Henry Cuesta device in a Firebase test lab reliably running your test returning UniFi test
results natural supports Google Cloud from your workstation you can deploy and run tests on the real device or on the virtual device. nitrogen Brooks seamless Libras robe Electric No weekend three drove electric as a simulated device. Withrow electric 4.0 We have made big improvements instead of time and memory consumption. Rolex at 4.00 is released today. If this is not enough. Nitrogen supports your custom needs for example. if you have a in-house if I slit
allow me to summarize. Previously you had to learn multiple approaches for doing the same thing. tools lectum ability to move between the layers of the pyramid you had to choose wisely. We have reduced a cognitive load by providing you a single set of apis the brook across environments for both on and off scenarios. And a single entry point for a drug test with the flexibility to customize any point in the past invocation. Jetpack with nitrogen is a child be forward in test automation for Android. ride your test once
run it everywhere. This is just the beginning. Now you have both the tools and the knowledge. To accelerate your testing experience. I strongly encourage you to check out a couple lives. Especially our latest addition building Android apps with basil. Basil is the open-source version of our internal filter system. Allowing you to build large Android apps at Google scale. If you have further questions, or if you would like to discuss your testing strategy. Come find us tomorrow morning at 11 a.m.
Buy this talk
Access to all the recordings of the event
Buy this video
With ConferenceCast.tv, you get access to our library of the world's best conference talks.