SeConf 2018 - Chicago
October 18 2018, Chicago, USA
SeConf 2018 - Chicago
Video
Run the load down your mobile app
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
878
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speaker

Christina is a Senior QA Engineer with 4 years of with experience in both Automation and Manual Testing. She also possesses strong skills in Test Automation with Selenium and Performance testing with JMeter. She comes from the development background as she has worked on PHP Web Development and Android Mobile Development before taking up the Software QA Engineer role at Zaizi. She is passionate about learning new testing tools and technologies. She has been very actively contributing for Test Automation. She has been talking at various meet ups and Selenium Conf 2016, Selenium Conf 2017 and the Colombo Test Automation 2017. Currently she works with for Sysco Labs.

About the talk

Topic: IT

Most of the time seldom do we think of our Mobile Performance? But do you think the end-users will be happy with using these apps that have very poor performance? Then let's learn to run the load on our Mobile Apps before delivering them to our end-users. We would love to keep our end-users happy.

I will be briefing on the importance of running performance tests in your mobile apps. And how it is done using JMeter, both Android and iOS applications. Then will be talking on how these scripts can be used in Blazemeter and how is the testing done through Blazemeter which basically uses the JMeter scripts to run the tests, I will be also showing on how it can be done directly through Blazemeter. Further, I will be talking on the efforts of data of the test can be grabbed and used to create your own dashboard or the use of the sophisticated report provided by JMeter alone. The above will comprise quick short demos. This can be helpful for many who are not aware of the benefits of doing mobile app performance testing.

Share

Good afternoon everyone. 00:00 How do you guys feel? 00:06 Awake after lunch. 00:08 Thank you so much charge for actually waking up with his short monster or something. 00:08 So I'm Christina and I work for Sysco Labs being in the industry for the past 4 years. 00:17 I'm very passionate about not just the functional automation but also the non-functional automation testing as well. 00:24 So this is you probably would have seen me in a couple of conferences. 00:34

An you can follow me on Twitter and also connect me on LinkedIn as well. 00:40 So very used an application, 00:46 a mobile application that actually made you trash the Phone. 00:48 I've used it when anyone. 00:52 Seriously, 00:52 there's been super slow then. 00:56 I really broke my Phone. 00:58 No, I'm kidding. 01:00 So by the way, 01:00 a piece of advice before getting married, 01:02 just make them use that because patience is a virtue. 01:05 Yeah and we need that in married life. 01:09

So jokes apart today we're going to talk about run lower down your mobile app, 01:12 right? 01:18 So the end of the day we're all end uses. 01:18 We've been using different mobile applications and so do. 01:21 Mobile app end-users as well. 01:26 So as end uses, you've been frustrated using slow applications. 01:28 Same. 01:34 It would be a feeling for guys or people who are using your mobile apps. 01:34 Do you have a wonder that performance is also key for mobile applications? 01:41

Yeah, because it really gets your end-user frustrated. 01:47 So today this is basically the agenda so I'm going to give you an insight into the performance testing the aspects of it and then we'll dig a bit deep. 01:52 Into the aspects and I'll also have a few demonstrations on a few open source tools on how we can actually address these issues. 02:05

So first of all, this is a survey that was actually done in 2015, 02:18 so that's like 4 years back and the expectation was Super High then right? 02:23 This is a survey done for mobile end users and sorry to say like most of the people who took this about 50% 02:30 edge wanted the application to respond in 2 seconds like 2 seconds and there's like half of that percentage. 02:39

Significant up around who actually said they want the application to respond in a second or even less so. 02:49 This is the expectation. 02:57 In 2015 we are hitting 2019, 02:57 you know, now the expectation is definitely grown right. 03:01 And then there's also the survey they asked if there's this app is slow. 03:06 Will they be as in the end, 03:12 users will reuse the app again or will they? 03:14 How many atoms will they do before? 03:18 Moving on. 03:20

So the Maxim was like most of them said 3, 03:20 right, less or even there's a significant amount who said less than 3 right? 03:25 So that's a wake up call. 03:31 We use, we lose. 03:33 I end users if we don't actually worry about the performance, 03:33 right. 03:39 So an if your app is super slow as draining your battery then India versus just move on. 03:39 If you have a competitor E that's it. 03:47

This is going to move on and then they also either they just stop using it and then just leave it just lying. 03:49 Then the mobile move. 03:58 Just uninstall the app and just move forward. 03:58 So this is basically the issue and we need to start looking into this site. 04:04 We we've all written functional tests or we do some kind of manual tests or so on right. 04:09 For when it comes to mobile. 04:16 But do we do the performance? 04:18 Do we see the performance in depth? 04:21

So that's the question that is then why do we feel more reluctant when it comes to mobile compared to the desktop or web applications? 04:24 So as you see, there is when we talk about mobile, 04:34 there's multiple clients. 04:38 So when when I say multiple clients evil, 04:38 we wander. 04:43 OK, there's iOS does Android this other few operating systems. 04:43 What is she talking about? 04:49 No, it's like multiple different. 04:51

Uh you different mobile devices right now we're just talking about. 04:54 If you take a iOS, 05:00 there's multiple different mobile devices. 05:01 It has different ram sizes, 05:05 memories I sizes and so on. 05:08 The battery consumption and so on right in the same way when it comes to Android is different. 05:10 Brands is a series BCDC series so on. 05:18 So then there's the technology. 05:22 So when it comes to web applications is just HTML based. 05:25

But when it comes to the mobile either will be HTML or native. 05:29 And when you guys use mobile applications. 05:34 Anyone? 05:37 OK, so like in restricted conditions, 05:39 right? 05:42 Like when you're in the bus in the tube in the subway somewhere when you are in a rush. 05:42 So there's restricted bandwidth. 05:50 When I say mobile data and so on, 05:50 it will be where restricted from using it on web application on your desktop, 05:55 right? 06:02

So this is the main issues where these are challenges and when these are challenges we need to face them and we need to look into. 06:02 These aspects when it comes to performance right. 06:13 So these are the main aspects of mobile performance testing, 06:17 so we always consider these serve API performance testing. 06:21 So even when it comes to web application or desktop web application, 06:25 we think of the API performance. 06:30

But when it comes to mobile, 06:33 there's other aspects like device performance as an end user, 06:35 you guys would have complained sync. 06:39 This particular app is draining my battery. 06:41 This particular app is using too much of my data. 06:45 Uh, Oh, it's just taking too much of memory, 06:48 so there actually apps that mobile app providers who have a standard up and they have a light app which works with less network or uses. 06:52 Less amount of battery, right? 07:05 And then. 07:08

So these are the main aspects that we need to consider, 07:08 not just the survey API. 07:13 So first of all, we talk about the survey API performance, 07:15 right? 07:20 So this is basically mobile API calls talking to the server whether the response comes in the expected time and so on. 07:20 How many of you use this aspect of mobile performance system? 07:29 So basically I'll just be showing this concept of how we will we we can do survey or mobile serve API performance right? 07:35

So basically this is the idea, 07:47 right? 07:50 The basic idea. 07:50 The HTTP request goes to the back end and then the responses come back to your mobile application, 07:50 right? 08:02 So we're going to. 08:02 Most of the tools that are available for this survey PA performance testing. 08:05 There are paid tools multiple paid tools as well, 08:10 and there's open source it today. 08:14

I'm going to actually demonstrate Jamie to see you guys, 08:17 but as J Meter it has a couple of restrictions when it comes to like using multiple users and so on. 08:21 So there's various other paid tools that actually help out with such constraints, 08:28 but let's just start with the basics right. 08:34 So basically this is the idea of it. 08:37 So we're going to have a recorder and it's going to literally. 08:41

Record the the calls that go from the mobile to your server and the responses will also be grabbed, 08:48 right? 08:58 So this is the basic idea, 08:58 so we're going to give the application. 09:02 Sorry, your mobile. 09:06 The ability to trust your load running test the test application and then with the proxy settings. 09:06 Will be able to record it and then you work on those tests right to run your mobile. 09:19

You basically will mimic the mobile application to run the tests on this server. 09:26 And then once you do, 09:34 you will be able to analyze the results. 09:35 So this is basically the demo. 09:38 So this basically starting my J Meter. 09:44 So I'm using Jamie to 4. 09:54 And I am using the recording template. 09:58 Right. 10:04 So in that you have the HTTPS test script recorder. 10:04 Then you set the port right Anne. 10:15

Also, I'm getting my IP because I'm going to record the tests on my mobile device. 10:18 Next size start the script recorder because it's going to create a certificate which which is valid only for 7 days. 10:25 So I will be sending it to my through email. 10:36 An I'll be using that in my mobile device. 10:40 So this is my mobile device, 10:49 right? 10:52 So I've already got the certificate install, 10:52 but just for you guys to have a look at. 10:56 So there is an expiry date as well. 10:59

So this like 7 days and so this is basically the CRT file. 11:02 All the properties. 11:07 And I'm just to be sure I'm switching on airplane mode and then going into a Wi-Fi and I'm setting my proxy so you can. 11:07 So this is how it's done on iPhone. 11:21 Android is more straightforward, right? 11:24 An you set we proxy and then we be now able to run the test. 11:28 So basically I'm going to start the test the recording on. 11:34

My J Meter and then start using my mobile application and start recording the calls right. 11:40 So I'm using a comma's application. 11:49 So, as you can see, 11:54 the visual feedback that's being given is it. 11:55 There's multiple API calls that goes through when you just load the application. 11:58 So as you can see, 12:03 once I load the application, 12:04 there was multiple caused and there's a post called that gives the details of my particular application. 12:06

So I love my shoes, 12:12 so I'm searching for shoes. 12:14 OK, so yeah, there you go and. 12:18 There's more cause because there's various different get API is because there's a various different shoes that I have appeared. 12:22 OK, so specifically selecting one, 12:36 so you can see that the recording is taking place in now what? 12:38 You can actually do is looking. 12:43

So if you actually have this as your own app, 12:45 then you'll be able to understand which APIs there because you can talk to the developers narrow down which aspects that need to be tested when it comes to performance 12:49 testing and then start testing. 13:01 So I'm just going to run this test for like about 5. 13:03 Concurrent users. 13:07 And I'm just going to disable all the other particular controllers. 13:10

I'm just going to use only the first where the opening of the app API and just going to run it. 13:15 So I've saved the I'm saving the test. 13:25 Or the GMX. 13:30 And you can see the tests are running. 13:44 And yeah. 13:48 So as I mentioned to you, 13:50 there are few restrictions when you have to have a massive amount of users. 13:52 Yes, as an open source tool Jimmy to doesn't handle it much. 13:57 But then there are various paid tools that actually work with J meter, 14:02 right? 14:07

So we've now had a look at the server API level of performance testing that couple of you already been aware of, 14:07 but. 14:16 These this particular aspect. 14:16 Device performance. 14:19 So as we know, 14:19 as I've already mentioned, we certainly complain that particular app is draining your battery or it's using too much of memory and the app takes so much time to load 14:22 or start up and it works with this particular Phone. 14:34

But it doesn't work with this particular hardware, 14:38 right? 14:42 It works on this software, 14:42 but it doesn't work on the. 14:45 All the version of Android, 14:47 so not everyone's using the same device, 14:49 but we have different end users, 14:53 right? 14:56 It's like every device is kind of unique, 14:56 so in that way we ready to face such issues. 15:01 Is application ready to face such issues then? 15:05 Then we talk about network performance right in that if we have different end users and. 15:09

From different parts of the world, 15:17 are we sure that it works in a certain continent with their network speed that they have an in a different continent? 15:20 It works with the network speed that they have or is there any sort of packet loss. 15:29 Is there any certain jitters and so on. 15:36 So these aspects we need to look at and how can we achieve that? 15:39 Is the question. 15:45 So we've come up with this. 15:45

The idea of using appium, 15:48 so there's 2 different frameworks that we're using. 15:50 One is with Android. 15:55 An one is for iOS, 15:55 so we've basically trying to get the performance data and then comparing or creating a trend. 15:58 Technically an understanding how it works, 16:06 so I'll give you a basic insight of what was the main essence that we use to start with this framework, 16:09 right? 16:18 So. 16:18 Currently with the How many of You used Android profile? 16:18 I've seen it so cool. 16:25

So basically this is pretty much similar to what Android profile it is right? 16:27 So F him is using ADB dump system and giving us the information an we be able to either understand a trend on aspects like memory info, 16:35 CPU info, battery info and network info for as of now. 16:47 So there's basic command where you use. 16:52 Get performance data. 16:56

You specify a particular package and then use up earth type as mentioned, 16:56 like as CPU info and so on and then particular time outs. 17:03 So particular poll time like about five seconds or 10 seconds it depending on your particular application. 17:08 Depending on the performance requirement for your particular application. 17:15 So before actually writing these tests, 17:21 you need to narrow down so that part. 17:24

Of the whole performance requirements need to be clarified. 17:27 You need to understand what is important. 17:31 Which particular scenarios that you need to do performance testing. 17:35 So if for example we do mobile functional testing with a PM so we just have a few other scenarios. 17:40 Which part of the performance package which helps us to basically create a performance trend? 17:47 So let's quickly have a demonstration on this. 17:56

So this is my Android studio and I'm going to first show you how Android profiler works because not many of you actually put your hands up. 18:06 So I'm gone into my Android profiler and I've already got my emulator set because for demo purposes. 18:17 But when you are actually running chess, 18:25 make sure you run it on real devices to actually get the right output right. 18:28 So uh, what I'm doing here is using Android profiler. 18:37

Meanwhile, doing some activities on my emulator and you can see that there is a memory trend and you can see that there is this stack usage the native usage 18:41 and so on. 18:54 All those details have been displayed so there is specific documentation for you guys to actually understand these aspects when it comes to performance. 18:54 Which is part of the Android profiler, 19:07 so you'll be able to understand this. 19:09

And once you do understand that, 19:12 then whatever data they get through the APM tests is also you be able to understand that quite easily. 19:14 So this is basically the same thing I'm going to do it with test automation using appear. 19:20 So, as I mentioned, I'm using the command that I showed. 19:28 So this is very basic for you guys to understand and take it back and build on this. 19:32 So I'm giving a specific call time. 19:40 A full time. 19:44

And then we also specify the particular package an currently I'm using memory info and I'm going to take the total proportional set size or the ram. 19:44 Basically the ram usage and I'm going to compare it. 19:54 So initially I take the ram usage and then compare it after doing the activities with this particular application. 19:58 OK, so that's what I'm doing. 20:06 So I'm initially taking the total proportional set size right just for you guys to have a look at it. 20:08

So we basically use a couple of APIs to send these and create some trends, 20:14 but just for you guys to visualize it, 20:18 I'm just going to print it on my console and you'll be able to see it. 20:20 So I'm I'm just changing the usage time and also I'm currently I'm going to do an assertion where I'm going to compare the total proportionals its eyes and say 20:31 that it's if it increases more than 5% 20:43 of how it how much it was. 20:46 Then please fail the test. 20:48

So this again, depends on your particular application according to your performance requirements. 20:50 So I just want this test fail now for you guys to have a look at it so. 20:58 That's why I have given a 5% 21:03 assertion. 21:06 So, as you can say the same thing that I did manually with Android profile is what the test is doing and you can see that I print the whole 21:12 memory info that's being grabbed, 21:21 right? 21:23 There's total proportional size. 21:23

The Dalvik proportional set size and so on. 21:26 So all these details for you to understand each of these aspects. 21:29 There is very good documentation provided by the Android Studio K. 21:33 So you can see that the test is failed because the assertion. 21:39 Obviously it was more than 5% 21:44 of the previous value. 21:46 So those are certain values that being printed so next will look at the network info, 21:51 right? 21:58

So there are the aspects like CPU info, 21:58 an battery info as well. 22:02 So let's also just look at networking for what are the aspects that come and how can we compare those values as well. 22:05 So I'm going to get rid of the total proportional set size comparison. 22:16 OK, and increasing the particular usage. 22:34 Wait. 22:38 And yet they're ready to run the test. 22:38 So as the test starts at a point, 22:44 once the test of the uh particular application starts. 22:47

You be able to see the particular network values being grabbed and printed. 22:53 So there is the transmitted packets that received packets, 23:00 so that's the received bytes. 23:04 This start time other operations that are running in the system. 23:07 Sorry. 23:12 The file and. 23:12 Yeah, 23:12 the transmitter packards and the receive packets. 23:15 The values. 23:18 So what? 23:18 You can actually do is create a trend with these values. 23:18 Give us particular poll time. 23:24

Keep checking and create a trend and then understand how this particular the flow of your performance particular whether the app is actually affecting your mobile app. 23:26 Mobile performance, your mobile device performance or whether there is any network issues that. 23:39 The particular app is having, 23:46 so that's basically the. 23:49

Idea of how we achieved or what we've been doing with Android and appium so that we be able to get performance data of our mobile app performance in the 23:52 particular device. 24:05 And then next will have a look at how this can be achieved in iOS, 24:05 right? 24:12 So it's not as straightforward as Android where you get like you know perfect data and then you create a trend. 24:12 But with iOS you can use something called. 24:21

Instruments anyones use instruments people instrument. 24:25 So yeah. 24:29 So basically maybe using a PM to record with instruments. 24:29 Run the particular test. 24:35 An Meanwhile the recording will be taking place right? 24:35 So for that we need to make it keep a point that you will have to start your RPM with relaxed security. 24:42 Otherwise the recording doesn't take place as of now. 24:51 And also you have to pass the arguments. 24:56

Like if you're looking for a particular profile name the time out. 24:58 And so on. 25:03 And then you start the recording right with that particular profile name. 25:03 You can use multiple profiles as well for now for the demonstration. 25:08 I'm going to use the time profiler. 25:12 So when I say time profiler, 25:15 I'll just show for those, 25:16 because not many of you have used instruments. 25:18 So I show you what are the other profiles that you can use? 25:21

Like again, you can get network info and stuff like that. 25:25 On iOS instruments as well and then once the recording is done, 25:28 you save the trace file in a use. 25:34 Zip it up and save it and then then use that file to then create your trend, 25:36 right? 25:42 So. 25:42 Let's quickly. 25:44 OK, so then we have the open developer Tools on my ex code, 25:55 right? 26:01 And then we are going to run the particular tests. 26:01 Manually right? 26:07 So I'm using time profile. 26:07

Are there other values as well? 26:11 Sorry other profile names as well. 26:13 Then we I'm going to run it on iPhone X, 26:16 which is actually not available in the Apple so. 26:19 But yeah. 26:23 Going to run. 26:25 So this is basically whatever I'm going to do, 26:25 but I'm going to do it manually at the moment so you can see that I'm just going into my simulator, 26:30 right? 26:37 And I'm going to do what. 26:37 So this is a free app that I've used. 26:40

So when you have the developer code, 26:43 you'd be able to narrow down which aspects are getting affected as well. 26:45 So you can see that there are few spikes, 26:52 but this is not a very complicated to particular application. 26:55 So there's not much of performance issues as well. 26:59 So you can see the day CPU usage that I've grabs again. 27:03 Same way you can get need network where info and so on as well. 27:07 OK. 27:11

Then you can use the profile mode, 27:11 the narrative more again. 27:15 This is an instruments or not. 27:15 Currently the particular profile name is the only value that you can pass with. 27:20 PM. 27:26 OK, so once you know if you've done this manually, 27:31 you will save it or you and then save it for the usage. 27:37 So now we're going to do the same thing on a PM tests, 27:42 right? 27:48 So I'm. 27:48 Going to start my apium basically on with restricted relaxed. 27:51 Sorry, relaxed security. 27:57

And then I'll be running a few of the same test multiple times, 27:57 right? 28:05 So before that, I specify the time profiler and then start recording. 28:05 OK. 28:15 And then finally, I would store the particular trace file and then after you can build your tests on that. 28:24 So now we can actually see that I'm starting my app PM with relax security. 28:34 As of now it works with relax security because yeah, 28:41 iOS iOS driver. 28:46 OK, so. 28:49

And now the code is set an we can run it and this time I won't be able to show you values right will only be able to see instrument 28:53 race fire. 29:05 So that I started running. 29:06 It's going to do the same thing that I just literally did manually, 29:08 right? 29:12 But. 29:12 A couple of times. 29:12 So now the trace file literally should be saved. 29:45 So let's go have a look at that. 29:48

So opening up the trace file with my ex code or the instruments file, 29:54 so it's it's saved with the time profiler. 30:00 And if you actually have the developer code, 30:04 you be able to match it up right. 30:07 And also there are certain ways that we can breakdown this file and create a trend as well. 30:10 OK, so you can see that it's gone into the main and then you can go into a particular class and have a look at it, 30:24 right? 30:34

So this is basically the idea of what we've started doing with device and network performance for applications in Android an iOS. 30:34 So I've just shown you like very basic or the very starting stepping stones that you can actually start with, 30:45 right. 30:54 An but then this is not it. 30:54 When I said we have multiple clients, 30:57 we need to simulate real mobile devices, 31:00 right? 31:03

Because it's not just checking it on one iphone and checking it on one particular Android phone. 31:03 There's various different clients and we need to check it with different memory usage. 31:12 Sorry ram memory and so on. 31:19 So we need to simulate a real device. 31:22 Farm or something of that sort to actually get this thing going right next. 31:25 Challenge that we have is to simulate different networks. 31:30

So for example, as I said, 31:34 if you have end users in different continents or different areas, 31:36 right? 31:41 It's not the same network speed that is in the US, 31:41 which is found in another place, 31:45 right on another continent. 31:48 It's different. 31:48 So if you have end users who are using the application. 31:51 Radius part of the world. 31:55 Then you want to simulate the network speed and so on. 31:57

So these are a few things that also we need to add on to actually understand the mobile app performance in particular mobile device and so on. 32:01 So if these things are incorporated, 32:12 then we can also we can be happy end users and we can also have happy end end users as well. 32:15 So customers and clients will be very happy. 32:23 To use applications and we wouldn't lose them for other competitors. 32:26

So these are few of the sources that I we use and hope this was helpful for you. 32:31 And you guys can actually go back and use them in your industries or start investing on the ideas that I've given you and thank you so much. 32:37

Cackle comments for the website

Buy this talk

Access to the talk «Run the load down your mobile app»
Available
In cart
Free
Free
Free
Free
Free
Free

Video

Get access to all videos “SeConf 2018 - Chicago”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic «IT»?

You might be interested in videos from this event

September 28 2018
Moscow
16
112
app store, apps, development, google play, mobile, soft

Similar talks

Harshit Dwivedi
Android Developer and Instructor at Coding Blocks
Available
In cart
690 ₽
690 ₽
$10
$10
€ 9
€ 9
Alejandro Hernandez
Senior Security Consultant at IOActive
Available
In cart
Free
Free
Free
Free
Free
Free
John C. Fox
Senior User Interface Engineer, iOS at Netflix
Available
In cart
690 ₽
690 ₽
$10
$10
€ 9
€ 9

Buy this video

Video

Access to the talk 'Run the load down your mobile app'
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
423 conferences
16905 speakers
6065 hours of content