Test Leadership Congress
June 27 2019, New York, USA
Test Leadership Congress
Video
Joshua Gorospe - Test Tool Strategies for Lone Testers
Available
In cart
Free
Free
Free
Free
Free
Free
Add to favorites
114
I like 0
I dislike 0
Available
In cart
Free
Free
Free
Free
Free
Free
  • Description
  • Transcript
  • Discussion

About speaker

I have been a tester for roughly 12 years. I have worked in agency and product companies. I love testing, and have an insatiable curiosity for researching anything related to testing. At the moment I am most interested in experimenting with software test oracles, a few machine learning tools, properties-based testing, fuzz testing, and model based testing.

About the talk

Topic: IT

These days, test teams in many companies all over the world are going through staff reductions to the point where probably only a few or one tester is left. In my proposed session I will present and demonstrate some unconventional tool strategies and approaches. This is from the perspective of a lone tester working on fast paced projects with very little assistance.


00:04 Introduction: I’m a tester for 12 years

03:27 Before we dive in: about what this interactive presentation is

05:06 There is just no silver bullets at all for anything

05:49 I can give you the tools in the understanding

06:10 A little funny story

07:23 Some tools that the presenter is familiar with: Docker, Robot Framework, etc

11:08 Before the lone tester shows up the projects and development team

13:15 After you jump into one of the biggest projects what you're going to notice

13:56 Some suggestions about “Omega Tester”: testing with a team of one

15:53 Presentation examples

Share

Alright so as I was saying, 00:04 Priyanka made some great slides and their beautifully well orchestrated slides and the way that these slides are structured there for multiple. 00:06 Audience is there for audiences that. 00:15 Armor of along the lines of if you teach them. 00:19 Or show them actually how to fish. 00:22 They can feed themselves for a day so with these slides, 00:25 I'm hoping to teach future testers. 00:29 For a lifetime, so I am a mind of. 00:31

Show people how to teach themselves and then they can learn through their own journeys how to use certain strategies and how do? 00:35 Self heal during events where I'm the only one left. 00:43 And no one else is around to show me. 00:48 How this works or that works so that's the one of the main goals of this so let's start off. 00:51 With introducing myself, my name is Joshua Grass feed. 00:57 This is test tool strategies for loan testers. 00:59

And again so short introduction, 01:02 it's not short actually it's pretty long and worded. 01:04 I've been a tester for roughly 12 years. 01:08 I've worked in agency in product companies, 01:11 originally came from background of working with very formal and traditional test teams. 01:13 And when I mean, formal this specifications were given. 01:17 And cetera, etc during the last 3 years of Mycareer and about yeah about 3 three and a half. 01:22 I became into mega tester. 01:28

So not by choice, it happened to me, 01:31 I started studying and teaching myself rapid software testing. 01:33 And when it was available. 01:37 I would read through James box. 01:39 RST PDF he took that down by the way 'cause. 01:41 He just recently did. 01:44 Eh. 01:44 Website redesignforsatisfies.com, I would often read through the context driven approach to automation testing and also the session based test management examples. 01:46 They're still there by the way you still get to those. 01:55

And I just started reading it to the big read. 01:58 It's a lot of knowledge to digest and I don't. 02:01 Recommend you to just digested in one weekend just take your time. 02:03 Lessons learned in software testing context really approach. 02:07 It's one of my favorite self study books. 02:11 So I enjoy experimenting with test tools. 02:14 I like concerning myself simply tester. 02:16 I'm very proud of that and. 02:19

Even though my my current role can't really mention is where it is right, 02:21 I work right now. 02:27 They work as a test engineer. 02:27 So I personally believe that automation is a part of the testers toolbox. 02:31 It's not the sorry. 02:36 So that that just give me a second. 02:36 Apologize this is actually. 02:42 Nice. 02:46 Try to keep 02:46 So I believe that the testers toolbox is very it's very. 02:51

It's very important to know that tools are not all that testing is about and that's what I try to teach. 02:56 Young testers when they? 03:02 Come to work with me or if I have to mentor a person. 03:02 To take over something? 03:08 So a funny fact is I looked up detective scientists. 03:08 That's where I feel like as a tester how I feel I found this stuff. 03:14 Will NASA? 03:18 Article. 03:18 Detective scientists it's pretty interesting. 03:18 So I have to keep touching. 03:26

All right before we dive in I just wanted Jeep clear about what this interactive presentations about FY I the. 03:27 The sketch schedule on test. 03:36 Leadership Congress has an update of the GitHub repo. 03:40 That's the companion repo for this presentation. 03:43 It also has the presentation slide so you can follow along read at your leisure. 03:45 I'm going to try and job through the slides so I can show you all the examples, 03:49 we could talk about it. 03:54 And then 03:55 Really. 03:58

Sorry. 03:58 OK, so one of the things that I want to make clear is. 04:00 This is not an exhaustive list of all the possible test strategies that can be applied to any project. 04:06 Or products. 04:11 Please. 04:13 OK and trying not to give you all silver bullets your project or product. 04:13 The unique and your tester journey experience career path may be completely different. 04:21 One of the one of the things that I really like from. 04:27

From the lessons learned in software testing is only you know the context of your work so. 04:33 You are the person that understands the underlying risks of what you're working on so I highly recommend you remember that. 04:39 As you go through these examples. 04:45 And evolving an ongoing experiment Vine this entire experiment is basically this GitHub repo that I've shared with you. 04:47 And also these slides that I constantly. 04:57

Update according to the conference, 05:01 I'm working with. 05:02 So. 05:02 Just keep an open mind, 05:02 I think that there is just no silver bullets at all for anything and. 05:06 Everyone is working on something completely different and I've come from. 05:10 I've come from companies where we work on POC's we work on all sorts of my previous company an intersection. 05:13 Used to work on several types of go sees and. 05:21 It was a very context, 05:24 switching was very common it's just. 05:25

It's hard to come up with test strategies that just apply to nearly everything 'cause. 05:29 Frankly, they just need to be. 05:35 Revisited re thought of. 05:37 Evaluate the risk. 05:37 Adjust accordingly. 05:42 So one of the things that I personally I'm a real believer in is. 05:42 Is giving people the tools in the understanding too? 05:49 Enhance your capabilities to work as fast as they can within their means and. 05:53

I've chosen tools that I think are very common place at the moment and we don't know the future. 05:59 We don't know if certain things like Docker Docker is a very. 06:06 Commonplace containerization, so and. 06:10 I a funny story is I was in a coffee shop 2 years ago and I was just doing some work, 06:10 and drinking coffee. 06:21 And I over heard these young gentlemen were talking about this is how you run. 06:21 This container here. 06:27 This I do this. 06:27 It's like this is that? 06:30

This is the stuff for example, 06:31 that the teacher taught. 06:33 I was like and I was curious. 06:33 I walked up to him and say, 06:35 I'm sorry like. 06:37 Where are you learning your doctor ization knowledge from? 06:37 Are you in a college nearby? 06:42 They told me know now we're in high school were in AP computer science, 06:44 I was like. 06:48 OK wow that's awesome. 06:48

How like this is great and just an FYII live in Bergen County, 06:50 so they were actually somewhere around Franklin Lacer That's a very wealthy area so. 06:54 It makes sense that they got Top notch uh as schooling and education in computer science that early so. 07:00 Just think about it the tooling and things we look at and say, 07:06 Huh well. 07:11 I don't really need daca right just. 07:11 Really all I need is a bare metal, 07:15 I just need 20 laptops. 07:17

Yeah, it's true today, but tomorrow. 07:20 I mean, who knows you know it's like the kids of today are learning the tools. 07:23 We kind of look at. 07:29 Today and say, I don't need that. 07:30 But just think about that. 07:33 So Docker is one of my one of my gotos. 07:35 There's several go to site it's just. 07:38 One of the things we covering often today, 07:41 an robot framework, so I unofficially kind of represents a group in. 07:44

The robot from Rick Foundation and because I believe in it, 07:50 so much and it's kind of changed my. 07:55 Outlook on automation versus testing. 07:58 And just the active checking that our tests do. 08:01 Is uh? 08:05 We can orchestrate that we can be. 08:05 Better testers enhancing our skills if we let our tools enhance our process, 08:10 not have our process and tools take over. 08:16 So I guess I'm very passionate about that. 08:20

Actually, I believe that generic test tools like robot framework where I can. 08:22 If I need to just automate a fully automated regression suite that just does checks that just do the same thing I expect all the time. 08:27 But that in the background. 08:38 I pull out another tool, 08:40 a hybridized sort of hybridized automated tool but waits for me me the decider. 08:43 Of this particular session. 08:50

Did it pass or fail and I have a number of things that I can implement in that particular session 2? 08:50 Either say, well, yeah, it did or didn't so robot framework is that it's an automation tool and it's. 08:58 Also, a one of the few open source robotic process automation solutions. 09:03 So if you are. 09:10 I've talked to people to people last night dinner alot of people are familiar with our PA. 09:10 So I highly recommend you look it up. 09:16

They're one of the leaders in open source RPM. 09:18 And UI Path is I'm sure it's a great tool, 09:21 it's a great piece of software. 09:24 It's uh it's also from where I come from right now, 09:27 it's not always feasible to. 09:31 By this license for X amount of money. 09:33 It wants. 09:36 Please look up the open source solutions parse that sort of thing now. 09:36 Many of these examples. 09:41 You don't need to use this framework for it's just a simple like. 09:41

I'm comfortable with it, I've created several prototypes of POC tools up with concept tools by the way. 09:46 I've been asked before what a proof of concept POC means and I have to make that clear so. 09:53 Maybe like to write all the code yourself and I'm. 09:59 That is awesome and. 10:02 I have to say that that's great for the people that can do that. 10:02 and I think it's a skill that I would love to improve. 10:08

I personally is for my own personal experience after you get used to the natural language syntax with robot framework. 10:11 You will be able to write. 10:18 Tests in Under 30 minutes or less. 10:19 It just it just makes sense to you 'cause it's using natural language, 10:22 it's using keyword. 10:26 Syntax and at the end, 10:26 you could also do interesting things like model based testing feeding in a model based generator. 10:28

Automatically generates a path for your test that that you're running either the background or for your own hybrid. 10:35 Uses so Let me know if some of this is familiar to some of you do you ever feel like? 10:43 It's simultaneous projects. 10:50 This is this is coming out there's there's 10:50 Fires everywhere there's all sorts of explosions and then at the same time. 10:55 It feels like this, it's the Wild West right so this is not happening first this then that it's. 10:59

At the same time so. 11:05 Before the loan tester. 11:08 All of you shows up the projects are there fast paced there frequently changing. 11:08 The requirements scope the client wants more features. 11:16 You have a client, who thinks that agile means you will get to production next week right? 11:20 That's what it means. 11:24 I've had that I had that like this my. 11:24 Uh my team uh and my uh I'm not going to mention where but 11:29 Mighty man my project product owner. 11:35

We're sitting down and they were like well, 11:38 they kind of expected this. 11:40 This feature to be done in 2 weeks or less so. 11:42 OK and then I looked at the developers like. 11:46 And like a And I'm like, 11:49 Alright well, then. 11:51 Let's see what we can do and they think of software development is just so you build it once it's right, 11:51 it's done. 11:57 It's done right, 11:57 yeah, so I never need to call you for any help so OK. 11:58 Give me all your documentation. 12:01

It's a furious when they see Oh my gosh, 12:03 there's a bug in production. 12:06 Just like you, said You You tested it yet and test it. 12:07 You have too many meetings. 12:11 Did you have possibly more than just your team working on this specific? 12:13 Project for this specific irey client so. 12:18 You're waiting for that company that's working with this other company. 12:22 They had that other piece, 12:25 they're working on. 12:27

There there like you know, 12:27 we have the easy thing, 12:29 so let's just let these guys you know they're going to. 12:30 I'm all over the hard back end work this and this and this and this they're going to do all the. 12:32 All the platform connections for us, 12:37 we're just going to do this. 12:39 No wait. 12:41 So I've I've seen it all. 12:41 Your development team is probably spread out across several projects like I said, 12:45 is there several months into their development. 12:48

They probably have a Wild West approach to tools into just check their work so. 12:51 They use different types of means to check that things are working as they expect they can always say OK. 12:57 This this is ready for you deploy to QA. 13:03 So have at it. 13:07 And maybe they're only doing it on the machine, 13:07 so in those kinds of worlds and now this is a. 13:11

This is just a Food for thought after you jump into the one of the biggest of those projects you're going to notice. 13:15 Very quickly. 13:22 Anything you do mean something else is or is not going to get done. 13:22 If you're too busy to planet prepare. 13:28 You feel too busy to mind your infrastructure. 13:30 Now your infrastructure at this point would probably be a text document. 13:33 That is like? 13:38

That long of just taking notes taking meds transcribing what you wrote down in meetings and transcribing it back into this. 13:38 Humongous document with just that project you have other text documents for other projects that are just doing this. 13:47 Doing this doing this so. 13:54 That that might be your infrastructure in like the first three or 4 days right so these are the challenges that you. 13:56 Mega tester. 14:03

Your face so this is all from JS box and they test your PDF I highly recommend you read it if it's still. 14:05 There, I think it's it's one of those PDFs that never. 14:12 Thank goodness 'cause it's it's uh. 14:16 It's a treasure for future testers senior testers to. 14:19 Look at it. 14:24 Right. 14:26 So some of the suggestions that change box. 14:26 Mega tester testing with the team of one, 14:31 the the several things that stood out to me. 14:34

The test activities in the testing story think about that. 14:38 Your strategy in terms of what you need to do what you need to create in order to do it. 14:42 Focus on the activity. 14:48 Uninterrupted sessions. 14:50 Use those laws those like just so I I mean, 14:54 I can't imagine what a lull is for one person versus the next but. 14:57 I've seen a lot less like I got 10 minutes, 15:01 So what can I build in 10 minutes? 15:04 Or wow, I got a real. 15:06

Blessing today I get an hour So what can I set up in an hour use that for preparation? 15:08 Reassessment. 15:14 Think about your strategy. 15:14 Use the recording tools to shorten bug investigation time. 15:17 So. 15:21 Think about all this as a? 15:23 Take that all in while we walked through these examples insights, 15:26 so before we move on. 15:30 If you have a computer. 15:31 You don't need a computer I'm here. 15:32

All day long, we can run examples together, 15:33 you could sit down in a corner, 15:35 but in between breaks in between sessions. 15:37 Examples with you, I'll show you some stuff. 15:39 We could we could do that, 15:41 if you have computer today, 15:43 if you have windows. 15:44 I recommend these. 15:44 Command line tools for git. 15:47 And also Cygwin or baboon. 15:50 Alright so getting to present presentation examples. 15:53

So Part 1 is adapting to frequently changing projects Part 2 is enhancing those existing tasks taking your process. 15:57 Further. 16:04 And one of my favorites is. 16:04 In Part 2 my personal favorite actually because when you are firing off your processes you are firing them off. 16:08 On your local via a remote process using a webhook. 16:14 Either way. 16:18 You have your right in front of you, 16:18 your hybrid tool that is. 16:23

Your brain machine did it pass or fail and have all these supporting tools that all some some interesting things as well. 16:25 And Part 3 is to visualize that whole journey put it together, 16:33 one of the things that robot framework. 16:36 Is that is? 16:38 Combining all test results under 1 pane of glass one reports one log. 16:38 Everything either running in cereal or running in parallel if you will have a very fast test process that. 16:46 Finishes in a minute. 16:52

I think that it's probably better for communication purposes to have it run in serial so that. 16:52 All these steps happens in under a minute OK. 16:59 That's great now versus it took 20 minutes. 17:02 All cops in one month. 17:05 20 minutes break that out into X amount of parallel cases and have that or sweets. 17:06 I should say it's actually a bigger than cases. 17:10 Had them all run in parallel across processes, 17:13 then since we're talking containerization. 17:16 Containers. 17:19

I've been at Kubernetes pods anything you need less ECS. 17:19 Whatever you can imagine so and the beautiful part is. 17:25 Robot framework comes up with this reusable output dot XML log. 17:30 And it also spits out X unit results, 17:34 too, so you can merge those all together with all the other frameworks. 17:36 That your team is using so you're all working under 1 pane of glass. 17:40 There's a polyglot team everybody likes working under like maybe Python. 17:44

Some guys like Scala, some guys like Java. 17:48 We could all come together talk through one pane glass, 17:51 so that is actually a another benefit of robot framework. 17:54 This together and people who are more comfortable with creating powerful strategies can just take the keyword approach, 17:58 maybe use some obvious testing which I'll show. 18:05 And I'll try to move quickly 'cause I got halfway through. 18:08 Alright so this is again. 18:11 It's a block of text don't worry bout. 18:13

These are commands that if you have a computer. 18:14 It just pull it off. 18:17 Run run among the running on your computer on your local if you have all this. 18:18 Every technical requirement installed don't worry about this. 18:22 And at the bottom says will answer questions right at the end, 18:25 so we will do that towards the end. 18:29 OK, so this part Part 1, 18:32 so tool strategy examples for adapting frequently changing projects. 18:33

So imagine in your team you are 18:39 introduced to a team that it's a hypothetical project your developers working on projects, 18:41 they decide to use tools that they're comfortable with confident in 18:47 and the loan tester just used to learn as quickly as possible so you could be contributing to the jury task that 18:51 request. 18:57 It would be helpful to reuse. 18:57 The approaches of the developers or try to set up a CI pipeline. 19:01

Process with your Work in there as well, 19:06 using whatever you want it doesn't have to be darker. 19:10 It could be of course. 19:13 Install the build machines. 19:15 But wherever they may be. 19:15 Now robot framework makes it easy to achieve that, 19:19 but I'm sure it also anything with a process runner. 19:21 Like let's say for example, 19:24 Postman Newman tests alot of I mean. 19:26 Making broad assumptions. 19:29

I've a lot of teams I personally worked with Postman is actually very popular right now for API testing. 19:29 And on Top of that it's really good at API documentation information also offering swagger files all sorts of. 19:37 Fun stuff like that, so it's actually got a built-in test runner an test framework itself. 19:44 But it also has a CLI auction. 19:50 So it can generates a Postman Newman collection that you can reuse in your CI. 19:52 Also. 19:59

If you are open minded to it, 19:59 you could also use it too. 20:04 We run from other tools like robot framework. 20:06 One of the things possible does not do is randomize test running as a? 20:08 Now I may be wrong, 20:12 maybe some other work about work arounds for that. 20:14 But using the folder option that's Built into Postman Newman's. 20:17 Test runner you can run specific folders in any order, 20:23 you like so imagine you have a mechanism to. 20:27

Based on a model a visual model of your a graphical model. 20:32 Let's say of your API behavior. 20:36 Have that. 20:39 Drive what Postman is going to do so that another example is curl, 20:39 sometimes developers. 20:45 I hate these graphical tools. 20:45 I love to just run coconut coconut coconut. 20:48 There, I checked everything I'm good my stuff is good so you could also use robot framework for those. 20:51

Just a straight commands pull them in run them in your process so here's an example 2. 20:58 As a long tester, you're going to be probably working. 21:05 Another example here is maybe maybe you're going to be working in a place where you're testing activities. 21:08 Are all done by you, 21:14 you have to do it yourself? 21:16 You should be able to run your own write your own Python library. 21:18 Maybe developer will walk in and just say OK, 21:21 yeah, you need help. 21:24

I can help you do that faster. 21:24 They'll want to contribute to it. 21:27 So from from this, I am recommending to from robot framework standpoint, 21:29 I'm not entirely sure if any equivalent. 21:34 You have a developer help you write the Python library that is going to support the resources that feed into robot framework. 21:37 But then again, robot framework in reality can pull in directly this Python. 21:45 Python library itself without any need for any of the other. 21:52

Robot framework resources. 21:55 So a simple folder structure here is kind of what I'm implying. 21:55 Having that simple, as simple as possible. 22:02 Just these are the tests. 22:04 These are the resources that talk to these tests. 22:06 And are used by these tests. 22:08 So. 22:10 If you need any guidance or have any questions. 22:10 This is where the code list just have them look at the Python. 22:15 Later. 22:20

Have a look at the under resources and go straight there and say, 22:20 Hey, this is where I I need the most help. 22:25 Please help me out with this part of the automation code. 22:26 So. 22:30 Let's say, and this is a different example. 22:32 The opposite of that example is also true so they're like, 22:35 too busy. 22:39 Look, I'm too busy figure it out. 22:39 Find a way to create your tests but I can't help you. 22:42 Is I don't have time? 22:46

Which could be the case it could be that you need to get that feature out there. 22:47 It needs to get out there as quickly as possible so there are I believe about 80. 22:51 You're about 80 or so. 22:59 Maybe it's increased it has to have increased by now by much more by. 23:00 Great percentage more. 23:04 Open source supported libraries that from the robot framework community. 23:04 There's many of them, they're covered the whole gamut of visual automation. 23:11 Silentium. 23:16

API testing and they even have some libraries for orchestrating Kubernetes pods docker itself. 23:16 It's up, it's pretty vast Ann. 23:26 I really recommend people try to give it a chance and take take a look. 23:28 So. 23:33 So time. 23:33 Alright so I have to move a little bit faster here. 23:33 So for This part here. 23:42 Going to run some some. 23:49 Examples here. 23:52 Are you going to be juggling windows and it's going to be a little challenging? 23:55 In 45 minutes. 24:00

So I'm going to probably run only 2 out of each of these. 24:00 So here's an example of a power one example using Postman collection so. 24:07 In here this docker container run the Postman Collection. 24:12 Read the results. 24:21 Show the status. 24:24 Just 200 OK. 24:24 And it finished in under a minute. 24:28 9 seconds So let's do another one real quick and I'll jump into Part 2 and Part 3 will just have to play it by ear. 24:31

So that was the Postman I could show curl, 24:38 but I'm sure a lot of you know how curl works. 24:41 This is a Python library. 24:43 Example right here. 24:46 So, in this Python library simpler accomplishing the same thing in a different container. 24:53 I've built the containers for so that it. 24:57 Runs. 24:58 And under Minute 12 custom in it, 25:01 so it was really fast. 25:04 We're going to go up here. 25:05 It's running in cereal. 25:07

And it's running in parallel parallel goes parallels you like robot framework can handle it. 25:07 I highly recommend running parallel, 25:12 but also don't. 25:14 Lose yourself in the parallel runs 'cause you'll have like. 25:14 So many tests to kind of like manage now you have all these X test managed so you have to. 25:18 Design your tests accordingly. 25:24 So that's a quick example for examples of from Part 1. 25:24 So let's move on to. 25:31

R2 so Part 2 the strategy is an I'm not going to spend too much time here. 25:35 All of those examples all the examples. 25:41 I was mentioning from before now we tie in some. 25:44 Watching processes some notification processes, 25:47 tying this into slack as people love to use slack there addicted to slack. 25:50 So now, what if I had a process now that talks to people through my tests through my hybrid tools for my automation. 25:55

And if talks to them directly as soon as something fails, 26:03 so that's basically what this slide is talking about. 26:07 And I also have an example. 26:11 I'd love to share with you, 26:13 that as not was not in my Robocon. 26:15 Workshop. 26:17 Triggering webhooks for your wherever they are remote test processes in ECS in AC2, 26:17 another physical machine. 26:25 This is a It's a very uh. 26:25 It's a very useful example and again not limited to robot framework. 26:30

You can just do this with any tooling you like any framework you like. 26:34 So let's go forward a little bit going to speed this up so we could just get to the examples. 26:38 So for this example here I am doing a combination of approaches that. 26:43 I'm just going to basically show these tests that fail on purpose, 26:49 so that they could talk to the team through slack. 26:52 And there could be orchestrated by me or they could be orchestrated by a docker container. 26:54

Run by me locally running a docker container. 27:01 So I really wanted to show this to everyone. 27:04 These are some. 27:07 Get log commands that I'd like to run for you. 27:07 Again, those slides there's nothing different that I'm showing you that are not in the slides that I've already shared with everybody through the? 27:13 Sketch.com. 27:20 So I have here has anyone here, 27:22 not heard of mule or mulesoft. 27:24 Our Salesforce. 27:28

Yeah, so Salesforce bought you'll soften this is their repo and just going to show you. 27:30 Quick. 27:36 I love love their repo this is their current community edition product. 27:36 So I'm going to quickly show a technique engineered at Google. 27:43 They don't use it anymore. 27:47 But it's still kind of useful for most people who love to test. 27:49 Sorry style and just want to know where the hotspots give it to me real quick so. 27:54 Honing in on using git log. 28:01

Where are the bug fixes? 28:04 Where are the fix is done? 28:05 What commits were bug fix commits? 28:07 What etc etc etc? 28:09 So I looked at just now 15 weeks worth of commits and that's that's what came out. 28:09 So that's a quick example. 28:17 I wanted to, well, too much on that have a bunch more to show you. 28:19 And the OK, so. 28:23 Right let's jump into the slack notification examples. 28:29 Here. 28:37 So with this example here. 28:47

Going to run a container it's going to fail some tests on purpose. 28:50 This is the wrong one pardon me. 28:55 What that other example did it ran randomized tests with friend my sub in terms of randomized test data? 29:07 So. 29:13 It's kind of hard to juggle these 'cause. 29:13 There's a lot of examples. 29:16 An I wanted. 29:17 I want everyone to walk out with something. 29:17 Today an This is the example with pass and fails and it also measures the time of how long each test ran. 29:20 So. 29:30

It says here 22122 milliseconds, 29:30 so there's a listener that also catches how long this is taking and also will fail, 29:33 it if it takes too long. 29:38 So from here, my process caught numerous failures in Python Library, 29:42 one and 2 example. 29:47 And so at this rate reflects so let's imagine this is the team slack channel. 29:47 So Oh Wow, Josh is uh. 29:53 Josh is tools caught something in the regression suite so let's talk to him about you know better service service so. 29:56

If you use your imagination on how you could use slack notifications. 30:03 OK so from here time check. 30:11 There's a little bit there's a lot more. 30:14 I need to show I'm going to jump into the graph Walker example. 30:16 Now, before I run this, 30:20 I would love to show you graph. 30:21 Walker is a model based test tool. 30:23 So this is an example of one of those models. 30:25 This is a model of a positive run so this is just going to do a simple walk. 30:28 Works. 30:33

Patch Post Lites four and a JSON placeholder endpoints, 30:33 so and the check is on the get so there is also the. 30:41 Negative testing path, which tries to break everything and also delayed the request by maybe 5 or 6 seconds. 30:47 I forget how long? 30:53 Here is the example running. 30:53 South. 30:59 OK, so this is an example of that of those models. 31:02 Running those parallel paths across split out across 2 robot framework suites. 31:07 So it's going to. 31:13

Generating the path 31:13 Generated the path running the path. 31:18 Now these tasks here. 31:21 These these checks. 31:21 They're not running in a sequence. 31:24 They're running a randomized sequence built by that graph. 31:26 Built by both graphs, so they run. 31:30 In parallel an also in Asia. 31:32 I should say actually this is this is all running in cereal, 31:36 my fault and my my error. 31:40 This one is running a very long path generated generated by the? 31:41

Negative and the positive so it pulls out together in one path. 31:45 Feed that into robot framework and you have a very quick way of setting up model based testing. 31:51 So. 31:57 This is a very powerful way of driving your tests. 31:57 I think that it's something that you should consider. 32:01 And it's based on a model of your systems behavior so this is actually one of my favorite examples and. 32:04 Should have a failure baked in there? 32:10 There there was. 32:13

And that will again go straight to slack. 32:13 So that your I wouldn't say exploratory it's actually technically just a randomized walk. 32:18 According to a model so your randomized walk from that model might have caused. 32:25 An edge case to show up. 32:30 And also having that running against a tool that slows down the randomly slows down the? 32:31 The connection will also help those in case just to check well. 32:38

How does it handle slow connections whatever it is you're testing is? 32:41 So doing acting closer time. 32:45 Alright so another example that I wanted to walk through. 32:48 But I needed to kind of to train this other tool that I wanted to show. 32:53 Going to jump ahead, a bit because we are almost. 32:58 Skipping ahead. 33:02 Alright so I 33:02 Found this tool by chance, 33:06 so we're enhancing our test in our process is there is a tool called get risky. 33:08

That will show you the risks of each commit so it uses supervised classification based machine learning to read through your git log metadata. 33:12 So this, this tool is actually very useful before you get into the hybrid example, 33:23 then I'll show really. 33:28 Hopefully I will get to your questions. 33:31 So this example here. 33:34 Is going to be based on my previous run because training this takes an hour so the model is built off of the git log metadata. 33:37

So here's an example of a commits that made all of these changes. 33:46 This is based off of git. 33:50 Command line. 33:52 This comes straight from get there is no code, 33:52 there is nothing that I did to make this appear this way. 33:55 This just comes from Git log the basic functionality of get log. 33:57 Now, this part here is good risky. 34:01 So it risky after training that model. 34:03 New that Hey this commit is other itself score of .6, 34:07 so think about that. 34:11 Josh. 34:11

Why would I why would I want to pay attention to that? 34:13 Well, I want to pay attention to that? 34:16 Over something that's 0. 34:18 Why I wanted to definitely pay attention to something that's a .9 out of one? 34:21 So think about that as you're exploring. 34:26 Put into the back burner of your brain. 34:28 Maybe I want to check this committee. 34:31 I know is committed on that date. 34:34 And with get log I can also find out food. 34:36 Let's just not done. 34:40

I don't want this to be something that finger pointing at tool. 34:40 This is just something for testers to have in their brains. 34:45 OK, I have this knowledge where could I go. 34:48 Where can I find the risk? 34:52 Where can I dig in and detect these problems? 34:54 So let's move a little bit faster, 34:57 I might not have time for the Charles proxy examples. 34:59 That's actually one of my actually religion today. 35:02

Now it's starting Charles proxy for me, 35:05 this is all run from robot framework. 35:08 I am starting Charles proxy up. 35:10 OK. 35:12 Of a real device here. 35:12 Now. 35:22 Sorry. 35:26 A lot of juggling. 35:29 Robot framework gives this convenience. 35:29 Prompting tools so I'm testing for the mobile changes. 35:36 Etc. 35:41 That rhyme with quickly because this is actually. 35:41 Just trying look like. 35:51 All right, 35:51 so this is a tool based on the basics of its interpretation. 35:53 It is not supposed to be. 35:58

The like one true method of doing such an based testing. 36:00 I interpreted this way, and feel free to try out your own implementation as well, 36:05 you know. 36:10 Fill out your 36:10 Your your messages passed through robot framework will end up in the robot framework log which I will show. 36:14 As soon as I'm done with this example, 36:19 so I want to fail because I found that the. 36:21 Mobile. 36:24 Kohl's. 36:27

I'm running out of time, 36:30 otherwise I could show everyone how Charles proxy works specifically later on, 36:31 because it's kind of running low on time right now like to move on. 36:35 Yeah, visualizations. 36:38 Are wrong? 36:41 Showing errors. 36:41 So. 36:45 OK. 36:49 Mobile homes. 36:51 This is an important part. 36:54 Show. 36:57 500. 36:59 So now that was orchestrated by myself and. 37:01 Since my my team in my organization, 37:05 Love Slack, so much. 37:08 I could talk to them directly from my tool. 37:10

Have all my docker containers running the background fire off all sorts of cool commands, 37:12 which I'll show right here as well. 37:17 I think that's a very useful example run. 37:20 Spoke. 37:24 2. 37:27 I've got a Crow commands running I've got my hybrid tool process running I am. 37:31 By myself doing this. 37:36 Might as well communicated in real-time concurrently with the team, 37:39 so that they know in real time? 37:43 What's going on. 37:44 So here's the. 37:44

My fault 37:49 Go back to the slides lot to show. 37:55 5 minutes ago 38:00 This is 38:09 An I could show the I could show the visualizations in. 38:12 OK, well for example, here. 38:23 So. 38:28 I have left. 38:28 You could also set up your test process as a service that anyone can call from anywhere. 38:45 So wherever this. 38:52 Is running from running wherever it is in AWSU C2? 38:52 That comes out the other end. 38:59 This side here is wrong, 39:02 wherever it's running from container ECS EC2. 39:03 Bare metal machine. 39:07

Cause it. 39:07 This is somewhere else. 39:10 This is from the local machine. 39:10 And and of course. 39:16 That. 39:16 That result is not wasted it's immediately shot off through slack. 39:20 To whoever needs to see it. 39:25 If I could find it. 39:28 So. 39:31 Again, same example from before triggered by a webhook. 39:31 Wherever that Webhook is talking to. 39:36 That will put could be uh. 39:38 I'll check It's Jenkins CI Machine, 39:40 it could be. 39:42 Doing it as post process trigger it could be someone in the. 39:42

Someone from working remote's Indiana. 39:48 Wherever there from and any country aren't enough weapons. 39:53 It just needs to be. 39:55 You need to be able to talk to that web book. 39:56 So. 40:00 We fast forwarded a bit. 40:00 Chords the end here. 40:05 And I'm probably not going to talk through the Appian example right now because we're at time. 40:08 Once you give time for people's questions. 40:15 So this is basically Part 3 is visualizing the testing story. 40:17

We get combined results here and I want to show everyone the. 40:21 Process of of combining everything under 1 painted glass. 40:26 So when I'm what do I mean by that. 40:30 So if I take this. 40:34 Which was wrong recently I I would run the command but it's just more effective to just show everyone? 40:36 There. 40:41 My hybrid process. 40:41 Can be deployed to a heroku? 40:41 Environments. 40:48 Or wherever you like to be something in an S3 bucket if you like it's a static HTML file. 40:48

So hard to read here, 40:58 so I'll just scream. 41:00 Yeah, 41:00 there, we go. 41:02 So this is the process I ran last week. 41:02 I got to see how long each session took. 41:05 What I was doing at each session? 41:10 How long it took at? 41:12 Each and every single piece of that session now. 41:14 You could cut that down. 41:16 You could trim it all down, 41:18 however, you like. 41:19

The The Robot Framework Text Dialogue Library is completely customizable and also open source so feel free to hack at it at your leisure. 41:19 So it doesn't need to be all these steps. 41:29 Most testers like to be accountants. 41:33 They like to have every single thing I want to measure every single. 41:35 Step away or maybe you want. 41:38 3 prompts boom boom bum. 41:41 You could do that with robot framework. 41:44

You do that with any framework really that handles text prompts I don't know of any that does text props that way. 41:47 Or drives one of these one of the fun things about this process that I personally enjoy I've dropped this in the hands of a developer. 41:53 I kind of turned it down and made a little simpler. 42:00 It actually helped them test better it made them focus on what it is I wanted them to do. 42:03

And they automatically talked to my test real's management, 42:08 Sis. 42:12 So they were automatically helping me with my work. 42:12 Closing out all these test runs closing out to test real test runs opening and closing them and at the same time talking to slack. 42:17 So that's that is multiple multiple multiple things done concurrently. 42:24 All in one session and I just gave them my package and they ran it and worked with it. 42:28

So that was one of the best parts of this, 42:34 this process. 42:37 So that is, 42:37 I mean does anyone have questions about Heroku. 42:40 I think that's pretty obvious what it does right. 42:43 It's It's built on Top of a WS it's owned by Salesforce. 42:46 Love Salesforce by the way I know a couple of friends of mine and the robot framework foundation are from Salesforce and they're amazing people. 42:51 So visualize your test process. 42:57 Under the same pane of glass now. 43:00

The pistol results. 43:04 What does that mean? 43:04 Gather everything together you could probably make a prettier one. 43:10 I highly recommend you do, 43:13 I'd. 43:14 Love you to show me your work? 43:14 Show me what you've. 43:16 You've accomplished. 43:16 Make a better dashboard. 43:20 Like this. 43:23 Make it easier to read. 43:23 But no one can argue. 43:26 But this part. 43:28 My favorite part. 43:28 Hey, I found something. 43:31

It's it's important to visualize that for the people in the boardroom serve whatever important meetings. 43:34 You need to join in. 43:40 Here's a URL. 43:41 Here's what I found here, 43:41 the metrics. 43:44 I can show you the whole thing if you like. 43:44 Sir Madam. 43:49 Well, we gotta do a meeting for that because this is the journey is the journey. 43:49 Etc, etc, etc. 43:54 You could also collapse, 43:54 it down. 43:57 It's very easy to navigate through. 43:57 He's gotta drive some law. 44:00 Scenarios down. 44:02

And I think that's 44:05 The Internet. 44:09 Which I think? 44:11 And references in case anyone's got questions while I was in it. 44:11

Cackle comments for the website

Buy this talk

Access to the talk “Joshua Gorospe - Test Tool Strategies for Lone Testers”
Available
In cart
Free
Free
Free
Free
Free
Free

Video

Get access to all videos “Test Leadership Congress”
Available
In cart
Free
Free
Free
Free
Free
Free
Ticket

Interested in topic “IT”?

You might be interested in videos from this event

September 28 2018
Moscow
16
122
app store, apps, development, google play, mobile, soft

Similar talks

Lina Zubyte
QA Consultant at ThoughtWorks
Available
In cart
Free
Free
Free
Free
Free
Free
Sami Söderblom
Testing & DevOps Coach at Elisa
Available
In cart
Free
Free
Free
Free
Free
Free
Mike Talks
Test Manager at Datacom
Available
In cart
Free
Free
Free
Free
Free
Free

Buy this video

Video

Access to the talk “Joshua Gorospe - Test Tool Strategies for Lone Testers”
Available
In cart
Free
Free
Free
Free
Free
Free

Conference Cast

With ConferenceCast.tv, you get access to our library of the world's best conference talks.

Conference Cast
505 conferences
19653 speakers
7164 hours of content