Test Leadership Congress
27 июня 2019, New York, USA
Test Leadership Congress
Видеозапись
Joshua Gorospe - Test Tool Strategies for Lone Testers
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
В избранное
126
Мне понравилось 0
Мне не понравилось 0
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
  • Описание
  • Расшифровка
  • Обсуждение

О спикере

Joshua Gorospe
Senior Tester в Intersection Co.

I have been a tester for roughly 12 years. I have worked in agency and product companies. I love testing, and have an insatiable curiosity for researching anything related to testing. At the moment I am most interested in experimenting with software test oracles, a few machine learning tools, properties-based testing, fuzz testing, and model based testing.

Перейти в профиль

О докладе

These days, test teams in many companies all over the world are going through staff reductions to the point where probably only a few or one tester is left. In my proposed session I will present and demonstrate some unconventional tool strategies and approaches. This is from the perspective of a lone tester working on fast paced projects with very little assistance.


00:04 Introduction: I’m a tester for 12 years

03:27 Before we dive in: about what this interactive presentation is

05:06 There is just no silver bullets at all for anything

05:49 I can give you the tools in the understanding

06:10 A little funny story

07:23 Some tools that the presenter is familiar with: Docker, Robot Framework, etc

11:08 Before the lone tester shows up the projects and development team

13:15 After you jump into one of the biggest projects what you're going to notice

13:56 Some suggestions about “Omega Tester”: testing with a team of one

15:53 Presentation examples

Поделиться

Alright so as I was saying, 00:04 Priyanka made some great slides and their beautifully well orchestrated slides and the way that these slides are structured there for multiple. 00:06 Audience is there for audiences that. 00:15 Armor of along the lines of if you teach them. 00:19 Or show them actually how to fish. 00:22 They can feed themselves for a day so with these slides, 00:25 I'm hoping to teach future testers. 00:29 For a lifetime, so I am a mind of. 00:31

Show people how to teach themselves and then they can learn through their own journeys how to use certain strategies and how do? 00:35 Self heal during events where I'm the only one left. 00:43 And no one else is around to show me. 00:48 How this works or that works so that's the one of the main goals of this so let's start off. 00:51 With introducing myself, my name is Joshua Grass feed. 00:57 This is test tool strategies for loan testers. 00:59

And again so short introduction, 01:02 it's not short actually it's pretty long and worded. 01:04 I've been a tester for roughly 12 years. 01:08 I've worked in agency in product companies, 01:11 originally came from background of working with very formal and traditional test teams. 01:13 And when I mean, formal this specifications were given. 01:17 And cetera, etc during the last 3 years of Mycareer and about yeah about 3 three and a half. 01:22 I became into mega tester. 01:28

So not by choice, it happened to me, 01:31 I started studying and teaching myself rapid software testing. 01:33 And when it was available. 01:37 I would read through James box. 01:39 RST PDF he took that down by the way 'cause. 01:41 He just recently did. 01:44 Eh. 01:44 Website redesignforsatisfies.com, I would often read through the context driven approach to automation testing and also the session based test management examples. 01:46 They're still there by the way you still get to those. 01:55

And I just started reading it to the big read. 01:58 It's a lot of knowledge to digest and I don't. 02:01 Recommend you to just digested in one weekend just take your time. 02:03 Lessons learned in software testing context really approach. 02:07 It's one of my favorite self study books. 02:11 So I enjoy experimenting with test tools. 02:14 I like concerning myself simply tester. 02:16 I'm very proud of that and. 02:19

Even though my my current role can't really mention is where it is right, 02:21 I work right now. 02:27 They work as a test engineer. 02:27 So I personally believe that automation is a part of the testers toolbox. 02:31 It's not the sorry. 02:36 So that that just give me a second. 02:36 Apologize this is actually. 02:42 Nice. 02:46 Try to keep 02:46 So I believe that the testers toolbox is very it's very. 02:51

It's very important to know that tools are not all that testing is about and that's what I try to teach. 02:56 Young testers when they? 03:02 Come to work with me or if I have to mentor a person. 03:02 To take over something? 03:08 So a funny fact is I looked up detective scientists. 03:08 That's where I feel like as a tester how I feel I found this stuff. 03:14 Will NASA? 03:18 Article. 03:18 Detective scientists it's pretty interesting. 03:18 So I have to keep touching. 03:26

All right before we dive in I just wanted Jeep clear about what this interactive presentations about FY I the. 03:27 The sketch schedule on test. 03:36 Leadership Congress has an update of the GitHub repo. 03:40 That's the companion repo for this presentation. 03:43 It also has the presentation slide so you can follow along read at your leisure. 03:45 I'm going to try and job through the slides so I can show you all the examples, 03:49 we could talk about it. 03:54 And then 03:55 Really. 03:58

Sorry. 03:58 OK, so one of the things that I want to make clear is. 04:00 This is not an exhaustive list of all the possible test strategies that can be applied to any project. 04:06 Or products. 04:11 Please. 04:13 OK and trying not to give you all silver bullets your project or product. 04:13 The unique and your tester journey experience career path may be completely different. 04:21 One of the one of the things that I really like from. 04:27

From the lessons learned in software testing is only you know the context of your work so. 04:33 You are the person that understands the underlying risks of what you're working on so I highly recommend you remember that. 04:39 As you go through these examples. 04:45 And evolving an ongoing experiment Vine this entire experiment is basically this GitHub repo that I've shared with you. 04:47 And also these slides that I constantly. 04:57

Update according to the conference, 05:01 I'm working with. 05:02 So. 05:02 Just keep an open mind, 05:02 I think that there is just no silver bullets at all for anything and. 05:06 Everyone is working on something completely different and I've come from. 05:10 I've come from companies where we work on POC's we work on all sorts of my previous company an intersection. 05:13 Used to work on several types of go sees and. 05:21 It was a very context, 05:24 switching was very common it's just. 05:25

It's hard to come up with test strategies that just apply to nearly everything 'cause. 05:29 Frankly, they just need to be. 05:35 Revisited re thought of. 05:37 Evaluate the risk. 05:37 Adjust accordingly. 05:42 So one of the things that I personally I'm a real believer in is. 05:42 Is giving people the tools in the understanding too? 05:49 Enhance your capabilities to work as fast as they can within their means and. 05:53

I've chosen tools that I think are very common place at the moment and we don't know the future. 05:59 We don't know if certain things like Docker Docker is a very. 06:06 Commonplace containerization, so and. 06:10 I a funny story is I was in a coffee shop 2 years ago and I was just doing some work, 06:10 and drinking coffee. 06:21 And I over heard these young gentlemen were talking about this is how you run. 06:21 This container here. 06:27 This I do this. 06:27 It's like this is that? 06:30

This is the stuff for example, 06:31 that the teacher taught. 06:33 I was like and I was curious. 06:33 I walked up to him and say, 06:35 I'm sorry like. 06:37 Where are you learning your doctor ization knowledge from? 06:37 Are you in a college nearby? 06:42 They told me know now we're in high school were in AP computer science, 06:44 I was like. 06:48 OK wow that's awesome. 06:48

How like this is great and just an FYII live in Bergen County, 06:50 so they were actually somewhere around Franklin Lacer That's a very wealthy area so. 06:54 It makes sense that they got Top notch uh as schooling and education in computer science that early so. 07:00 Just think about it the tooling and things we look at and say, 07:06 Huh well. 07:11 I don't really need daca right just. 07:11 Really all I need is a bare metal, 07:15 I just need 20 laptops. 07:17

Yeah, it's true today, but tomorrow. 07:20 I mean, who knows you know it's like the kids of today are learning the tools. 07:23 We kind of look at. 07:29 Today and say, I don't need that. 07:30 But just think about that. 07:33 So Docker is one of my one of my gotos. 07:35 There's several go to site it's just. 07:38 One of the things we covering often today, 07:41 an robot framework, so I unofficially kind of represents a group in. 07:44

The robot from Rick Foundation and because I believe in it, 07:50 so much and it's kind of changed my. 07:55 Outlook on automation versus testing. 07:58 And just the active checking that our tests do. 08:01 Is uh? 08:05 We can orchestrate that we can be. 08:05 Better testers enhancing our skills if we let our tools enhance our process, 08:10 not have our process and tools take over. 08:16 So I guess I'm very passionate about that. 08:20

Actually, I believe that generic test tools like robot framework where I can. 08:22 If I need to just automate a fully automated regression suite that just does checks that just do the same thing I expect all the time. 08:27 But that in the background. 08:38 I pull out another tool, 08:40 a hybridized sort of hybridized automated tool but waits for me me the decider. 08:43 Of this particular session. 08:50

Did it pass or fail and I have a number of things that I can implement in that particular session 2? 08:50 Either say, well, yeah, it did or didn't so robot framework is that it's an automation tool and it's. 08:58 Also, a one of the few open source robotic process automation solutions. 09:03 So if you are. 09:10 I've talked to people to people last night dinner alot of people are familiar with our PA. 09:10 So I highly recommend you look it up. 09:16

They're one of the leaders in open source RPM. 09:18 And UI Path is I'm sure it's a great tool, 09:21 it's a great piece of software. 09:24 It's uh it's also from where I come from right now, 09:27 it's not always feasible to. 09:31 By this license for X amount of money. 09:33 It wants. 09:36 Please look up the open source solutions parse that sort of thing now. 09:36 Many of these examples. 09:41 You don't need to use this framework for it's just a simple like. 09:41

I'm comfortable with it, I've created several prototypes of POC tools up with concept tools by the way. 09:46 I've been asked before what a proof of concept POC means and I have to make that clear so. 09:53 Maybe like to write all the code yourself and I'm. 09:59 That is awesome and. 10:02 I have to say that that's great for the people that can do that. 10:02 and I think it's a skill that I would love to improve. 10:08

I personally is for my own personal experience after you get used to the natural language syntax with robot framework. 10:11 You will be able to write. 10:18 Tests in Under 30 minutes or less. 10:19 It just it just makes sense to you 'cause it's using natural language, 10:22 it's using keyword. 10:26 Syntax and at the end, 10:26 you could also do interesting things like model based testing feeding in a model based generator. 10:28

Automatically generates a path for your test that that you're running either the background or for your own hybrid. 10:35 Uses so Let me know if some of this is familiar to some of you do you ever feel like? 10:43 It's simultaneous projects. 10:50 This is this is coming out there's there's 10:50 Fires everywhere there's all sorts of explosions and then at the same time. 10:55 It feels like this, it's the Wild West right so this is not happening first this then that it's. 10:59

At the same time so. 11:05 Before the loan tester. 11:08 All of you shows up the projects are there fast paced there frequently changing. 11:08 The requirements scope the client wants more features. 11:16 You have a client, who thinks that agile means you will get to production next week right? 11:20 That's what it means. 11:24 I've had that I had that like this my. 11:24 Uh my team uh and my uh I'm not going to mention where but 11:29 Mighty man my project product owner. 11:35

We're sitting down and they were like well, 11:38 they kind of expected this. 11:40 This feature to be done in 2 weeks or less so. 11:42 OK and then I looked at the developers like. 11:46 And like a And I'm like, 11:49 Alright well, then. 11:51 Let's see what we can do and they think of software development is just so you build it once it's right, 11:51 it's done. 11:57 It's done right, 11:57 yeah, so I never need to call you for any help so OK. 11:58 Give me all your documentation. 12:01

It's a furious when they see Oh my gosh, 12:03 there's a bug in production. 12:06 Just like you, said You You tested it yet and test it. 12:07 You have too many meetings. 12:11 Did you have possibly more than just your team working on this specific? 12:13 Project for this specific irey client so. 12:18 You're waiting for that company that's working with this other company. 12:22 They had that other piece, 12:25 they're working on. 12:27

There there like you know, 12:27 we have the easy thing, 12:29 so let's just let these guys you know they're going to. 12:30 I'm all over the hard back end work this and this and this and this they're going to do all the. 12:32 All the platform connections for us, 12:37 we're just going to do this. 12:39 No wait. 12:41 So I've I've seen it all. 12:41 Your development team is probably spread out across several projects like I said, 12:45 is there several months into their development. 12:48

They probably have a Wild West approach to tools into just check their work so. 12:51 They use different types of means to check that things are working as they expect they can always say OK. 12:57 This this is ready for you deploy to QA. 13:03 So have at it. 13:07 And maybe they're only doing it on the machine, 13:07 so in those kinds of worlds and now this is a. 13:11

This is just a Food for thought after you jump into the one of the biggest of those projects you're going to notice. 13:15 Very quickly. 13:22 Anything you do mean something else is or is not going to get done. 13:22 If you're too busy to planet prepare. 13:28 You feel too busy to mind your infrastructure. 13:30 Now your infrastructure at this point would probably be a text document. 13:33 That is like? 13:38

That long of just taking notes taking meds transcribing what you wrote down in meetings and transcribing it back into this. 13:38 Humongous document with just that project you have other text documents for other projects that are just doing this. 13:47 Doing this doing this so. 13:54 That that might be your infrastructure in like the first three or 4 days right so these are the challenges that you. 13:56 Mega tester. 14:03

Your face so this is all from JS box and they test your PDF I highly recommend you read it if it's still. 14:05 There, I think it's it's one of those PDFs that never. 14:12 Thank goodness 'cause it's it's uh. 14:16 It's a treasure for future testers senior testers to. 14:19 Look at it. 14:24 Right. 14:26 So some of the suggestions that change box. 14:26 Mega tester testing with the team of one, 14:31 the the several things that stood out to me. 14:34

The test activities in the testing story think about that. 14:38 Your strategy in terms of what you need to do what you need to create in order to do it. 14:42 Focus on the activity. 14:48 Uninterrupted sessions. 14:50 Use those laws those like just so I I mean, 14:54 I can't imagine what a lull is for one person versus the next but. 14:57 I've seen a lot less like I got 10 minutes, 15:01 So what can I build in 10 minutes? 15:04 Or wow, I got a real. 15:06

Blessing today I get an hour So what can I set up in an hour use that for preparation? 15:08 Reassessment. 15:14 Think about your strategy. 15:14 Use the recording tools to shorten bug investigation time. 15:17 So. 15:21 Think about all this as a? 15:23 Take that all in while we walked through these examples insights, 15:26 so before we move on. 15:30 If you have a computer. 15:31 You don't need a computer I'm here. 15:32

All day long, we can run examples together, 15:33 you could sit down in a corner, 15:35 but in between breaks in between sessions. 15:37 Examples with you, I'll show you some stuff. 15:39 We could we could do that, 15:41 if you have computer today, 15:43 if you have windows. 15:44 I recommend these. 15:44 Command line tools for git. 15:47 And also Cygwin or baboon. 15:50 Alright so getting to present presentation examples. 15:53

So Part 1 is adapting to frequently changing projects Part 2 is enhancing those existing tasks taking your process. 15:57 Further. 16:04 And one of my favorites is. 16:04 In Part 2 my personal favorite actually because when you are firing off your processes you are firing them off. 16:08 On your local via a remote process using a webhook. 16:14 Either way. 16:18 You have your right in front of you, 16:18 your hybrid tool that is. 16:23

Your brain machine did it pass or fail and have all these supporting tools that all some some interesting things as well. 16:25 And Part 3 is to visualize that whole journey put it together, 16:33 one of the things that robot framework. 16:36 Is that is? 16:38 Combining all test results under 1 pane of glass one reports one log. 16:38 Everything either running in cereal or running in parallel if you will have a very fast test process that. 16:46 Finishes in a minute. 16:52

I think that it's probably better for communication purposes to have it run in serial so that. 16:52 All these steps happens in under a minute OK. 16:59 That's great now versus it took 20 minutes. 17:02 All cops in one month. 17:05 20 minutes break that out into X amount of parallel cases and have that or sweets. 17:06 I should say it's actually a bigger than cases. 17:10 Had them all run in parallel across processes, 17:13 then since we're talking containerization. 17:16 Containers. 17:19

I've been at Kubernetes pods anything you need less ECS. 17:19 Whatever you can imagine so and the beautiful part is. 17:25 Robot framework comes up with this reusable output dot XML log. 17:30 And it also spits out X unit results, 17:34 too, so you can merge those all together with all the other frameworks. 17:36 That your team is using so you're all working under 1 pane of glass. 17:40 There's a polyglot team everybody likes working under like maybe Python. 17:44

Some guys like Scala, some guys like Java. 17:48 We could all come together talk through one pane glass, 17:51 so that is actually a another benefit of robot framework. 17:54 This together and people who are more comfortable with creating powerful strategies can just take the keyword approach, 17:58 maybe use some obvious testing which I'll show. 18:05 And I'll try to move quickly 'cause I got halfway through. 18:08 Alright so this is again. 18:11 It's a block of text don't worry bout. 18:13

These are commands that if you have a computer. 18:14 It just pull it off. 18:17 Run run among the running on your computer on your local if you have all this. 18:18 Every technical requirement installed don't worry about this. 18:22 And at the bottom says will answer questions right at the end, 18:25 so we will do that towards the end. 18:29 OK, so this part Part 1, 18:32 so tool strategy examples for adapting frequently changing projects. 18:33

So imagine in your team you are 18:39 introduced to a team that it's a hypothetical project your developers working on projects, 18:41 they decide to use tools that they're comfortable with confident in 18:47 and the loan tester just used to learn as quickly as possible so you could be contributing to the jury task that 18:51 request. 18:57 It would be helpful to reuse. 18:57 The approaches of the developers or try to set up a CI pipeline. 19:01

Process with your Work in there as well, 19:06 using whatever you want it doesn't have to be darker. 19:10 It could be of course. 19:13 Install the build machines. 19:15 But wherever they may be. 19:15 Now robot framework makes it easy to achieve that, 19:19 but I'm sure it also anything with a process runner. 19:21 Like let's say for example, 19:24 Postman Newman tests alot of I mean. 19:26 Making broad assumptions. 19:29

I've a lot of teams I personally worked with Postman is actually very popular right now for API testing. 19:29 And on Top of that it's really good at API documentation information also offering swagger files all sorts of. 19:37 Fun stuff like that, so it's actually got a built-in test runner an test framework itself. 19:44 But it also has a CLI auction. 19:50 So it can generates a Postman Newman collection that you can reuse in your CI. 19:52 Also. 19:59

If you are open minded to it, 19:59 you could also use it too. 20:04 We run from other tools like robot framework. 20:06 One of the things possible does not do is randomize test running as a? 20:08 Now I may be wrong, 20:12 maybe some other work about work arounds for that. 20:14 But using the folder option that's Built into Postman Newman's. 20:17 Test runner you can run specific folders in any order, 20:23 you like so imagine you have a mechanism to. 20:27

Based on a model a visual model of your a graphical model. 20:32 Let's say of your API behavior. 20:36 Have that. 20:39 Drive what Postman is going to do so that another example is curl, 20:39 sometimes developers. 20:45 I hate these graphical tools. 20:45 I love to just run coconut coconut coconut. 20:48 There, I checked everything I'm good my stuff is good so you could also use robot framework for those. 20:51

Just a straight commands pull them in run them in your process so here's an example 2. 20:58 As a long tester, you're going to be probably working. 21:05 Another example here is maybe maybe you're going to be working in a place where you're testing activities. 21:08 Are all done by you, 21:14 you have to do it yourself? 21:16 You should be able to run your own write your own Python library. 21:18 Maybe developer will walk in and just say OK, 21:21 yeah, you need help. 21:24

I can help you do that faster. 21:24 They'll want to contribute to it. 21:27 So from from this, I am recommending to from robot framework standpoint, 21:29 I'm not entirely sure if any equivalent. 21:34 You have a developer help you write the Python library that is going to support the resources that feed into robot framework. 21:37 But then again, robot framework in reality can pull in directly this Python. 21:45 Python library itself without any need for any of the other. 21:52

Robot framework resources. 21:55 So a simple folder structure here is kind of what I'm implying. 21:55 Having that simple, as simple as possible. 22:02 Just these are the tests. 22:04 These are the resources that talk to these tests. 22:06 And are used by these tests. 22:08 So. 22:10 If you need any guidance or have any questions. 22:10 This is where the code list just have them look at the Python. 22:15 Later. 22:20

Have a look at the under resources and go straight there and say, 22:20 Hey, this is where I I need the most help. 22:25 Please help me out with this part of the automation code. 22:26 So. 22:30 Let's say, and this is a different example. 22:32 The opposite of that example is also true so they're like, 22:35 too busy. 22:39 Look, I'm too busy figure it out. 22:39 Find a way to create your tests but I can't help you. 22:42 Is I don't have time? 22:46

Which could be the case it could be that you need to get that feature out there. 22:47 It needs to get out there as quickly as possible so there are I believe about 80. 22:51 You're about 80 or so. 22:59 Maybe it's increased it has to have increased by now by much more by. 23:00 Great percentage more. 23:04 Open source supported libraries that from the robot framework community. 23:04 There's many of them, they're covered the whole gamut of visual automation. 23:11 Silentium. 23:16

API testing and they even have some libraries for orchestrating Kubernetes pods docker itself. 23:16 It's up, it's pretty vast Ann. 23:26 I really recommend people try to give it a chance and take take a look. 23:28 So. 23:33 So time. 23:33 Alright so I have to move a little bit faster here. 23:33 So for This part here. 23:42 Going to run some some. 23:49 Examples here. 23:52 Are you going to be juggling windows and it's going to be a little challenging? 23:55 In 45 minutes. 24:00

So I'm going to probably run only 2 out of each of these. 24:00 So here's an example of a power one example using Postman collection so. 24:07 In here this docker container run the Postman Collection. 24:12 Read the results. 24:21 Show the status. 24:24 Just 200 OK. 24:24 And it finished in under a minute. 24:28 9 seconds So let's do another one real quick and I'll jump into Part 2 and Part 3 will just have to play it by ear. 24:31

So that was the Postman I could show curl, 24:38 but I'm sure a lot of you know how curl works. 24:41 This is a Python library. 24:43 Example right here. 24:46 So, in this Python library simpler accomplishing the same thing in a different container. 24:53 I've built the containers for so that it. 24:57 Runs. 24:58 And under Minute 12 custom in it, 25:01 so it was really fast. 25:04 We're going to go up here. 25:05 It's running in cereal. 25:07

And it's running in parallel parallel goes parallels you like robot framework can handle it. 25:07 I highly recommend running parallel, 25:12 but also don't. 25:14 Lose yourself in the parallel runs 'cause you'll have like. 25:14 So many tests to kind of like manage now you have all these X test managed so you have to. 25:18 Design your tests accordingly. 25:24 So that's a quick example for examples of from Part 1. 25:24 So let's move on to. 25:31

R2 so Part 2 the strategy is an I'm not going to spend too much time here. 25:35 All of those examples all the examples. 25:41 I was mentioning from before now we tie in some. 25:44 Watching processes some notification processes, 25:47 tying this into slack as people love to use slack there addicted to slack. 25:50 So now, what if I had a process now that talks to people through my tests through my hybrid tools for my automation. 25:55

And if talks to them directly as soon as something fails, 26:03 so that's basically what this slide is talking about. 26:07 And I also have an example. 26:11 I'd love to share with you, 26:13 that as not was not in my Robocon. 26:15 Workshop. 26:17 Triggering webhooks for your wherever they are remote test processes in ECS in AC2, 26:17 another physical machine. 26:25 This is a It's a very uh. 26:25 It's a very useful example and again not limited to robot framework. 26:30

You can just do this with any tooling you like any framework you like. 26:34 So let's go forward a little bit going to speed this up so we could just get to the examples. 26:38 So for this example here I am doing a combination of approaches that. 26:43 I'm just going to basically show these tests that fail on purpose, 26:49 so that they could talk to the team through slack. 26:52 And there could be orchestrated by me or they could be orchestrated by a docker container. 26:54

Run by me locally running a docker container. 27:01 So I really wanted to show this to everyone. 27:04 These are some. 27:07 Get log commands that I'd like to run for you. 27:07 Again, those slides there's nothing different that I'm showing you that are not in the slides that I've already shared with everybody through the? 27:13 Sketch.com. 27:20 So I have here has anyone here, 27:22 not heard of mule or mulesoft. 27:24 Our Salesforce. 27:28

Yeah, so Salesforce bought you'll soften this is their repo and just going to show you. 27:30 Quick. 27:36 I love love their repo this is their current community edition product. 27:36 So I'm going to quickly show a technique engineered at Google. 27:43 They don't use it anymore. 27:47 But it's still kind of useful for most people who love to test. 27:49 Sorry style and just want to know where the hotspots give it to me real quick so. 27:54 Honing in on using git log. 28:01

Where are the bug fixes? 28:04 Where are the fix is done? 28:05 What commits were bug fix commits? 28:07 What etc etc etc? 28:09 So I looked at just now 15 weeks worth of commits and that's that's what came out. 28:09 So that's a quick example. 28:17 I wanted to, well, too much on that have a bunch more to show you. 28:19 And the OK, so. 28:23 Right let's jump into the slack notification examples. 28:29 Here. 28:37 So with this example here. 28:47

Going to run a container it's going to fail some tests on purpose. 28:50 This is the wrong one pardon me. 28:55 What that other example did it ran randomized tests with friend my sub in terms of randomized test data? 29:07 So. 29:13 It's kind of hard to juggle these 'cause. 29:13 There's a lot of examples. 29:16 An I wanted. 29:17 I want everyone to walk out with something. 29:17 Today an This is the example with pass and fails and it also measures the time of how long each test ran. 29:20 So. 29:30

It says here 22122 milliseconds, 29:30 so there's a listener that also catches how long this is taking and also will fail, 29:33 it if it takes too long. 29:38 So from here, my process caught numerous failures in Python Library, 29:42 one and 2 example. 29:47 And so at this rate reflects so let's imagine this is the team slack channel. 29:47 So Oh Wow, Josh is uh. 29:53 Josh is tools caught something in the regression suite so let's talk to him about you know better service service so. 29:56

If you use your imagination on how you could use slack notifications. 30:03 OK so from here time check. 30:11 There's a little bit there's a lot more. 30:14 I need to show I'm going to jump into the graph Walker example. 30:16 Now, before I run this, 30:20 I would love to show you graph. 30:21 Walker is a model based test tool. 30:23 So this is an example of one of those models. 30:25 This is a model of a positive run so this is just going to do a simple walk. 30:28 Works. 30:33

Patch Post Lites four and a JSON placeholder endpoints, 30:33 so and the check is on the get so there is also the. 30:41 Negative testing path, which tries to break everything and also delayed the request by maybe 5 or 6 seconds. 30:47 I forget how long? 30:53 Here is the example running. 30:53 South. 30:59 OK, so this is an example of that of those models. 31:02 Running those parallel paths across split out across 2 robot framework suites. 31:07 So it's going to. 31:13

Generating the path 31:13 Generated the path running the path. 31:18 Now these tasks here. 31:21 These these checks. 31:21 They're not running in a sequence. 31:24 They're running a randomized sequence built by that graph. 31:26 Built by both graphs, so they run. 31:30 In parallel an also in Asia. 31:32 I should say actually this is this is all running in cereal, 31:36 my fault and my my error. 31:40 This one is running a very long path generated generated by the? 31:41

Negative and the positive so it pulls out together in one path. 31:45 Feed that into robot framework and you have a very quick way of setting up model based testing. 31:51 So. 31:57 This is a very powerful way of driving your tests. 31:57 I think that it's something that you should consider. 32:01 And it's based on a model of your systems behavior so this is actually one of my favorite examples and. 32:04 Should have a failure baked in there? 32:10 There there was. 32:13

And that will again go straight to slack. 32:13 So that your I wouldn't say exploratory it's actually technically just a randomized walk. 32:18 According to a model so your randomized walk from that model might have caused. 32:25 An edge case to show up. 32:30 And also having that running against a tool that slows down the randomly slows down the? 32:31 The connection will also help those in case just to check well. 32:38

How does it handle slow connections whatever it is you're testing is? 32:41 So doing acting closer time. 32:45 Alright so another example that I wanted to walk through. 32:48 But I needed to kind of to train this other tool that I wanted to show. 32:53 Going to jump ahead, a bit because we are almost. 32:58 Skipping ahead. 33:02 Alright so I 33:02 Found this tool by chance, 33:06 so we're enhancing our test in our process is there is a tool called get risky. 33:08

That will show you the risks of each commit so it uses supervised classification based machine learning to read through your git log metadata. 33:12 So this, this tool is actually very useful before you get into the hybrid example, 33:23 then I'll show really. 33:28 Hopefully I will get to your questions. 33:31 So this example here. 33:34 Is going to be based on my previous run because training this takes an hour so the model is built off of the git log metadata. 33:37

So here's an example of a commits that made all of these changes. 33:46 This is based off of git. 33:50 Command line. 33:52 This comes straight from get there is no code, 33:52 there is nothing that I did to make this appear this way. 33:55 This just comes from Git log the basic functionality of get log. 33:57 Now, this part here is good risky. 34:01 So it risky after training that model. 34:03 New that Hey this commit is other itself score of .6, 34:07 so think about that. 34:11 Josh. 34:11

Why would I why would I want to pay attention to that? 34:13 Well, I want to pay attention to that? 34:16 Over something that's 0. 34:18 Why I wanted to definitely pay attention to something that's a .9 out of one? 34:21 So think about that as you're exploring. 34:26 Put into the back burner of your brain. 34:28 Maybe I want to check this committee. 34:31 I know is committed on that date. 34:34 And with get log I can also find out food. 34:36 Let's just not done. 34:40

I don't want this to be something that finger pointing at tool. 34:40 This is just something for testers to have in their brains. 34:45 OK, I have this knowledge where could I go. 34:48 Where can I find the risk? 34:52 Where can I dig in and detect these problems? 34:54 So let's move a little bit faster, 34:57 I might not have time for the Charles proxy examples. 34:59 That's actually one of my actually religion today. 35:02

Now it's starting Charles proxy for me, 35:05 this is all run from robot framework. 35:08 I am starting Charles proxy up. 35:10 OK. 35:12 Of a real device here. 35:12 Now. 35:22 Sorry. 35:26 A lot of juggling. 35:29 Robot framework gives this convenience. 35:29 Prompting tools so I'm testing for the mobile changes. 35:36 Etc. 35:41 That rhyme with quickly because this is actually. 35:41 Just trying look like. 35:51 All right, 35:51 so this is a tool based on the basics of its interpretation. 35:53 It is not supposed to be. 35:58

The like one true method of doing such an based testing. 36:00 I interpreted this way, and feel free to try out your own implementation as well, 36:05 you know. 36:10 Fill out your 36:10 Your your messages passed through robot framework will end up in the robot framework log which I will show. 36:14 As soon as I'm done with this example, 36:19 so I want to fail because I found that the. 36:21 Mobile. 36:24 Kohl's. 36:27

I'm running out of time, 36:30 otherwise I could show everyone how Charles proxy works specifically later on, 36:31 because it's kind of running low on time right now like to move on. 36:35 Yeah, visualizations. 36:38 Are wrong? 36:41 Showing errors. 36:41 So. 36:45 OK. 36:49 Mobile homes. 36:51 This is an important part. 36:54 Show. 36:57 500. 36:59 So now that was orchestrated by myself and. 37:01 Since my my team in my organization, 37:05 Love Slack, so much. 37:08 I could talk to them directly from my tool. 37:10

Have all my docker containers running the background fire off all sorts of cool commands, 37:12 which I'll show right here as well. 37:17 I think that's a very useful example run. 37:20 Spoke. 37:24 2. 37:27 I've got a Crow commands running I've got my hybrid tool process running I am. 37:31 By myself doing this. 37:36 Might as well communicated in real-time concurrently with the team, 37:39 so that they know in real time? 37:43 What's going on. 37:44 So here's the. 37:44

My fault 37:49 Go back to the slides lot to show. 37:55 5 minutes ago 38:00 This is 38:09 An I could show the I could show the visualizations in. 38:12 OK, well for example, here. 38:23 So. 38:28 I have left. 38:28 You could also set up your test process as a service that anyone can call from anywhere. 38:45 So wherever this. 38:52 Is running from running wherever it is in AWSU C2? 38:52 That comes out the other end. 38:59 This side here is wrong, 39:02 wherever it's running from container ECS EC2. 39:03 Bare metal machine. 39:07

Cause it. 39:07 This is somewhere else. 39:10 This is from the local machine. 39:10 And and of course. 39:16 That. 39:16 That result is not wasted it's immediately shot off through slack. 39:20 To whoever needs to see it. 39:25 If I could find it. 39:28 So. 39:31 Again, same example from before triggered by a webhook. 39:31 Wherever that Webhook is talking to. 39:36 That will put could be uh. 39:38 I'll check It's Jenkins CI Machine, 39:40 it could be. 39:42 Doing it as post process trigger it could be someone in the. 39:42

Someone from working remote's Indiana. 39:48 Wherever there from and any country aren't enough weapons. 39:53 It just needs to be. 39:55 You need to be able to talk to that web book. 39:56 So. 40:00 We fast forwarded a bit. 40:00 Chords the end here. 40:05 And I'm probably not going to talk through the Appian example right now because we're at time. 40:08 Once you give time for people's questions. 40:15 So this is basically Part 3 is visualizing the testing story. 40:17

We get combined results here and I want to show everyone the. 40:21 Process of of combining everything under 1 painted glass. 40:26 So when I'm what do I mean by that. 40:30 So if I take this. 40:34 Which was wrong recently I I would run the command but it's just more effective to just show everyone? 40:36 There. 40:41 My hybrid process. 40:41 Can be deployed to a heroku? 40:41 Environments. 40:48 Or wherever you like to be something in an S3 bucket if you like it's a static HTML file. 40:48

So hard to read here, 40:58 so I'll just scream. 41:00 Yeah, 41:00 there, we go. 41:02 So this is the process I ran last week. 41:02 I got to see how long each session took. 41:05 What I was doing at each session? 41:10 How long it took at? 41:12 Each and every single piece of that session now. 41:14 You could cut that down. 41:16 You could trim it all down, 41:18 however, you like. 41:19

The The Robot Framework Text Dialogue Library is completely customizable and also open source so feel free to hack at it at your leisure. 41:19 So it doesn't need to be all these steps. 41:29 Most testers like to be accountants. 41:33 They like to have every single thing I want to measure every single. 41:35 Step away or maybe you want. 41:38 3 prompts boom boom bum. 41:41 You could do that with robot framework. 41:44

You do that with any framework really that handles text prompts I don't know of any that does text props that way. 41:47 Or drives one of these one of the fun things about this process that I personally enjoy I've dropped this in the hands of a developer. 41:53 I kind of turned it down and made a little simpler. 42:00 It actually helped them test better it made them focus on what it is I wanted them to do. 42:03

And they automatically talked to my test real's management, 42:08 Sis. 42:12 So they were automatically helping me with my work. 42:12 Closing out all these test runs closing out to test real test runs opening and closing them and at the same time talking to slack. 42:17 So that's that is multiple multiple multiple things done concurrently. 42:24 All in one session and I just gave them my package and they ran it and worked with it. 42:28

So that was one of the best parts of this, 42:34 this process. 42:37 So that is, 42:37 I mean does anyone have questions about Heroku. 42:40 I think that's pretty obvious what it does right. 42:43 It's It's built on Top of a WS it's owned by Salesforce. 42:46 Love Salesforce by the way I know a couple of friends of mine and the robot framework foundation are from Salesforce and they're amazing people. 42:51 So visualize your test process. 42:57 Under the same pane of glass now. 43:00

The pistol results. 43:04 What does that mean? 43:04 Gather everything together you could probably make a prettier one. 43:10 I highly recommend you do, 43:13 I'd. 43:14 Love you to show me your work? 43:14 Show me what you've. 43:16 You've accomplished. 43:16 Make a better dashboard. 43:20 Like this. 43:23 Make it easier to read. 43:23 But no one can argue. 43:26 But this part. 43:28 My favorite part. 43:28 Hey, I found something. 43:31

It's it's important to visualize that for the people in the boardroom serve whatever important meetings. 43:34 You need to join in. 43:40 Here's a URL. 43:41 Here's what I found here, 43:41 the metrics. 43:44 I can show you the whole thing if you like. 43:44 Sir Madam. 43:49 Well, we gotta do a meeting for that because this is the journey is the journey. 43:49 Etc, etc, etc. 43:54 You could also collapse, 43:54 it down. 43:57 It's very easy to navigate through. 43:57 He's gotta drive some law. 44:00 Scenarios down. 44:02

And I think that's 44:05 The Internet. 44:09 Which I think? 44:11 And references in case anyone's got questions while I was in it. 44:11

Комментарии для сайта Cackle

Купить этот доклад

Доступ к видеозаписи доклада «Joshua Gorospe - Test Tool Strategies for Lone Testers»
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно

Access to all the recordings of the event

Доступ к записям всех докладов «Test Leadership Congress»
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Билет

Интересуетесь тематикой «Разработка и программирование»?

Возможно, вас заинтересуют видеозаписи с этого мероприятия

29 августа 2019
Москва
7
39
геймдев, игры, киберспорт, медиа, разрвлечения

Похожие доклады

Lina Zubyte
QA Consultant в ThoughtWorks
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Sami Söderblom
Testing & DevOps Coach в Elisa
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Mike Talks
Test Manager в Datacom
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно

Купить это видео

Видеозапись

Доступ к видеозаписи доклада «Joshua Gorospe - Test Tool Strategies for Lone Testers»
Доступно
В корзине
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно
Бесплатно

Conference Cast

ConferenceCast.tv — архив видеозаписей докладов и конференций.
С этим сервисом вы можете найти интересные лекции специально для вас!

Conference Cast
1105 конференций
29928 докладчиков
13533 часа контента