Agile Testing Days - Day 3

Yesterday, after the last keynote I went to look at the car building party, which looked awesome and chaotic at the same time. People were really into it, and looked sweaty and excited. I went to the bar with a colleague of mine to have a nice talk over a glass of Talisker.And that was the only alcoholic drink I had that evening. One hangover a conference is enough, thank you very much.

OH, and I had my copy of “More Agile Testing” signed by Janet & Lisa, woohooo 😀

Then I went to bed at the most reasonable time ever, but I have my own consensus talk today. I wanted to feel rested for that… Nerves will kill me later today. 

Keynote Antony Marcano – Don’t Put me in a Box

I haven’t read anything about this talk up front and the title makes me really curious! 

He goes around in a public asking them “What do you do”, and people will answer with “I am a…[x]”. And you’ve already put yourself in a box. It limits the agility in our teams. When you say I am an agile tester you probably say that because you are proud of your identity. 

#noslides. Makes it harder for me sometimes to do the liveblogging. Oh well. 

Your salary is partly based on how awesome your job title is. “Grand wizard jedi software guru”. That would cash pretty well. 

If you tell others you are a tester, they’ll expect you to do the testing. But what if you do stuff that doesn’t relate exactly to your job title? Antony prefers not to have a title. He wants to be part of a group that shares a single responsibility. “I kind of do stuff in software-like”. People want a short answer, that fits in a box. People like to make assumptions and want clarity. When in reality, people do (are?) many things.

In the past your role would be clearer. Antony gives the example about parenting in the fifties. Fathers would work and mothers would take care of the children and the house. Roles have become much more dynamic since then. 

Antony’s first XP experience was when he built a computer programme with his dad that helped his dad do a part of his work much faster. But then he went to uni and they told him that Waterfall was the way to go. 

Then he experienced Rapid Application Development (I have never heard of that, I’m only 28 excusez-moi). He wanted to become a system analyst. But first he had to be a Junior Analyst, aka a Servant. He does whatever the manager doesn’t want to do. When he went to look in the code, the manager didn’t approve because that wasn’t his job title! He wasn’t allowed to touch code! (I really think it’s insane that things like this happen in the world and I’m glad to have skipped that phase). The manager wasn’t even happy that he finished his job in a couple of days (cleaning data), because he automated it.

He then went to work as a tester on an XP project in a law firm. But still, he wasn’t close to the developers. And of course, IT was stationed in the BASEMENT (Hello, Office Space?). He found bugs, but it would take a lot of time before anyone could pair up with him because the developers were somewhere else. And they went to the pub without him because of that. He decided to move his desk: awesome! It helped tremendously on everything: communication, pairing, coding…He became more of a test lead, and also developer. His team moved past the whole ‘your job is x’ and everyone did more parts in the process. 

He just wants to do awesome stuff. He can do more than one thing, so what’s the big deal. Screw those boxes (job titles), people have more than 1 skill. He dislikes the idea of ‘putting people into projects’. Well, I AGREE. And talking about people as ‘recourses’, HORRIBLE. People aren’t dispensable.  

The mindset shift, paradigm shift, is really important. I find it very nice to hear his (his)tory. I missed this history because I am still young, and I’m glad I missed it. I’m glad that I have a manager who trusts me and not tells me how to do my job. 

I love the energy from this talk, really good. He seems so relaxed when speaking, awesome!

A Test Management Christmas Carol, The Ghosts of test management, past, present and future – Ben Williams & Tom Roden

I have talked with these guys at the bar and during the diner on day 1 and they’re really nice. I think their presentation ought to be fun! 

We all got cards, which are obviously shameless publicity, but we also need them for voting later on, apparently. *wink*

They start of with a video of the test management past. Omg this is too hilarious to write down. Words fail me here.

“Tests at least a 100 steps long, you know, quality tests!”. That sums it up pretty well. 

Test Management Past in a nutshell: Utterly Immovable Live Date

We were trying frantically to estimate our testing efforts, given that they had to be performed in a time box (if you were lucky). And even when you said ’80 days’ to the project manager he would put 20 in the project plan. 

Smoke testing = smoking a cigarette

Exploatory testing = going out in a boat

Stress Testing = FUUUUUU

Penetration Testing = yeah, lets not go there.

Stay on the target, was very important!!

Step 1> Remove Brain. Step 2> Login to QC Step3> follow 67 steps  test script Step4>test fail because test step wasn’t doable Step5>Is it a bug, because it didn’t follow the script. 

Funny stuff, partly true, but lets get serious!

Agile is still on the increase, more people are trying to start working that way. But still a lot of fallacies about agile testing exist. In none of the agile frameworks a ‘test manager’ exists. So what does this mean? 

Where are all the test managers going, they aren’t vanishing into thin air right? The roles have changed! They did a survey at a population of IT-people and 90% said that responsibilities have changed. Decrease in: Planning, Reporting, Test Strategies, Project Admin, Project Management, Reviewing Documentation.

No Change in: budgeting, training, reviewing tests, metrics. 

(again, I see a parallel to my own talk, where I will breach upon the subject of test strategies).

How can test management change into the future?

– role will stop to exist

– managers need a different role, broader scope and different skill set

– teams need to be accountable for their test activities (more so than in the past)

Now we’re going to play a game: Inside or Outside the team!

Defect Management: everybody said “inside the team”. Although I hate the term ‘defect management’. In scrum, there is no such thing as ‘defect’, you either have a ‘user story’ or you fix the defect immediately. If you need defect management, you are either doing things wrong or you are working with many teams on the same product (that’s harder).

Test Automation: almost everybody said “inside the team”. Also a bit harder when you’re working with more teams, sometimes I see a team specifically focussed on automation in large projects. Tom and Ben show us a version of the agile testing quadrants and their practices when it comes to test automation. Seems familiar to me, and I agree wholeheartedly with their approach. 

Test Tool Selection: 50% said inside here. I waved my card to both sides, because again, if you’re working on a large project it’s not really feasible for every team to decide which tools they want to use. You need a general agreement on that. If you’re working in a small project, you might be lucky enough to wield complete power over your tool selection. 

Principles & Strategy: most people said ‘outside the team’. That surprises me, because I think agile is all about giving TRUST to the team. So trusting them to be able to figure out HOW they want to test. I personally want that responsibility to be in my team. 

Tom gives us the “Principles of Testing”, which seems like a sort of framework for an agile test strategy. It looks really good!

Teams are responsible for more! Is that a good or a bad thing? It’s a good thing if people have the right motivation, not if they hate their job. I’m thinking about the book “Drive” here. 

Chris George – Easing the Pain of Legacy Tests

Talked to Chris earlier on the day, he seems a nice bloke, and legacy tests ARE a pain in the a**. So I’m curious to know more. 

“Once upon a time” (there’s the unicorn again), a project had 250k lines of code, more than 10k regression checks and CI in place.

Now, the REAL story. The automated test were SLOW, 16 hours runtime! (OMG). They never all passed, they had random failures. This created a lack of confidence, so it would take longer to release. 

In his project, the testers were mostly responsible for the test code. It ended up being a little bit like a house of cards, it would fall apart easily. Instability was the reality. 

They came up with a plan to shorten the release cycle. The original planned time was 6 months. A guy with 30 years of experience thought it would be easier “it’s only code at the end of the day”. But Chris thought it would be better to not dive into the code. They started to analyse the failing automated tests (checks) to see if there was a pattern. It ended up being 3 categories of failures: infrastructure (issues with the databases, network), resources the tests required weren’t there, potential product failures.

He first went for the quick wins: 50% was easily solvable in the end. He was able to fix the resource problem first. 50% improvement already in 2 days!

The next challenge was infrastructure. The tests talked to a lot of databases and there seemed to be a critical number. Whenever more than 70 databases were involved the performance dropped. 

Also, if there were more than a thousand objects in the database a comparison test took more than 2 hours! Crazy shit. It took 1 week to improve this challenge, but 95% of the tests now passed! Great work, although the sceptic person in me asks “did you look at the value of the tests first?”. 

Now the developers could go back to coding, run the tests after they were finished and have quicker feedback. The release cycle would shorten (? no answer on that).

(was the goal to make the tests pass or to make sure you have valuable tests?) 

The developers started to care about the tests. They became leading in development. But 6 hours runtime is still a lot. The next goal was: get it to run in 1 hour.

Geez, this guy has BEAUTIFUL slides. Awesome drawings!

Now, they started looking at the actual value of the tests. Do you need to test everything all the time?? No, maybe not. They unshackled the tests from the database by mocking the database stuff they needed. 

Next step was to visualise the test progress reports for anyone to see. Chris was very proud of this achievement. 

— And then nerves hit me. I was up for a consensus talk at 2:30 and I’ve never presented at an event as great as this. So I had some lunch (couldn’t eat much), and then skipped the keynote to practice a little bit in my room and make some last tweaks based on the new info I had since this morning (Don’t put me in a box, test manager stuff). 

15 minutes before me talk I felt like I was dying. Then I actually delivered my talk, and David Evans, Ben, Tom, Eric and many more people were in the audience supporting me. I really so much appreciated it!! I think my talk went ‘okay’, could be better, but for a first time I was very happy to have survived it with a little confidence. I hope that next year I can make it as a ‘real speaker’ at the conference.  — 

Keynote Alan Richardson – Helping Testers Add Value to Agile Projects 

Ok, he’s beautifully dressed, let’s be honest. He looks like a pirate, which is awesome. 

Also, his voice is so relaxing!

On to the topic: When we don’t impact reality the way we want, we feel disillusioned…So what can we do? Work with the reality, rather than an ideal!

Testers adopt different belief systems, so we notice different things each time when working with a system. 

A tester makes other people look good! Lol 🙂

But don’t think that bringing donuts to your team is adding real value. We add value by doing stuff related to testing. We find problems, check acceptance criteria and so forth. We get comfortable working with the system under development. 

He survived Waterfall projects, omg!! He tried to remove waste, explore, not make huge test scripts etc. Waterfall teaches us how to work around the process. (SO true!)

“If I can handle Waterfall, I can handle any of this stuff”. 

Ugh, I notice that I’m waaaaaay to tired to give you a summary of this talk that actually does it justice. I’m going to just pay attention and enjoy it for now.