Thoughts: Test automation strategy

I am a bit behind the curve on this I am sure but the free platform Test Automation University by Angie Jones is pretty great. There even is a more high level short course on test automation strategy. I was super grateful for this, because of a few reasons which I will cover in a moment.

Why was I looking at the platform in the first place? Well we have objectives around test automation. (A discussion on objective setting and personal development plans could actually be a whole series. I still very much refer back to Stephen Janaway and what he wrote a while back on the subject and 1:1s. Read the whole series, it is great!)

Don’t assign objectives or tasks if the person doesn’t have time or isn’t motivated

But back to the objectives. Now it is pointless assigning someone an objective or personal development task when they are not motivated to act on it. If someone does not want to learn to code or cannot put in the time for whatever reason, then there is no point in them needing to achieve actually writing some code. At the end of the day we want our teams to succeed and feel like they are growing in their role. Now for some that is simply being a valued team member on their team. But we do like pushing people and letting them explore their role from different angles.

As a company approach we want to aim for continuous delivery where feasible (regulations), which means more automation. But not many of us can or want to learn to code.

So we decided to split our automation focus and take on the approach of not just doing but also understanding the whole approach. This is something I really liked about the automation in testing approach by Richard and Mark. It gives test automation another dimension because it no longer is about automating away testing tasks such as regression but also understand why you are doing it and what other things it enables you to do and how it supports testing activities.

Test automation is a whole team approach

Sometimes though these sort of words need to be in a course or some sort of material for people to accept it and listen to it. So finding Angie’s course on test automation strategy makes the whole subject of test automation way more approachable. Most of us testers lead scrum ceremonies or meetings. We have to be good communicators, so armed with some great questions from that course, anyone can help lead a discussion on a test automation approach to make sure it becomes a whole team approach.

Not being a coder can be such an advantage when designing a test automation strategy

Not being a coder can be such an advantage when designing a test automation strategy, not a disadvantage. I have seen people feel intimidated and scared but all the technical details, but being a functional tester often means you have a whole picture of the application and can really help the team automate the more risky parts of the application and the most valuable to the business.

I found that functional testers can easily feel less valued than the ones who can write automated tests and by also focusing on understanding test automation it made this objective a lot more approachable in my mind, and shows that you can still sit at the automation testing table, even if you cannot code the test. It might even free your mind. You don’t have a predetermined level of “automating that is hard so I won’t even go there” mind block. You are free to ask all the questions around what is possible and then is it feasible given the estimated effort.

So high level what do you need for a test automation strategy? Well watch Angie’s course. ­čśÇ

My summary of the first video with some of my own thoughts below.

WHAT?

A goal! What is your goal? This made so much sense to me. This can help you identify where are automation efforts are most value-able.

The most common goal I have seen is to reduce regression testing time. But here it can be important to then not jump to conclusions. Just because regression testing happens by manually clicking through the UI, it does not mean your tests need to do the right thing. Be clever about which layer is gets the automated tests and what your regression tests actually check and think about the least costly design that is most effective and build on that.
WHO?

Agile Testing: Frustrations can make us creative

  

creative space design of a restaurant

I love the fact that I have worked at a few different companies and hence have gotten to know so many awesome people.

It also has the great benefit that I still get to take part in different conversations around Agile Software Development practices and especially when the focus is on testing.

The other day I was forwarded a little article by Mountain Goat Software which focused around the fact that in an agile team, at the end of the sprint everyone tests. And some developers may not like manual testing so they will do automation testing or pair with testers to find efficiencies.

“At the end of the sprint, everyone tests.”

I am wondering a few things around this line of text. Why does everyone only test at the end of the sprint?

I know there have been many discussions that “testing” means manual testing being done by a human. But I do think that developers are often using Test Driven Design to write code and hopefully are also running the code they write to test that it does what they intended and maybe also how it does not (depending on experience, mind set, time pressures and learned habits).
Nevertheless the above quote is important for agile teams. I have experienced that as testers you can easily become a bottleneck because the developers think their part is done. Ideally they will have written the code and written automation tests as well. Now the manual testers get to test the new feature works and hasn’t broken anything.

But this leads to the problem that the developers are sitting there twirling their thumbs or starting more work, adding it to the bottleneck until the pipeline completely collapses.

So I like the fact that there was an emphasis put out that everyone should test as the team is responsible for shipping the work altogether.

I do hope though that testers and developers in a team get to test early as well, or maybe even designers or product managers, to make sure that the feedback loops is as fast as possible. The statement has the danger to be interpreted as such that testing should only happen at the end of the iteration. (I don’t think that is actually what they are saying.)

The excerpt I read actually states that the better the agile team the later the testing can occur. This seems wrong to me, but maybe I am taking it out of context and it is referring to the time needed for the whole team to be testers.

 

“Establishing a mindset of we’re all testers at the end, does not mean that programmers need to execute manual test scripts, which is something they’ll probably hate.”

 

The article went out to say that developers may dislike using manual test scripts for testing and hence maybe could focus on other things such as automation scripts.

I actually thing that it will be really beneficial for developers to do manual testing, not necessarily follow test scripts but explore the features they wrote instead.

By all means make sure your code is well tested using automation tools but you will not know if you have written the right code if you do not manually test it. You will only know that the code is written in the right way.

 

Frustrations can make us creative

I recently watched a video that Chris recommended on How frustrations can make us creative and it highlighted how doing something you maybe don’t like can actually result in you finding ways to make it better a lot easier than doing stuff you enjoy all the time. So developers definitely can really gain from doing some manual testing. And not just developers but hopefully the whole team.

A sort of related example of this is, that at Songkick we try and learn about what each department does and were encouraged to help the support teams. This has actually led to improvements being made to the admin systems when developers used it to help solve support cases. It is OK here to JFDI, when an improvement can really help someone.

A small example was that someone added extra information to be pulled in on one screen, so that you do not need 2 tabs open anymore to find all the information you need to close a support case. This was a huge time saving for everyone and was only achieved because a developer used the tools he created for the support team.

So I encourage everyone to test, collaborate and try something that frustrates you and see if you can make it less frustrating to do.

 

 

Brigh#Testactually and terrible UI tests

It’s been a big quiet here recently and this had to do with me changing job and trying to get used to a commute. Luckily our meet up didn’t suffer and thanks to the co hosts we had another great evening.

This time we had a talk on terrible UI browse tests in selenium and how to not write them in a terrible way.

 Rich the speaker, in my opinion made a great point that got me thinking a little bit further.

He mentioned the automation test pyramid and how at the top it always states UI tests. However what it actually means is end to end tests run through the UI that test the full stack of code.

You could even argue that end to end tests are a sub form of UI tests because really UI test do not need to be run against the full stack of code if you mocked the API calls and other third party integrations.

So the UI could be written with unit type tests from day one run on the CI server if it was a separated layer.

Do we distort the message of what types of tests we want at the top of the automation pyramid by letting us be boxed in by its shape?

I think we shouldn’t necessarily just put UI tests at the top. It should be end to end tests run through the browser. Is it just me that also thought UI tests have to take a long time, be brittle and use the full stack?

It hadn’t occurred to me that if we separate the UI layer and apply a few other tricks we could test just the UI fast and free up manual tester time even more to explore those edge cases.

Do you think we boxed our definition and understanding of what UI tests are in due to the shape of the pyramid?
The slides from Richs talk