Thoughts: Tamagotchi design

I was talking to a colleague the other day and we were talking about some proposed tool and architecture that had been talked about, which seemed to need a lot of human intervention in its weekly, or monthly existence and it may even just randomly shit on everything. Figuratively and this is when we coined the term “tamagotchi design”.

Tamagotchi design is an anti pattern. So not one to follow. If you are not familiar with what a tamagotchi is:

“The Tamagotchi (たまごっち Tamagocchi?) is a handheld digital pet, created in Japan by Akihiro Yokoi of WiZ and Aki Maita of Bandai. It was first sold by Bandai in 1996 in Japan.”

The idea is that you have to look after it or it dies, or gets in a  mess. So you regularly feed it, clean up after it, play with it and cuddle it, maybe?

Do you regularly have to restart servers or micro-services?

What we were saying was, if you find yourself in a world where you have to regularly restart your servers or micro-services, need to check they haven’t lost connection to plug ins or maybe you have implemented a queuing system that needs a manual kick once in a while to keep on processing, then you may have tamagotchi design on your hands.

To avoid tamagotchi design, I would always try and have a plan form an architectural point of view and talk about points of failure a lot and how you will recover from those. Of course some playbooks may include stuff like restarting servers and sometimes that is inevitable but you don’t want this to be in your general run book.

Use an architectural diagram

Drawing or asking your team member to draw an architectural design is so useful. Then just take away the lines and ask what happens. Are there transactions in the system that can still happen if that line is missing? Do we know what the customer sees when this happens? If you don’t know, start testing and providing that information.

Can you also test your playbooks, if they will actually work?

If your system does require frequent human intervention, can these be automated? Can you create a feeding schedule so to speak? Or a cleaning up schedule? How about alerts so you can make sure the system does not get into a right mess?

Monitoring and alerting

How well are you monitoring your system? Do you really know what it needs to be happy and what happy and unhappy really looks like? Maybe you are playing with the system when really it just wants to be cleaned up? Monitoring can tell you a lot about what is going on and learning what happy looks like versus different unhappy states can be invaluable when working on a system.

The worst bit about a tamagotchi was always having to react to it. If you could prepare and feed it a bit earlier or play with it before it got bored, you would have a much happier pet. Alerts could do the same for you in your tamagotchi architecture. They could warn you that queues are getting full for example and you could try and remedy this early.

Do you work with a tamagotchi design? I hope no-one does but maybe you do and can share a bit about your experience?

EDIT: When I told my other half about this anti pattern he mentioned pets and cattle analogy which is very similar to this and maybe I am just muddying the waters with my tamagotchi design message? You can read the history of pets vs cattle on the linked blog. Here a quick explanation from the same blog.

Pets

Servers or server pairs that are treated as indispensable or unique systems that can never be down. Typically they are manually built, managed, and “hand fed”. Examples include mainframes, solitary servers, HA loadbalancers/firewalls (active/active or active/passive), database systems designed as master/slave (active/passive), and so on.

Cattle

Arrays of more than two servers, that are built using automated tools, and are designed for failure, where no one, two, or even three servers are irreplaceable. Typically, during failure events no human intervention is required as the array exhibits attributes of “routing around failures” by restarting failed servers or replicating data through strategies like triple replication or erasure coding. Examples include web server arrays, multi-master datastores such as Cassandra clusters, multiple racks of gear put together in clusters, and just about anything that is load-balanced and multi-master.

 

 

 

 

Thoughts: Test automation strategy

I am a bit behind the curve on this I am sure but the free platform Test Automation University by Angie Jones is pretty great. There even is a more high level short course on test automation strategy. I was super grateful for this, because of a few reasons which I will cover in a moment.

Why was I looking at the platform in the first place? Well we have objectives around test automation. (A discussion on objective setting and personal development plans could actually be a whole series. I still very much refer back to Stephen Janaway and what he wrote a while back on the subject and 1:1s. Read the whole series, it is great!)

Don’t assign objectives or tasks if the person doesn’t have time or isn’t motivated

But back to the objectives. Now it is pointless assigning someone an objective or personal development task when they are not motivated to act on it. If someone does not want to learn to code or cannot put in the time for whatever reason, then there is no point in them needing to achieve actually writing some code. At the end of the day we want our teams to succeed and feel like they are growing in their role. Now for some that is simply being a valued team member on their team. But we do like pushing people and letting them explore their role from different angles.

As a company approach we want to aim for continuous delivery where feasible (regulations), which means more automation. But not many of us can or want to learn to code.

So we decided to split our automation focus and take on the approach of not just doing but also understanding the whole approach. This is something I really liked about the automation in testing approach by Richard and Mark. It gives test automation another dimension because it no longer is about automating away testing tasks such as regression but also understand why you are doing it and what other things it enables you to do and how it supports testing activities.

Test automation is a whole team approach

Sometimes though these sort of words need to be in a course or some sort of material for people to accept it and listen to it. So finding Angie’s course on test automation strategy makes the whole subject of test automation way more approachable. Most of us testers lead scrum ceremonies or meetings. We have to be good communicators, so armed with some great questions from that course, anyone can help lead a discussion on a test automation approach to make sure it becomes a whole team approach.

Not being a coder can be such an advantage when designing a test automation strategy

Not being a coder can be such an advantage when designing a test automation strategy, not a disadvantage. I have seen people feel intimidated and scared but all the technical details, but being a functional tester often means you have a whole picture of the application and can really help the team automate the more risky parts of the application and the most valuable to the business.

I found that functional testers can easily feel less valued than the ones who can write automated tests and by also focusing on understanding test automation it made this objective a lot more approachable in my mind, and shows that you can still sit at the automation testing table, even if you cannot code the test. It might even free your mind. You don’t have a predetermined level of “automating that is hard so I won’t even go there” mind block. You are free to ask all the questions around what is possible and then is it feasible given the estimated effort.

So high level what do you need for a test automation strategy? Well watch Angie’s course. 😀

My summary of the first video with some of my own thoughts below.

WHAT?

A goal! What is your goal? This made so much sense to me. This can help you identify where are automation efforts are most value-able.

The most common goal I have seen is to reduce regression testing time. But here it can be important to then not jump to conclusions. Just because regression testing happens by manually clicking through the UI, it does not mean your tests need to do the right thing. Be clever about which layer is gets the automated tests and what your regression tests actually check and think about the least costly design that is most effective and build on that.
WHO?

Yes, developers should verify their bug fixes, but …

First of all, thank you to everyone who has spoken to me about this on twitter, on the blog, in person and even in the aisle of Tesco! 😉

This is part 2 of the conversations about my blog post “Should developers verify their now bug fixes?”. Thank you to all participants!

The main consensus is that “Yes developers should verify their bug fixes.”. And now come the many “buts” (is that even a word?).

Yes developers should verify their bug fixes, but…

  1. … still use the 4 eyes principle or two heads are better than one. It does not have to be a tester necessarily who helps with the bug verification, it could be another developer, a product owner/manager, scrum master or designer. -Basically, most have stated that another pair of eyes makes the team feel comfortable that the bugs were fixed correctly.
  2.  should this be a discussion about quality? I expect devs to replicate the bug and test it’s fixed as part of fixing it , regardless of if there’s another step in the process or not. –I would like to add here that I find the pointer towards quality interesting and want to explore that more. And I think both point 1 and 2 also point towards the fact that you should be owning your work and make sure it is getting the attention it needs.
  3. clear bug reports help build trust. – I totally agree. If you can rely on your team members to communicate issues effectively and clearly this builds trust and you know that if you follow the steps indicated you can then verify if the bug is now fixed.
  4. if we fall back to “aren’t the AC enough?” then why test at all? Aren’t the AC enough for a Dev to verify all their work? –I think this went in a slightly different direction. I still feel that testers’ time is well spent exploring stories and testing implementation of features. In the process they may find bugs and report these (hopefully clearly). And the steps and expected results in those bugs should be enough for anyone to verify the bug is fixed.
  5. is verifying a fix, not more than that? Is it not also retesting a full feature and doing a regression of the system’s area? Hence it should be a tester doing the verification. -Well, I think they are different tasks and exercises. At least in my current context. If the fixes to bugs are so large that the whole feature or area needs regression testing than maybe there are other process issues to think about.
  6. there should be testing expertise available if needed. -Yes!
  7. in certain context the risk is too high to not involve testers in bug verification. -This was generally the consensus from people working in financial industries. I totally agree and think this makes sense. These are often large legacy systems and you never know what you may find as a result of a bug fix. I mostly deal with newish code (less than 2 years old)

 

So this is my second post on this subject. I think a lot of my thinking comes down to team sizes and speed of working. I am trying to work out if we have enough testers across the projects and how we may have to change the way we work if more projects drop on us in the near future. One of these changes may involve the way we think about testing and what a tester’s role is on the team. Will their role still involve verifying all  bug fixes? I think i’d like to push for No and see what happens. More on this if I get somewhere, or not. 🙂

So far this has been great in getting an idea of how people may react and how it may affect the projects we are working on. Thank you all!

Thoughts: Accessibility testing for the web

screen-shot-2016-10-23-at-21-58-16

 

Myself and Emma recently organised an intro into accessibility testing with the awesome people at Test Partners.

They are really up for doing talks or tutorials with companies and are super friendly and knowledgeable. Do reach out to them if you have any questions on how to get started.

Why is accessibility important?

Up to 20% of the UK population has a disability but moreover research shows that 57% of the UK population benefit from accessibility features:

    • 20 million people over 50 in the UK – diminishing eye sight
    • 6 million + people with dyslexia in the UK
    • 1.5 million people with arthritis in the hand or wrist – use keyboard shortcuts over a mouse
    • Anyone can have a temporary accessibility need – like breaking an arm or hand

 

I have been aware of accessibility testing and really enjoyed the talk by Michael Larsen on the subject at the end of last year but did not look into it anymore.

This is a shame really because the internet is meant to be for everyone and therefore be inclusive but many websites are not.

Do we as testers do enough or even know enough about accessibility testing to point these things out? Are we knowledgeable enough to have these conversations on why we are excluding certain members of the population on using your website or services?

Have you ever had these conversations?

One place I worked at, I had just hired a new tester who came from an external testing services provider. He knew quite a bit about accessibility and started to raise bugs against our website for the images missing alt text for example.

On the one hand the things he was raising were just good practice to have really. But why did we not raise this before?

I never knew what to look for. I was more focused on functionality for able bodied users, maybe because I could easily be that sort of user.

The positive effect the other tester had, was that the designers and front end developers became more interested in how to make the website more accessible and started to reform from within. They started to design good patterns in accordance with the Web Content Accessibility Guidelines (WCAG) 2.0. I think this is the hard thing when it comes to accessibility, you can raise bugs and therefore awareness against accessibility features missing or keyboard navigation not working but if the designers and front end developers and maybe more importantly, the product team are not considering accessibility from the start then it is really hard to add it back in.

Why do we not learn about these things when we start our testing career? Was I just blind to it? The ISTQB certainly did not tell me about it.

If you are on the dojo there is a great resource to get you started. Maybe there will be a 30 day testing challenge for accessibility testing soon as well. I would love that.

One thing that Steve and Paul from the Test Partners showed us was this tool. You can add the bookmarklets to your browser bookmarks toolbar and then click on them to get visual feedback on how your website is built.

This helps with understanding the semantics of your page. How it is built and structured. For example headings are usually large and lists usually have bullet points. However, the semantic structure also need to be conveyed programmatically by means of tags in the source code. And these bookmarklets highlight where something has been tagged as a list or image for example.

This is a good starting point I feel, to understand what flaws your page may have for screen readers. Screen readers use the tags in your source code to tell the user what the item on the page is and does.

I would love to pair with some users of accessibility related tools at some point. On a similar note this podcast on a blind architect was incredible to find out how blind people actually use screen readers. TL;DR at a way faster setting than you think.

Have you ever paired with a blind user? Or someone who cannot use a mouse? Or maybe someone who uses a magnifyer?

 

 

Thoughts: Pair Testing…sort of

Pair Testing

At the beginning of the year, I attended TestBash in Brighton and there was a talk that just stuck with me with practical application.

This was the talk by Katrina Clokie on Pair Testing. She even outlined how she trailed it in her job.

I recently got a buddy at work and we were talking about sharing knowledge. I really wanted to try pair testing, so we did a version of it.

Step 1 – finding the right task

The team did some work on a tool that I wasn’t too familiar with and as part of our development process we created a testing mind map.

During coding the developer will use this mind map to test his code and depending on risk and sadly often time, the tester will also do some testing using the mind map.

In my pair testing example, the developer had done some testing and so had I, noting down some questions before involving my buddy. I then walked him through what we had tested so far and how the application was working.

Step 2 – What happened

Just due to different experience and knowledge he asked some other valuable questions which aided my testing to go a bit deeper and got us thinking of other testing types such as performance and database behaviour.

For me this was an invaluable experience, as I learned to think about other testing types and techniques and scenarios and I think for my buddy it was also a great experience as he got to see bits of the company’s product catalogue he wouldn’t necessarily get to see on a day to day basis.

I want to try and make this a more regular thing, and also try it before I test something and let the other tester drive.

The other side of this is, that we have a weekly test team meeting and I will try to show features or functionality that the other members may not necessarily see but may have to pick up, when I go on holiday. During this session we can also ask valuable questions of the new feature or product. Which is easier than listening to a monologue when handing something over. I think!

As testers I feel we generally want to know about the company’s products and about all the things they can do for customers and ourselves, so that is also why I envision this knowledge share to be good.

Do you do regular pair testing sessions? How do you structure them? Do you do knowledge sharing sessions with other team members?