First of all, thank you to everyone who has spoken to me about this on twitter, on the blog, in person and even in the aisle of Tesco! 😉
This is part 2 of the conversations about my blog post “Should developers verify their now bug fixes?”. Thank you to all participants!
The main consensus is that “Yes developers should verify their bug fixes.”. And now come the many “buts” (is that even a word?).
Yes developers should verify their bug fixes, but…
- … still use the 4 eyes principle or two heads are better than one. It does not have to be a tester necessarily who helps with the bug verification, it could be another developer, a product owner/manager, scrum master or designer. -Basically, most have stated that another pair of eyes makes the team feel comfortable that the bugs were fixed correctly.
- should this be a discussion about quality? I expect devs to replicate the bug and test it’s fixed as part of fixing it , regardless of if there’s another step in the process or not. –I would like to add here that I find the pointer towards quality interesting and want to explore that more. And I think both point 1 and 2 also point towards the fact that you should be owning your work and make sure it is getting the attention it needs.
- clear bug reports help build trust. – I totally agree. If you can rely on your team members to communicate issues effectively and clearly this builds trust and you know that if you follow the steps indicated you can then verify if the bug is now fixed.
- if we fall back to “aren’t the AC enough?” then why test at all? Aren’t the AC enough for a Dev to verify all their work? –I think this went in a slightly different direction. I still feel that testers’ time is well spent exploring stories and testing implementation of features. In the process they may find bugs and report these (hopefully clearly). And the steps and expected results in those bugs should be enough for anyone to verify the bug is fixed.
- is verifying a fix, not more than that? Is it not also retesting a full feature and doing a regression of the system’s area? Hence it should be a tester doing the verification. -Well, I think they are different tasks and exercises. At least in my current context. If the fixes to bugs are so large that the whole feature or area needs regression testing than maybe there are other process issues to think about.
- there should be testing expertise available if needed. -Yes!
- in certain context the risk is too high to not involve testers in bug verification. -This was generally the consensus from people working in financial industries. I totally agree and think this makes sense. These are often large legacy systems and you never know what you may find as a result of a bug fix. I mostly deal with newish code (less than 2 years old)
So this is my second post on this subject. I think a lot of my thinking comes down to team sizes and speed of working. I am trying to work out if we have enough testers across the projects and how we may have to change the way we work if more projects drop on us in the near future. One of these changes may involve the way we think about testing and what a tester’s role is on the team. Will their role still involve verifying all bug fixes? I think i’d like to push for No and see what happens. More on this if I get somewhere, or not. 🙂
So far this has been great in getting an idea of how people may react and how it may affect the projects we are working on. Thank you all!
This is my attempt at starting to document this discussion I am having in my life about if developers should check their own bug fixes?
In my previous job, we worked very differently to how the teams I work in now work. Not only because my previous teams worked in Continuous deployment and delivery compared to a more rigid 2 weekly release cycle, it is also partially due to the fact that I now work on projects rather than a product the company owns and develops and maintains through and through. The projects can be anything from 6 weeks to 2 years long with the long term projects being a newer thing.
In the old job we were 2 testers across around 2 product teams and our time had to be used in the most useful and productive way. One task traditionally executed by testers is to verify bugs are fixed. I had forgotten about this because I hardly checked bug fixes in the last 2 years. Why did I not verify bug fixes for 2 years? Well, one argument is that in general a bug report has a well defined steps to reproduce and an actual and expected result. Hence anyone should be able to check if the bug is fixed. Why not the developer? It hardly takes testing expertise to check a bug is no longer happening. With verifying bug fixes off of my plate, I could use my time doing exploratory testing of new features and user journeys through the application. I would of course still log bugs and discuss them with developers, but there was less of the bug fix ping pong going on.
Now with me changing my job, I saw an opportunity because a big time sink of the few testers I now work with is doing bug verification. I tried to argue that maybe they should not be testing the bug fixes but it should be the developers’ responsibility to test their own fixes. I heard some interesting pros and cons for this internally and through twitter as well. So here we go:
- Frees up testers’ time to do actual exploratory testing. Bug fixes could be classed as “checking” and the bug reports should be clear enough for anyone to check the fix.
- Trust – no-one wants todo bad work and a bit of trust goes a long way for good team dynamics.
- If developers do more testing, they may find ways of making it easier/faster by writing scripts or little tools, that the test team may not have the skills for doing on the fly.
- It may lead to more pairing to avoid bugs in the first place.
- Lack of trust – historically bugs were not fixed properly and there may not be trust that the developer completely fixed the bug.
- Would you check your own homework? Devs testing their own code is frowned upon, does this also apply to bug fixes?
- Is it really fixed if no-one else has seen the fix?
- What about knock on bugs that appear as a result of the fix?
What are your pros and cons for verifying bug fixes? Do you check bug fixes? Does it take away from your time testing the application?
I do think there is an element of “it depends” here as well. And this applies to the severity of the bug and the further risk of something else being wrong after the bug fix. But I would hope that someone checking a bug fix would notice if something else was wrong but maybe I have worked in very trusting environments the last 2 years and I need to re-adjust my thinking.
|Random Picture from my holiday in Sicily
In a recent blog by Eric Jacobson’s he argues that reporting trivial bugs tends to waste everybody’s time and that you shouldn’t log them.
I really struggle with this concept so thought I would explain why and maybe see what I learn this year and then reflect on this in a later post.
Why should we not log trivial bugs?
‘If a trivial bug gets logged, often…
- a programmer sees the bug report and fixes it – I personally have no issue with this as trivial bugs can be very visible even they do not threaten the inherent value of the feature immediately.
- a programmer sees the bug report and wonders why the tester is not testing more important things – this I find a silly assertion but maybe it can happen? Has anyone experienced this?
- a team member stumbles upon the bug report and has to spend 4 minutes reading it and understanding it before assigning some other attribute to it (like “deferred” or “rejected”) – I see how this wastes time but in the beginning it could help create a shared understanding of trivial that may be wasting time
- a team member argues that it’s not worth fixing – that is the case with a few bugs in general, but in the end our product owner decides and needs allege information we can provide so we would log all trivial bugs
- a tester has spent 15 minutes documenting a trivial bug. – I try to note down the more trivial ones and keep them tip the end to log but yes I understand the value of using the 15 minutes to try and test to find something more juicy to log.
I think I will struggle with this concept for a while. Is it more of an issue in very big teams?
Why do I struggle?
I was taught to log anything and everything but maybe that is because I used to work for a third party games testing company providing a service and then after that I worked with a distributed team across different time zones.
So there was no time to discuss that we were aware of minor/trivial bugs but decided to not log them. We just documented everything.
Currently I still preach the same philosophy but in my last job we had a nice turnaround where any bug not touched in 6 months was closed, BUT trivial/minor bugs which tended to be grammar or spelling mistakes were actually prioritized as they are customer facing and cause the customer an annoyance. They tend to lower my own trust in a product as well.
I think it also depends how you define minor or trivial? This will vary from context to context and from project to project.
Maybe we can use the Jerry Weinberg quote “Quality is value to some person.” and state:
Bugs are minor/trivial to some person during some project.