Webinar: New Model for Testing

Every model is wrong but might be useful.

Box, George E. P.; Norman R. Draper (1987). Empirical Model-Building and Response Surfaces

Paul Gerrard as part of the Eurostar webinars did a webinar on a New Model for Testing. This sounded really intriguing and struggling a little bit with adopting a pure agile testing method I was really intrigued about this talk. 
Paul has been a great influence in my testing career. He was one of the first speakers I ever talked to at a testing conference in Spain.
He started by illustrating the growing trend of technically orientated testers and assurance staff who may not be doing the testing job but overseeing it and providing guidance on testing.

Their job is to think about testing rather than doing testing perhaps. There seems to be a growing pressure on the testing community as technology involves and progresses.

In the webinar the following areas where identified:
  1. Stampede to mobile – technology on the go
  2. Big Data – comes with a pressure to understand performance testing and scalability
  3. Internet of Things
  4. Wearable computing – I personally haven’t looked into this area much yet, but it is very interesting as it often touches on things such as health and fitness
  5. Continuous Delivery – This seems to be the next big thing in terms of the software development lifecycle now that agile isn’t new anymore
  6. Analytics, Data Driven Development – Test Analytics – drive development through the data from new features in the production environment rather than develop based on assumptions
  7. No test teams but embedded testers – This is something we are attempting and recently it has worked well to make testing everyone’s responsibility in the scrum team
  8. Has Agile peaked?

Paul moved on to 3 Innovations:

1. Being agile
  • “Agile doesn’t work but being agile might” – Agile is not innovative anymore – rapid feedback is good, challenge is to be an agile organisation – the decision process needs to be agile in order for agile software development to work.
  • Often agile doesn’t work because there is mostly a cultural business problem – when the business isnt agile, agile development may not work therefore Hybrid approaches often called wagile are most common.
  • In software development we shouldn’t get stuck with agile processes but use the things that work – after all agile is a philosophy and not the goal so to speak

2. Shift left 
  • Examples of this are dev in test, testers in development, testers everywhere
  • The focus of test from the end to across the life cycle – test early and test often!
  • Make testing an activity over a process.
  • Redistributed testing – test requirements and stories by asking questions.
  • BDD/Test driven
  • Move testing effort to the beginning of the software life cycle – challenge requirements through examples – ie gherkin format – EDIT: I brought this up straight away with our scrum masters and am hoping to trial this in the next project we are going to be working on. Hopefully it will be beneficial to the whole team. I will report on the outcome of this trial as I found this really interesting and had not thought of applying BDD model without a BDD framework, as in without actually applying automated tests myself.
  • This will result in test cases being applied as part of requirement defining.
  • The idea of shift left is to save time at the end of the development process by investing testing effort into requirement and design elements
  • Who does this? Depends on the company set up.
  • “Continuous deployment is not the same as going live” – EDIT: My dev manager and one of the tech leads walked up as this slide was displayed and stated that this was the most sensible statement they had seen in connection with CD in a long time.
  • CD not the same as agile

3. Analytics – from production, from test … EDIT: Here I was unfortunately interrupted and didn’t get any more notes so if anyone has these please comment 🙂 EDIT2: I now have the link to the recording so will also add more info later.

The webinar goes on to discuss where these innovations lead us?
We can identify 3 patterns of software development:
  1. agile
  2. structured
  3. continuous

These may have commonalities:
  • agile and structured are very goal driven
  • agile and continuous are a lot more autonomous
  • structured and continuous both have very high process and high discipline
  • Therefore there are parallels between all 3 patterns
  • BUT there are many more patterns to consider and are 3 patterns enough? Hybrids were mentioned and these are just as valid to be considered.

  • Is the right thing to do to adopt a hybrid approach that suits the context?
  • The industry is seemingly moving from high process to DIY process. EDIT: I liked this a lot. Some may want to make a too small shoe fit but adapting processes to fit your context makes complete sense to me.
  • “Software is a people problem not a technical problem”
  • Agile is high process in the small and true agility is adaptation on the fly

Paul asks “Do we need a new model of testing free from logistics?”
Below the key bullet points I took from this.

  • Do we need some flexible process
  • The two extremes waterfall vs agile
  • Strip logistics away
  • ALL testing is exploratory
  • Explore knowledge sources to build test models that inform our testing
  • All testing is based on models
  • models are predictions of reality or versions of reality
  • models are innate, essential and human – we use them everywhere in everything all the time
  • our brains make and use models all the time

Interestingly enough I went to a talk at Brighton Java that same evening and the developer doing the talk talked about their process and it went along similar tangents. Use hybrid models when defining your software development process. Even as much as having different ways of developing in different teams across the same whole development team. He stressed how it is not important necessarily which tools you use as long as you are comfortable and produce good code.

To come back to Paul’s webinar he then proposed his new model. The basics as I understood them being:

  1. exploration of sources to create models and as we explore we model and at some point we decide the models are useful enough to inform testing
  2. sometimes models are not adequate so we explore again
  3. judgement is used to make the decision (this can be experience in the job, with the technology or product)
  4. developers explore the same way but with a different goal in mind.
EDIT: Again the developer at Brighton Java stressed to not just go and code but close your laptop/turn your PC off for a moment and do some exploration before you start your task.

The full paper about the new model can be found here http://dev.sp.qa/download/newModel

  1. Consequences of this new model?
  2. dev and testers have v similar exploration process
  3. TDD not about testing – a mechanism to implement testable code – design process
  4. BDD – is modelling using stories
  5. Testing vs Checking – check is a test that can be applied by a human or tool – but only humans can interpret

The webinar ends with a summary on how the capabilities of a tester may change as a result of such a model for testing.

  • inquiring, modelling, predicting, challenging, informing, applying, etc
  • shift left is more important than agile
  • everyone can test
  • end manual feature checking – free people to do the testing
  • we test systems, not software
  • we test early, we test often but it may not be called testing
  • testing is about measuring achievement and not quality – align systems delivery with project goals and risks and you win
  • testers do not own testing anymore – assurance through the lifecycle – support the test activities
  • are devs and testers the same? v similar skills
  • testing must align with development not compete with it or rescue it
  • embed testing early

Overall I found this to be very interesting and want to try and apply some of these things. I actually already started questioning the stories before the sprint planning (if they are available) in a “given when then” format to see if this will help us define acceptance criteria early and more precisely.

Note: images were taking from the webinar as screenshots can be found in the link at the top.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s