1. Testing provides answers to questions
2. In testing you think of the whole picture
3. Consider who is invested in the project and what they are expecting
4. When automating tests make sure they do fail as well as pass
5. Checking vs testing the result
6. Automation myths – 100% can be automated
7. Not everyone experiences bugs in the same way
8. To move to continuous integration or not we would have to evaluate our values as a company and development team
- what are our values?
- ship new features asap?
- developers responsible for their code in production?
- if you are writing code it needs to be good enough for production quality?
- what would the test strategy be? A team approach?
- remove overlap
- make sure testing is not time wasting – efficient automation
- team agrees – whole team approach – define how to split the testing effort
- how much testing needs a human aspect and what is “just” checking?
- tests should be risk based
- team is the key word – have everyone test – designers test designs – PO their acceptance criteria – would designers be too involved in their designs to test efficiently? Where is the whole picture approach that a tester would contribute?
- freedom to decide how to cover the testing for each project
- testers responsible for quality – helping everyone test – more of a consultancy role perhaps?
10. Fast feedback
- question things continuously
- shared ownership of testing means it goes beyond one person or team and is constantly a focal point meaning quality is a focal point from every angle
- every team member has a say and an influence on testing and therefore development
- remove duplication of automation test
- benefits in cutting tests down include faster more reliable feedback – by cutting down on the automation tests everyone can have an overview about what each single test does
- everyone knows what each test does and how important the test is – why is it being and tested and what it tested
11. Acceptance tests become journeys defined by the business
- user journeys
- realistic user journey defined by the business
- limit the journeys and all tests have to finish in a certain time – 10 mins? Realistic for us?
12. Perform a risk assessment
- what are we trying to achieve
- identify risks – too risky? what is the potential return?
- agree on how to mitigate risks – more manual testing?
13. Flexible release pipeline
- testing the right things for each feature – how are they defined?
- feature toggles to do exploratory testing on production – we already do this but does it add too much overhead?
- test data in production environment – benefits include realistic test scenarios
- feature toggles need to be accessible from UI – is an admin tool enough?
- can control type of user to view the new code and use it – involve specific user group or just turn features on for users that sort of fit the desired user?
- trust the test results – let jenkins make the decision – do we trust our automation?
- be aware and honest about flakey tests such as integration tests with other systems
- Brain! Think, explore, question!
- use people from different aspects of the business – everyone has expertise and knowledge
- Be brave and have the awkward conversations
- Define a “what if it goes wrong” strategy – role forwards/backwards? who will respond? more monitoring
- Talk! A Lot!
- Team work – continuous integration cannot be individuals
- everyone cares about testing and quality
- motivated to maintain automated tests
- understand the value of the pipeline
- new area of code – need to ensure coverage becomes a natural thought process
- bug fixes just take minutes
- respond to tweets – “Thanks fixed!”
Automation doesn’t make you safe!