Sunday, 25 November 2012

Blood in the Water - why bug 'feeding frenzies' are a good sign

I've been fortunate that my company has enjoyed a successful year and I've been in a position to take on new testers. As the team has grown an interesting phenomenon has become more apparent relating to patterns of bugs that are raised against our system . At first glance it is an apparently worrying trend, however if we look deeper I think it is something that gives me confidence that we're hiring the right folks and ultimately provides an indication of a healthy testing culture.

A feeding frenzy

We enjoy a highly collaborative relationship between the testers and developers across the team. One way that this manifests itself is through open discussion on any issues that we find through testing. Rather than limiting our communication to playing tennis with the bugs database, we'll discuss emerging problems openly with developers. We'll chat about the risks relating to an issue and decide whether to tackle immediately or raise in the tracking system for future prioritisation. Most importantly, though, we'll try to identify the contributory factors behind the issue to pin down exactly what has caused it to occur.

The phenomenon that I find interesting is that, after one of the testers in the team has been discussing an issue in the office, over the next few hours or days I will often see more bugs which have common characteristics to the original issue being raised. The interesting part is that these will come, not from the original tester, but by other members of the team not immediately connected to the original discussion. These will always differ sufficiently from the original issue to not be mere bug plagiarism, but I'll see a clear correlation between the original bug and the subsequent ones to know that the occurrence of the latter was directly influenced by the former.

A Sign of Distraction?

So are the testers getting distracted? That is one way of looking at it. Another is that they are simply copying each other, but given the fact that I place no value on the number of issues a tester raises, this would be pointless. I take a different view. An extremely useful source but rarely mentioned source of testing information is simply listening in on conversations that take place in the office, eavesdropping if you will. If you were to ask me to list my top 10 most powerful testing techniques then eavesdropping would probably rank in there somewhere. When a crop of issues is raised around an area which has recently been the subject of discussion this tells me that the testers in my team have their ears open and are listening to the information that is being generated and shared between their colleagues daily. Not only that but they are using that information, processing it and applying it to their testing.

The Birth of a Heuristic

Testers use heuristics to generate testing ideas. These will be based on combined information from a variety of sources including documentation, conversations, articles and publications, testing theory and simple prior experience. Many of the 'major' issues that I and my team have identified in my system over recent years have been rooted very much in the specifics of the application and its design rather than in programmatical error. The use of context relevant testing heuristics has certainly allowed us to identify and resolve such issues that may otherwise have gone undetected through more traditional 'requirements traceability matrix' based approaches (As I discussed in this post an effective way of sharing these heuristics between team members is to generate and maintain a team heuristics sheet on a Wiki). The fact that a discussion around a new problem area can result in a flurry of related issues being generated to me is an indication that the team are mentally processing new information to generate new heuristics, and then using their instincts to identify other areas of the system where their application might be relevant. This process of heuristic evolution via 'mutation' to me is a valuable sign of critical thinking on the part of my team and is certainly a positive behaviour that should be encouraged. This is why, whenever I see a series of issues being raised within the team that looks like the testers have been copying each others homework, it always makes me smile. After all, you can't turn off instinct in the best hunters, and when there's blood in the water expect a feeding frenzy to follow.

Image: Wikipedia

Tuesday, 13 November 2012

Sparing the Time for Personal Development

A couple of things that I have read recently have really got me thinking about the subject of personal development. Firstly, I read on twitter that Darren MacMillan will not be attending EuroStar this year as he wanted to focus his holiday time on his family. Although he clarified later when I quizzed him on this that he probably could get the time off from his company to attend, it seemed strange to me that he would consider needing to use his annual leave entitlement to attend a conference. In a similar vein I was talking to Anna Baik a while ago who said that she too used her holiday time to attend conferences.

Secondly, Huib Schoots wrote a blog post on his new job where they operate a 4+1 system. 4 days of working then "1 day per week for the gathering and sharing knowlegde and expertise". This sounds like a great culture with a healthy appreciation of the need for personal development time.

A worthwhile investment?

Perhaps I am very lucky but I've never had to take holiday to attend a conference. I have always justified both the time and cost of attendance on the basis that it is important to develop myself and doing this will in turn improve my work. I do apply self imposed limits though. I'm happy to justify the costs for the great one day conferences such as the Skillsmatter Agile Testing and BDD Exchang, or Ministry of Testing's TestBash. I was lucky enough to attend Eurostar 2011 as a speaker but the full cost of a ticket marks a significant investment relative to e.g. the per-tester hardware budget for a small company and may not always constitute a worthwhile investment compared to smaller scale events and other internal options. Comparing the cost with putting all testers through the same certification based training course, for example, and suddenly it is looking less expensive and I feel that the potential benefits to the company are higher.

A culture of learning

While not quite extending to the 4+1 culture in Huib's company, we do try to promote a culture of learning in my organisation. Some of the ways that I aim to promote this.
  • Each tester has a bi-weekly review of their personal development needs
  • I maintain a mind map of personal development areas for each team member and we work together on what interests them and tasks they can work on to build their skills and knowledge. If they have an interest in a skill or technology not directly related to their current workload then we'll try to find some tasks to get exposure to that area.

  • We allow team members to choose their own tools
  • Their laptops are their own to configure with whatever tools help them to be productive. The restrictions being that the software must be safe and legal, and that they must share the knowledge of any useful tools with others on the Wiki and in a lunch and learn session.

  • Testing research tasks
  • I often give team members background tasks to research a relevant testing subject and present their findings to the team to prompt discussion. Subjects we have covered so far include Exploratory Testing, State based testing, Risk Based Testing, Feature Injection, User Stories and upcoming sessions on Model Based Testing and Testing Oracles

I suggest that team members take a day per sprint on their personal development items and provide the option of attending conferences and meetups. I'm sure that what we do is very limited in comparison to some companies. I have, however, encountered testers who've spent 10 years or more in the job and not met up with others to discuss their craft. One of the arguments that I've heard levied against investing in conferences, for example, is how much of what they encounter the attendee will be able to apply back in their own company. I'd counter this argument in a number of ways:-

  • People will always have an eye on their career progression.
  • It is easy to feel that the world is moving on without you when in the confines of your organisation. Allowing your testers to interact with the wider community will give them a feeling of self progression but also reassure them that they are abreast with the latest trends.

  • The grass is not always greener
  • Personally I find one of the greatest benefits of talking with other testers is to reassure myself that other people encounter the same issues as we do. I recently discussed the problems of testing large data stores with test lead testing in a significantly bigger and well known organisation than my own and was pleased that they had hit many similar issues to the ones we face.

  • Losing staff through lack of personal development will cost more
  • I've been lucky to have had very low turnover of testers in my teams over the last 8 years and I think one of the main reasons for this is the opportunity for development in the individuals concerned, combined with working hard to find individuals who will thrive in such an environment.

  • You never know what is going to be relevant
  • Conference speakers and more importantly attendees come from a variety of companies and cultures. You never know when you'll hear an interesting technique or technology which can be directly applied to improve your own testing. Not attending conferences on the assumption that your process is fixed and nothing applicable can be learned from outside is myopic.

As with all things I think a healthy balance and a consideration for your context needs to be applied. Given the volume and range of conferences and meetings available it would be quite possible for a tester to spend an entire year (and a lot of money) just attending these and not actually doing any testing. I think that for a company, allowing some regular time on personal development items combined with time and budget for a few days per year getting out of the office and learning from others, is an sensible investment for the continuous, development , happiness and ultimately success of your testers.

Image :