Tuesday, 19 June 2018

Textual description of firstImageUrl

How Not To Get Into Test Automation

I’m going to have a bit of a moan. I’m not inclined to rant, however this has been a bugbear of mine for a while which came up again recently. I’m hoping that sharing my thoughts will hopefully both get them off my chest, whilst at the same time providing some useful career advice for budding testers.

Here’s an example of a conversation I recently had with a candidate for a testing job

Me - So what would you ideally like to be doing in a new role?

Candidate - Well I’ve got lots of experience of manual testing so I’d like to use my experience there, and ideally I’d also like the opportunity to get into automation

Me - have you done anything so far to develop automation tooling skills?

Candidate - No, the opportunity hasn’t come up in my previous roles so ideally I’d get a role with a company that’s going to help me develop in this area

Me - Sigh.

I have lost count of the number of times I’ve had this conversation and I’ve never hired a candidate off the back of such a discussion. I’m sorry to these candidates - but if I am hiring with automation in mind then the last person I’m going to hire is someone who claims to want to develop skills in that area but has taken no steps in acquiring those skills themselves.

If I want someone to create automation tools then the characteristics that I'm looking for are typically

  • They are capable of writing code to automate tests - this doesn’t have to involve knowing a specific language - just that they can understand coding structures
  • They are capable of understanding the benefits and limitations of automating test execution
  • They are capable of self learning

If you are not able to demonstrate these skills then it's naïve to expect an employer to land you with a juicy test role where you get trained in test automation.

To be absolutely clear, I'm not in any way stating that personal development is the sole responsibility of the employee and any new skills need to be progressed in their own time. What I am saying is that the chances that I would choose to invest time and money into building the skills of an individual increase significantly if that person can demonstrate the ability to learn and improve themselves. Opportunities for self progression don't often land unexpectedly, they tend to gravitate towards those people who appear to deserve and merit the opportunity.

Teach yourself

Across my previous testing roles I've created many test tools and harnesses

  • I created a multi-server database test scheduling harness in Linux shell after teaching myself shell scripting
  • After a manager rejected my proposal to automate ODBC tests, I did it anyway, demonstrated the value and got justification to build on and maintain these
  • I taught myself java in my evenings and created two test harnesses for acceptance and load testing the JDBC interface
  • I learned Perl to output our test build results in a readable html report
  • I learned python to create a simple test harness to parse JSON to test a REST interface
  • I learned ruby to create a random SQL query generator as a side project

In every single instance I learned the language off my own back in order to add the value in what I felt was the best way. Sometimes this started with some help from developers. Other times this involved sitting up late on Friday nights starting from "Hello World" with a beer and a few web pages to get me started on the basics of the language. Now I’m not advocating that everyone spend their Friday nights working, but if you’re driven to do something and are learning new skills it genuinely doesn’t feel like work (the beer helps with that!).

The intention here isn’t to show off - after all most of these harnesses were picked up, maintained and often improved by other members of my team who developed themselves and their skills in the process. Not only that but they then went on to create their own tools - not because I asked them to but because they wanted to and because they could. And the reason that they could was because I’d hired people with a demonstrable ability to self learn to improve their skills whilst adding value.

The technology doesn’t matter...

When hiring testers I’m not interested in whether they know a specific language or tool. I am, however, absolutely focussed on finding people that have shown the ability to drive their own improvement and make opportunities for themselves. This doesn’t necessarily have to involve tool creation, however the people who I have hired for this have shown clear evidence of researching and intelligently creating or introducing tools to improve the testing in their companies.

I remember once being particularly excited about a CV that landed on my desk. The tester in question had clearly self-learned new technologies in order to demonstrate value and suggest a number of improvements in previous roles that improved testing and saved time/money. Naturally I hired this person and she went on to become one of the best testing hires I’ve ever made. That CV became my go to example when talking to recruiters about what a good tester CV really looks like (a great way to discuss tester recruitment - I call it "recruitment by example" )

... and technologies alone aren’t enough

On the flip side, I’m also very sceptical of hiring individuals who have been trained in one technology, such as Selenium, and shown no flexibility in looking outside that at other tools or approaches. Testing ain’t programming people and, whilst coding with one technology is useful it’s not what makes a great tester.

One of the best testers I’ve hired has no coding skills but has delivered great automation. She used her strong testing knowledge and relationship building skills to motivate developers to help her in introducing automated acceptance testing. The same tester also introduced Session Based Exploratory Testing to cope with a cutting of testing headcount in the business. I’d rather have a tester with no coding skills who has effected change in this way over someone that’s worked exclusively on Selenium automation for 5 years.

Don't just sit there

So that’s it - rant over. I hope that if you’re a tester who is looking for work then you take some of this on board. Don't expect prospective employers to do the work for you. Be hungry, be proactive, be a pioneer of tools and approaches in your organisation, and ultimately be prepared to take the first few steps on whatever path you wish to pursue yourself...

..oh and if you live in commutable distance from Gloucestershire UK let me know - I’m always happy to speak to you as I'm frequently looking for proactive and passionate testers to join my team.

Photos by Fabrizio Verrecchia and Etienne Boulanger on Unsplash

Monday, 4 June 2018

Textual description of firstImageUrl

The Wave of Risk Perception

I had an interesting chat with a former member of my team recently. We were in the pub and she was sharing with me her woes in a new role. The problem, as she explained it, was that the members of the leadership team in her company were pushing back against her efforts to devote development time to improving existing features, robustness and stability of the product. They were solely interested in new features. The issue seemed to be that they didn't have an appreciation of the levels of risk involved in their existing product. To help her to understand the reasons for their position I drew up a diagram that I've used in a few talks to explain the perception of risk. The conversation reminded me that I'd not written about this, so here goes...

Destined to be Different

It is not a coincidence that in many situations people close to software development, particularly testers, have a very different perception of risk than some leadership roles. One of my favourite historical blog posts is this one on why testers and business leaders inevitably differ in their perception of risk. I recommend reading that one for a deeper exploration, but to summarise here:-

  • Humans are susceptible to the 'Availability Heuristic' that leads us to attribute higher risk to outcomes which we can recall experiences or stories around. Therefore roles with greater visibility of issues that have occurred through a process, or of software problems in general, will attribute higher risk to a software release than those who have not
  • Research such as this paper has additionally shown that managers who don't have a detailed understanding of negative outcomes will aggregate multiple separate negative possibilities into one 'risk of failures' which they then attribute a lower level of risk to than is appropriate due to lack of availability.
  • The 'Affect Heuristic' means that we naturally attribute lower risk to things that we associate a strong positive outcome to, and vice-versa, so a manager focusing emotionally on the benefits of a software release for customers will typically attribute lower risk to the release than those roles with a focus on the challenges of technical delivery

These three factors combined give good cause to expect leadership roles to have a much lower perception of risk around a software delivery than testers and other development roles.

The Wave of Perceived risk

Understanding differences in risk perception is all well and good, however unless it is in understanding and recognising the potential outcomes of this disparity that the theory becomes practically useful. What would we expect to see as a result of this somewhat inevitable difference in perspectives?

When creating a talk around risk and the reasons that I came up with the 'risk questionnaire' I used the following diagram as a means to represent the scenario that can arise when there is a strong disparity of risk perception between testers and management. The idea here was to represent perceived risk over time in a tester and a product role on a new development and to explain the shape of testing that we would expect to see as a result.

Initially when commencing development what we expect to see is a large difference in their perceived levels of risk around the same situation. The perception of risk in management is low and decisions on approach and expected pace of development are driven based on this. The tester is aware of the lack of rigour in place and will have an elevated perception of risk driven by their experience of the many issues that could surface.

Inevitably one of these potential problems goes bang and the manager is suddenly and painfully aware of a specific risk.

Additional effort is justified to tackle 'the problem'. This is likely to be in the form of a process improvement or the creation of some new tests, however the scope of this work will in all likelihood be limited to covering the issue that emerged, possibly in excess of an appropriate level given the testers elevated perception of risk.

The manager is satisfied and returns to a having a lower perception of the risk situation, however knowledge of the problem does give some increased awareness of the chances of hitting problems. The Tester, on the other hand, knows that only one of the many areas of potential concern has been mitigated and their confidence is only slightly increased due to their maintained perception of the other potential negative outcomes in their process.

Inevitably another issue cross up in another area with similar consequences

over time the encountering of issues and improving testing reaches a point where, whilst not necessarily aligned, the perception of risk between tester and manager is closer and in balance with a consensus around the level of testing needed and the resulting chance of hitting issues.

The testing is likely to be formed around pockets of high (potentially excessive) coverage where risks have manifested themselves, with lighter coverage in other areas.

Surfing the Wave

I was slightly nervous when I first presented this at a conference, however I've presented this a few times now and had many testers confirming that this looks very much like the way that testing evolved in their organisation. At the last talk one audience member said it was the best way of representing this situation that he had seen.

The same person asked an interesting question - given the choice would you reduce the amplitude or the frequency of the curve? To rephrase - is the way to tackle the problem to reduce the number of issues or the difference in perception to start with? For me the two are separate but go hand in hand, and it is necessary to address the second to justify focussing on the first. Working towards a more common perception of risk between roles will not in itself reduce the problems encountered. Without addressing this disparity, however, it can be hard for testers to establish and justify a level of rigour around development required to do so. Such a change for a manager with a low perception of risk would appear to be an increase in development time and cost for little benefit. If we want backing to improve testing, without just bouncing from issue to issue to justify point improvements, then helping to establish a more consistent perception of risk is clearly a key area to focus.

The good news here is that, from our understanding of the causes of the difference in perception, we have a great starting point in understanding this might be achieved. I identified at the start of this post three biases that impact our perception of risk that we can act to influence.

  • Tell stories to convey risk information. Human risk perception simply isn't driven by the logical brain processes that deal with facts and figures. Risk perception is influenced far more through 'availability' of stories and experiences that convey the potential for negative outcomes. My personal experience supports this - one excellent example involved a conversation with a CEO once around improving data security that I was expecting to be a challenge, however he had just recently read an article on a data breach from a similar company which made headline news, and he was therefore very open to addressing the risks in that area.

  • Don't aggregate all conversations on issues into 'bugs'. Discussing different potential problem areas separately will help to establish a more realistic understanding of the multiple negative outcomes that could arise in our development. Actually discussing these collectively as bugs could result in an under-appreciation of the different risks in those not close to the details of the development.

  • Include testers early in value conversations - Testers are often excluded from early conversations around developments, and are only brought in when the solutions are scoped. Yet it is in those early conversations that the real value of the development is established and discussed. It carries then that one positive way to avoid the curve by influencing their own risk perception is by getting themselves involved from the start and understanding the value that each development could deliver.

It is worth remembering here that our focus needs to be on establishing a realistic and consistent perception of risk across roles, which involves more than just increasing awareness of risk in management. Despite the superpowers to find issues and save projects, testers are only human and just as susceptible to biases in their risk perception as everyone else and the tester's default position is as likely to involve over-estimation of risk as management roles are to under-estimate.

A key element to the testing role is to understand the biases that can influence the perception of risks in our own roles as well as others. Understanding these can provide the key to being able to have genuinely influential conversations with business leaders in ways that will effectively convey risks in a way that is meaningful to them, and hopefully avoid the painful process of progressing up the wave that comes when different roles start from very different perspectives.