Sunday, 26 February 2012

Textual description of firstImageUrl

How Can Testers Develop Their Technical Skills?

A while ago I responded to a post on the STC website asking where testers might look to develop their technical skills. Lisa Crispin and Rosie Sherry suggested that I expand the response into an article for the Testing Planet, which I duly started to do. Shortly after I needed some material for a couple of short talks including this one on the EuroSTAR website (sorry - needs a subscription) and so I used the same ideas. I felt after this that it was unfair to provide the same material for an article and wrote something fresh, so I'll publish this here instead. Please feel free to add any further suggestions that you may have on this subject on the comments page, I'm open to any good ideas on ways to learn.

Sources of Learning Technical Skills for Software Testers

An important area of expertise for Software Testers to develop skills is in our ability to gain an understanding of how our software behaves and interacts with its environment. When testing software the temptation can be to focus on the product features, however the manner that the software interacts with the operating system, network, databases and other resources that reside within the operating environment can throw up as many significant issues. The problem that we face as testers in this regard is that there is no de-facto source for learning this stuff. Whereas there are many approaches and schools documenting testing theory and the design and management of tests, there are no such schools teaching the skills of monitoring and measurement of application resource usage. Given that the possible range of skills that a tester may need to draw upon depending on the task at hand is so diverse, it is understandable that this is the case, but this doesn't really help us. The good news is that, with an enthusiastic approach and a good dose of persistence there are plenty of channels via which we can learn valuable skills which will help us in our testing endeavours.

Here I present a few ideas of potential sources of information that can help enthusiastic testers to develop their skills.

Potential Angry Stakeholders

One of the first places to look when trying to establish how to measure the resource usage of your system within its environment is to consider who's going to get mad with you if the system performs terribly in that regard. Finding those individuals whose domain your poorly performing software will be messing up and finding out how they measure what is important to them is a great start. In my experience people are always happier to share a skill with you to allow you to progress on your own than for you to bother them repeatedly requesting assistance. Asking for some devoted time for one on one training will not be rejected if it allows you to be less dependent on their skills in the future.

  • System Admins
  • Internal IT folks and system administrators are a great source of information on understanding the behaviour and configuration of operating systems. If you have a friendly system admin team, arrange some time to spend with them learning the attributes of software that are important to them and the tools that they use to measure these. A few years ago I learned a great deal on the monitoring of Windows processes and nework usage from the system admins at the organisation that I was working with, which helped me to pinpoint some network problems to a specific device in our installation architecture.

  • DBAs.
  • At some point in your testing career you will almost certainly work with a system that interacts with a database. Being able to query to check that your data is present in the tables is a good start, but it will really benefit your testing to avance your skills beyond this basic level. Your DBA can help you to understand what can impact the performance of a database system, and how to monitor activity as your application runs. Being able to examine the queries that your software is executing against the database and critically assess the efficiency of these is a valuable skill to learn. Reporting performance is one thing, but if you can go to the programmers with a set of recommendations of how they might actually improve the performance of the application through better query design or indexing, this will earn you a whole heap of respect with the team.


As well as potential stakeholders, a tester's colleagues also have a vested interest in their ability to monitor the application's use of resources. It can be easy to overlook those closest to you when looking for ways to improve your knowledge and skills.

  • Programmers
  • Programmers are highly skilled and inquisitive (and sometimes, dare I say it, a bit geeky) so usually have a range of knowledge that extends well beyond cutting code for the product in question. If you are lucky enough to work with a friendly crowd (not always the case) chat to your colleagues and see if they know any tools that might help you in your testing endeavours. In my experience many programmer write their own tools to help with their day to day work, often including tools to view key system information. Developer written tools in my toolkit include handy command line odbc query tools that query at various interface levels within the system, a tool to measure the in memory size of an object and numerous flags and debugging options to obtain runtime information from the application. In addition to learning some excellent tips and tools, I find that actively trying to learn from the programmers helps to elevate your reputation with them and give greater credence to any issues that you raise.

  • Your tester colleagues
  • As I wrote about here a test team should ideally comprise individuals with a matrix of skills and experience. Each tester may possess skills or expertise that could be shared across the team. In your own company try developing a culture of communal learning among your fellow testers. Creating a wiki with sections for useful testing tools and tricks in a similar way to what I describe in this post is a great way to share knowledge. Taking it in turns to do lunchtime presentations on tools that you have been using will help to advance the knowledge of the team as a whole. Sharing your learning will help not only improve your skills as a team but consequently the quality of your product as well.

Looking Externally

Not every tester has the benefit of expert stakeholders or skilled colleagues to help them to advance their craft. Although appropriate for all testers, in this situation particularly the use of external resources is key.

  • Operating system tools
  • There are many tools and utilities available integrated with or in extension to operating systems that can help with monitoring such as procmon/filemon/perfmon/procexp for windows, top/ps/iostat/netstat/sar/lsof to name but a few of hundreds for unix/linux. Looking at the man pages of the tools that are available by default in the operating system command line can give some great information on options that might make those tools useful for testing purposes.

  • Open Source Tools
  • There are many open source tools for more specific tasks such as WireShark/Fiddler for monitoring network/http activity, cports for monitoring open ports and FireBug for debugging web page content. Try searching around and see what is available, and assign time in your schedule for researching and learning new tools. This investment of time and effort will pay for itself in the longer term with the skills that you will develop and the insights those abilities will give you into the performance of your software.

  • Testing forums
  • For any testing/monitoring problem that you face the chances are that someone has hit something similar before and there will likely be a forum discussion somewhere discussing related issues. If not then there are many great forums online for software testers where you could post your question such as the Agile Testing Group, the Context-Driven Testing Group as well as the Software Testing Club. Ask specific questions on testing forums and you will have a wealth of experts to help you. Be careful not to waste time with questions that are too general or not well thought out, if you have put no effort into researching your question then don't expect busy folk to spend their time furnishing you with an answer.

  • Training Courses
  • Although sometimes not the most cost effective method of learning, it might be that a training course is an option to get you started in working with a particular technology. If the relevant expertise is not available in house then a training course might be the best avenue available to you. If you need database skills, for example, and don't have an in house DBA then discuss with your manager about getting some training. If you outline the activities that you hope to be able to achieve with your skills and the resulting benefits then an understanding manager may well agree to a training budget for you. On more than one occasion in the past I've successfully lobbied for budget for training courses to get me started with technologies that my product was interfacing with or languages to aid test automation.
This is, of course, not an exhaustive list, but each of the elements here has one thing in common. They won't come to you. It is my belief that employers need to respect the testing role and provide time and resources to allow the tester to develop their skills. In return, however, testers have a responsibility to advance themselves and develop their abilities. A major part of this is by actively looking for new tools and techniques to assess how the software interacts with its environment.

Sunday, 5 February 2012

Textual description of firstImageUrl

Survival of the Fit-Tester (3) - Making Decisions

In my previous two posts I discussed how as testers we might work to protect ourselves from the various risks that face us in the current market, primarily those of outsourcing and developer only testing. Both of those posts focussed around the building of strong relationships in the organisation, firstly by getting engaged with the developers within the team, and secondly offsetting this with symbiotic relationships outside the team.

One key relationship for a tester that I haven't yet discussed, and one that is particularly relevant when working in an agile environment, is that of the product owner. In this post I'll review the idea of the agile product owner and the tester's relationship with them, and how the testing role can make an evolutionary step to help take on the product owner's responsibilities and address some of the challenges that can accompany that role.

Making Decisions

In his 2009 post The Mythical Customer Problem Gojko Adzic highlighted one of the potential flaws in the model of an Agile team. The notion of an individual who has all of the requisite domain knowledge to make all of the business decisions for the team is an optimistic one. Even if there was such an individual the chances are that someone with this level of knowledge is going to be in high demand and not available on a daily basis to furnish the team with all of the decisions required to progress their work. In my organisation the product owner is an extremely busy man working on customer engagements and management duties. If we deferred to him for every minor design decision then it would seriously compromise his ability to fulfil his many responsibilities. A product owner, almost by definition, needs other responsibilities which interface with the customers or users of the product to qualify them for the product owner role. These responsibilities may impact on their availability to the team. For many teams, as Gojko suggests and we have certainly found, the reality is that the team will evolve to take on the responsibility of making many of the day to day decisions on the project collaboratively, and referring to the valuable time of the product owner when the way forward is unclear or major tactical decisons need to be made.

An opportunity to shine

This process of "collaborative specificaton" affords testers with a fantastic opportunity. As covered in my previous post, by building strong relationships with customer facing roles testers can develop key knowledge on the customers and their demands and uses of our products. The product owner/customer falls into this category as well. As testers I believe we should work to build an excellent relationship with the product owner, to the extent that we can expand our remit as the 'proxy' to that role within the development team and help to make business related development decisions on the product owner's behalf.

One complaint that we regularly hear from testers in late stage testing projects is that they are not involved in the requirements early enough. Requirements in an agile process generally come in the form of user stories. User stories in the early development stages are little more than "placeholders for a conversation" and are far more fluid around the implementation details than traditional specifications. Small agile teams rarely have the luxury of a devoted business analyst, and I think this affords an excellent opportunity for the smart tester to extend their role. By getting involved the specification process the tester can evolve from a role of requirements verification to one of guidance, starting from the very early specification discussions.

In my organisation the excellent domain knowledge of the senior testers has helped us to become an key part of the elaboration meetings when discussing user stories. I'm not suggesting for a second that we manage the specification discussions, however there are many inputs needed into this process both from a product technical and a business perspective. With the developers providing the former the testers take on the responsibility for the latter based on information gained from the product owner and their other customer facing relationships. Where a detailed specification is lacking the tester will help to fill in the detail that is required based on their knowledge of customer implementations and priorities from the customer facing roles in the organisation.

In the area of bugs too we have evolved such that the product owner is often happy to delegate to the team on the decisions regarding bug priorities. We have a partner delivery model with many implementations in live use on various historical release versions. Myself and the other more experienced testers will often help choose between related risks and make the call whether or not to push fixes to release branches.

Potential pitfalls

There is inevitably a potential downside to increasing the remit of the tester's role. Testing is viewed by many in the profession as a position of information service provider. We are not there to make decisions, we simply furnish those that make the decisions with the information to do so. The advantage of such a position is that, should those decisions prove wrong, we are abstracted away from the responsibility for any failure through our lack of involvement in making them. The idea of involving ourselves in the specification process removes the safety net of blaming absence or poor quality of requirements in our inability to perform effective testing. By stepping up and involving ourselves in the decision making process we are exposing ourselves to far greater personal risk in terms of responsibility for failure.

My argument to counter these pitfalls would be this. I would rather commit to a project wholeheartedly and risk sharing in the responsibility for its failure than see it succeed having had the frustrations of others making the decisions and taking all of the risks. Engaging in the specification process helps to complete the picture of integrating the testing role fully into the product development stack. Refusing to do so may expose us to the greater risks that I've already discussed. Given the choice I'd always prefer to have more control over my own destiny than less, and the stresses of additional responsibility for me will always win out over the frustrations of being an afterthought in the decision machine.

Future of the Fit-Tester

I hope you've enjoyed this small series of posts. Needless to say these are my own opinions and I'm sure others see things differently. One thing is clear to me, and that is that the risks that I've discussed here do exist. Based on surveying UKTMF members Paul Gerrard last year predicted that 50% of UK testing jobs would be outsourced to other countries in the next 5 years. In the US recently Lanette Creamer blogged about a Selenium Meetup in San-Francisco where the "test lead" of a lean startup made proud claims of not needing a QA team at all. These are isolated instances, but give an indication of trends that can affect us if we are not aware of them. If those of us who consider ourselves "Testers" want to survive and thrive then we need to be prepared to evolve. In my this small series of posts I've discussed some of the directions in which I and my team have done just that in recent years, for better or worse. Personally I would rather face the problems that come with being aware of risks and taking positive action than those that come with being made redundant through maintained ignorance of those risks.

I'd be happy to respond to any comments you have below or on twitter. Additionally I'll be speaking on the subject of "Survival of the Fit-Tester" at the Ministry of Testing TestBash in Cambridge on 23rd March so please come and chat to me then if you can attend the event.