Wednesday, 23 December 2009

3 Cases for communication

I have recently been making a drive towards increased customer communication in our development processes. By a happy coincidence soon after starting this drive I attended the Agile Testing and BDD Exchange and watched Gojko Adzic's presentation on Challenging Requirements. At RainStor we have strong relationships with our partners, and in the majority of cases deliver working software, on time, doing what the customer wants. However, on occasion I feel that the underlying customer requirements can sometimes be misinterpreted through the story definition and elaboration process. My aim was to get our customers more involved throughout the development cycle to ensure that the number of surprises on both sides is kept to a minimum.

Only a few weeks into this incentive I have encountered 3 separate cases where communicating with the customer directly has resulted in a significant benefit:-

1. Clarifying the usage reduced the scope

One of the requirements was for the customer to be able to view metadata fields alongside their existing data fields in their archive from ODBC/JDBC clients. These fields should only be visible when the customer wanted to view them, and not appear the rest of the time. Solutions in discussion were all centering around enabling the fields somehow through syntax functions or hints in syntax comments. Perceived problems were whether the client tools would strip out comments before passing back to the server, and whether tools would hit issues having cached the data schema on connection if suddenly the schema were to change. I contacted the customer to clarify the requirement and established that it would be acceptable to create a separate database connection if using these metadata fields. This meant that we could switch on the metadata fields in the database connection, thereby avoiding any complex testing of syntax parsing and wouldn't be susceptible to unknown client tool behaviour. Simply communicating with the customer reduced the scope and complexity of the requirement to a far simpler case than was being considered.

2. In removing an assumption which was not valid

Being a pattern based storage system, one of the limitations on building a table is that the table have more than one column in each table. One of the requirements being developed was to allow the populating of some data columns with fixed literal values rather than the customer having to add those values into the source data for every row of import data. One of the development/test assumptions was that imports of single column source data would not be supported. However, on contacting the customer I established that it could be the case that they did require multiple column tables where all but one column in the table could be specified as a literal, leaving only a single column to populate from the source data file, thus the assumption was not a valid one. Removing the assumption caused extra effort to work and test the new use case, but the result was functionality that would support what the customer wanted to achieve and avoid any nasty surprises on delivery.

3. A demonstration of functionality flushes out inconsistencies.

On a long running, multi-story 'epic' requirement I felt that some uncertainty regarding the customer requirements provided an opportune time to run a demo of the functionality developed so far and also what we were working on at that time. As a result of the demo a number of inconsistencies between the use cases in development and the customer's perception of how they wanted to use the delivered functionality were uncovered which necessitated some further negotiations and clarification on both sides to resolve.

Now some folks who are working alongside their customers in an Agile context may say that these kind of cases are obvious and dealing with the customers is what we should be doing on a daily basis. I agree that this is great if you have that level of access to the customer, however as Gojko Adzic points out in his blog post The Mythical Customer Problem, the idea of a single point of access 'Product Owner' is often not realistic and instead, as we do at RainStor, collaborative specification is the most effective way to ensure that what is delivered is actually what the customer wants. In this kind of environment it is not always clear who is responsible for communicating issues, questions and progress back to the customer and at what stage. By taking the responsibility of doing this I intended to, and succeeded in, demonstrating the value that more frequent communication with our customers can add. I can then use these cases as arguments for putting in place more defined methods and responsibilities for customer communication on further developments, and ensure that cases such as the ones listed here are cleared as a natural part of our development approach in future.

Friday, 20 November 2009

Testing the patient

A lot has been written recently about the concept of Testing vs Checking (the first reference I have seen on the subject was Michael Bolton's developsense blog) The argument makes an important distinction between the 'Checking' that is performed by either a machine or human performing scripted testing against a set of narrow pass/fail criteria, and 'Testing' which is a sapient process of investigating quality, constantly reviewing and amending the approach based on discoveries made and the expertise of the Tester.

While I agree wholeheartedly that the distinction is relevant and necessary, I am not completely comfortable with the meme chosen to represent the concept, and the re-definition of terminology involved. A recent discussion on this point in the Agile Testing discussion group got me thinking on this point and trying to produce a suitable metaphor to explain my thoughts.

What I came up with was the concept of a doctor examining a patient. The doctor will perform a series of tests in order to make a diagnosis of the patient's condition. Initially these are likely to be general tests, or related to specific symptoms, however as information comes out of these tests, the doctor will use their knowledge and expertise to identify further tests to gather more detailed information and allow them more accurately assess the patient's condition.

Once a diagnosis is made, and a course of treatment prescribed, often the patient is required to be checked regularly to ensure that their condition does not worsen, and that the initial assessment and course of treatment remains valid. These checks will most likely take the form of performing some kind of test (e.g. a blood sugar test for a diabetic) and having some simple criteria to check against the result of the test to give confidence that things are still OK. The patient does not have the medical knowledge to reassess their condition or change their medication, they simply have the ability to perform a test and check the results against a set of check criteria. Should the results differ from expectation then the expertise of the doctor will again be required to re-assess the patient's condition and recommend the appropriate action.

The point that I am trying to make with this example is that, both the doctor and the patient are testing. One tests as part of and assessment process based on their expertise and knowledge, the other as part of a basic checking process to give confidence in the persistence of the previously performed assessment. To label one of these actions "Testing" and one "Checking" could potentially be confusing, and, to my mind is not necessary given the wealth of words we have that may be more appropriate. I'd personally find it easier to present the concept in the sense of testing being made up of two separate important strands, rather than attempt to redefine "Testing" away not only from its literal meaning but also the context specific meaning that it already carries in the development community.

Thursday, 15 October 2009

Learning how to cook

My favourite cookery writer is Nigel Slater. The reason for this is that Nigel, by his own admission, rather than providing a set of recipes to follow, instead aims to teach the reader how to cook. His opinion is that there are too many variables in play in the preparation of a meal for the blind following of a recipe to always produce a successful result, instead he aims to provide the skills required to apply to allow us to handle the variables and still produce good food.

My stance of software development and testing is very much on this basis. We cannot expect a single approach or set of technologies to be universally applicable to every development project. Instead we must use the skills and experiences that we have developed to try to apply the most appropriate processes in any given situation. In this I find myself aligning very strongly with the Context Driven approach to software testing advocated by Cem Kaner and James Bach.

Although in my current organisation we are striving to work in an incremental agile scrum approach, I know that some of the methods that we use do not align with the 'classic' approach to Agile. Whilst some of the differences may indicate 'bad smells' that we should aim to resolve, in some cases it is simply the nature of the product and the needs of our customers and the rest of the organisation that dictate our approach. For example, our customers do not want high frequency deliverables of small amounts of new functionality. Given that each release that they receive must be run through their own thorough test processes prior to a significant upgrade process on multiple client sites, they want major functional changes delivered in their entirety on longer term release cycles. We still aim to deliver working software at the end of each sprint but, for most sprints, this software will not be delivered to the customer.

Does the fact that we deliver large scale changes to the customer on an infrequent basis mean that we are not 'doing agile'? I do not believe so. We still produce working software regularly, verified by continous integration builds and nightly regression tests. We aim for continous improvement with daily standups and sprint end retrospectives with a team that is empowered to make decisions on how to run itself. If we have some aspects that do not fit perfectly with someone else's idea of an agile development process, I am not overly concerned, after all as Nigel Slater says "Blindly following a recipe as if it were a chemical formula is not really cooking, it is just going through the motions"

Tuesday, 6 October 2009

Are testers paperweights?

Jack Dee once did an excellent stand-up routine on craft fairs, in which he was particularly vitriolic about a stall selling paperweights. Something along the lines of "A paperweight is what you end up with when you started out to make something else. You start out with grand ideas and then after a few minutes of trying you realise that, well , it is just another ***! paperweight".
I sometimes feel that software testing is perceived as a bit of a paperweight career, one you end up in when you started out to be something else. People start out writing code and can't cut it, or start out in a business area and want to "get into I.T." so end up in testing.

I'd agree that for many people in testing, it is not the role that they were working towards from the outset of their career, however, for many who come into testing from other areas, it can be a surprisingly challenging and rewarding career which shows a number of significant advantages over the traditional development roles.
- Testing develops non-technical skills which translate across technologies and remain valid over time
- Automation allows opportunities to develop/script in a range of languages, often with the flexibility of choosing the technologies to use
- Finding and diagnosing faults requires analysis and problem solving skills often to a greater degree than was required to create the original code
- Turning user requirements into test cases requires excellent communication skills at both a technical and a business level which can be invaluable skills to take into higher level managerial positions.
- There is a growing community of professionals doing excellent work to improve the perception and the status of software testing within the IT community.

Although for many people a software testing role was not their first choice, there are many out there who remain in the job thanks to the challenging, varied and demanding role that it can be.

Thursday, 30 July 2009

A practical example of lean principles

My hobby is sport climbing and myself and a friend spend a couple of hours a week down at our local climbing gym climbing sport routes. Now we are both reasonably fit men, well into the age that professional sportsmen are described as 'veteran' but nevertheless not in bad shape. We are also competent climbers, by no means excellent but we can generally climb a 6B grade if we push ourselves. While at the centre I often watch the better climbers to appreciate their technique and try to pick up some tips. It is not uncommon that these expert climbers that I find myself most in awe of are children and teenagers. They seem to effortlessly scale routes that my partner and myself wouldn't even consider attempting. Now I have the advantage of strength, I hope that I have more of the wisdom that comes with age, and I have certainly been climbing for longer, so how is it that these youngsters can make it look so easy? The answer is that they are leaner, both in their physique and their approach, which helps them to achieve their goal more efficiently than I can. It is possible to identify a number of factors for success in the sport that translate well into lessons for software development and testing:

Only apply effort that helps you to achieve your goal

My temptation when climbing, as with most men, is to 'brute force' with your arms. In this way I often apply excess energy which is lost on the wall, when a touch more finesse to apply the effort in a more focussed manner would achieve the same result using less energy. If we can identify the activities and actions that contribute specifically to achieving our goal and perform only these actions then we will achieve success more quickly and with resources to spare.

Only carry the weight that is needed to help you achieve your goal

Although not fat, I am bulky when compared to a 12 year old, and this excess weight can be a cause of drag. Similarly if we have bulky processes, excessive documentation to maintain and too much manual testing then this can limit our progress and use up valuable energy.

Do not fear committing yourself with just what you need to progress

One of the real skills in climbing is knowing how much of a purchase on the wall you need in order to perform a move. The mediocre climber such as myself will not trust in his ability to make a move with limited hand/foothold and so will waste energy finding unnecessary reassurance, when the better climber will move when they have just enough purchase to progress. Similarly a the development/test team can learn that it is possible to commit to work and progress without technical specifications, design documents and the like and just start with a one-line story and a plan to discuss it, then this will free us up to embrace agile/lean processes without the false reassurance that those documents provided.

I'm off down the climbing centre again on Thursday, and will hope to practise what I preach. As with all things, it is easy to talk about the best way of doing something, however whether I can apply these principles when I'm halfway up a wall and my legs have just started shaking like Elvis' is another matter ....

Sunday, 19 July 2009

A Sisyphean Task?

I thought that as my first foray into blogging I would elaborate on the title and purpose of my blog. If you are unfamiliar with Greek Myths, Sisyphus was a king who was punished with the task of rolling a boulder repeatedly up a hill only to watch it roll back down. A Sisyphean task is one that can be seen as repetitous and unavailing.

A somewhat depressing title for a blog, you may think, however for I think that many peoples' perception of the task of Software Testing is not too far removed from this. If we equate the height of the boulder to be the confidence in the piece of software under test (SUT) then the task of testing the software could very much be seen as a Sisyphean Task. We execute tests to push up the confidence in a piece of software, only for new functional and environmental changes to be introduced that cause this confidence to roll back down. We then have to push the boulder back up, negotiating all of the newly introduced obstacles whilst at the same time covering the old ground of existing functionality to ensure that no new problems have been introduced.

Where then, lies the appeal? If testing is truly a Sisyphean task then how can it appeal as a career to intelligent and creative minds? I think that the answer can lie in how the task is perceived. If we amend our viewpoint of Sisyphus' labour from a manual task to an engineering one, then the challenge becomes significantly more interesting. How can we speed up the process of moving the boulder up the hill? What solutions can we engineer to reduce the manual effort required to increase our confidence in the system. What are the most efficient ways of increasing confidence without the need for repeated effort? How can we ensure that these solutions are flexible enough to cope with the changing landscape in which they are required to work? It is in creating the most elegant, flexible and efficient solutions to performing the Sisyphean tasks that define the work of the softare tester that we find one of the most sastisfying challenges for the increasing number of creative minds that are choosing software testing as their area of expertise.