Once upon a time…

Realizing the challenges

During the years at the university I learned many basic skills but what really caught my interest was when we studied large scale development and software development processes. We came across the waterfall model, the spiral model and I remember a lecture with Tom Gilb discussing iterative development. At this point I and my fellow students started to understand how difficult software development could be but it was not until we spend a whole semester working on a specific project we realized what it was all about.

We were assigned a real-world customer with an idea of what they wanted. During the following months we gathered requirements, learnt new technologies, decided on processes, templates and how artefacts were supposed to be version handled. The system was developed, tested and finally delivered to a satisfied customer. Later we studied many of the problems seen in more detail. We discussed software quality, inspections, verification and validation at a number of occasions but there was no course that focused on this!

Many questions; few answers

After my M.Sc. there still were many questions but few answers so I started as a Ph.D. student in search for them. At this point in time CMM was very popular and the processes should be well defined and repeatable. Experience Factories were discussed and the intention was to deliver software with a specified quality to a specified cost. The idea was to utilise experience and artefacts from earlier projects to tailor processes to achieve these goals. Another hot topic during this period was formal requirement specifications. According to my knowledge this never took off, neither the experience factory concept, probably because it required too rigid structures.

I found many answers during my Ph.D. studies but as you probably can guess, more questions were generated. The dependent variable in my prediction models for fault-prone components I used “number of defects found in test”. Unfortunately I had little insight into how things were done and I wanted answers about how the testing had been carried out, what tools had been used and which test design methods had been applied.

… and here we are today

As a result of this I started my professional carrier in 2001 as a consultant focusing on test. I worked with many different test activities and roles but focused on improving test processes and ways of working. When companies contacted me they wanted to improve their testing, but very often it turned out that they actually managed to do their best, based on the circumstances. Instead we had to improve other parts of the development process, for example, requirements handling and configuration management.

Companies are still struggling

Now it is the end of 2010 and after two different management positions I am back as a consultant working with quality assurance. So what has happened the last ten or even twenty years? Have we made any progress within the test area and what does our processes look like? Do we still try to create large rigid structures as a base for our development? Is testing becoming an accepted carrier path?

My first impression was that it was the same old story that was told. Companies want to improve their testing but when looking closer the problem is not test itself. Again it is other parts of the development, for example, poorly written or non-existing requirements, people who thinks configuration management is something from outer space and a very meagre understanding in the management teams why testing is important. Ten years have passed and the companies still struggle with the same problems!

The agile way

At the same time I started to think about development processes and how rigid they used to be and I must say that more things have happened here. Companies try to work more iterative and agile. The introduction of eXtreme Programming, Scrum, Kanban and other lightweight processes has started to diminish the rigid structures and put more focus on delivering business value. In that sense things have changed compared to when we first thought that we could solve the problems using formal specification methods.

Test taken seriously

Of course there has been a lot of progress within the test area as well. Tester is nowadays recognized as a profession, more training is available, different certificates can be taken and test tools are developed. Looking at the increasing number of attendees on different test conferences we can see that things are happening. So one reason for seeing the same problems again is that it is not the companies that have done the improvement journey who contact me; it is companies that previously have not seen IT as a main part of their business and now realized that they need help and a direction for their journey.

Some kind of “Scrumish” process

Looking at the conference program I notice that a popular topic is how test should be handled when Scrum or some other agile method is applied. It is herein the problem lies, i.e. when setting up cross functional teams the test competence is often foreseen. The solution then seems to be to adopt the “normal” way of carrying out test so it would fit. The result is some kind of “Scrumish” process that has a non-optimized mix of old and new ways of working.

Time to embrace development and test as an entity

From my point of view it is now time to do something more drastic and not try to adapt to the developers way of working. There has been too much walking around on the toes and the developers have been untouchable. It is now time to tear down the old barrier between development and test and see developers and testers as equal valuable competences. We need embrace development and test as an entity focusing on quality assurance and providing business value to the customer.

Cross functional teams is a way forward

According to my opinion I think the cross functional teams is a way forward but one obstacle is the nature of the test role. A developer can focus most of the time developing code. New requirements are coming in and the language might change from C to Java but the tasks are more or less the same. The test role is more multifaceted. A tester is expected to be able to write specifications, automate test, mange teams, control test environments etc. Like a circus artist the tester has to be able to juggle at the same time as taking care of the elephants. You all realize that it is impossible to become a specialist in all of these areas but that is often what is expected. Here I think the consultant companies play an important role to provide the right competence in the right project phase and in companies´ different maturity phases.

there is more to do

I do not have the final solution today but the building blocks are there and we need to put them together so that they fit different organizations’ maturity. The whole development process needs to be in focus together with quality assurance and not test only, because I do not want to listen to the same old story again in another ten or twenty year. Things have improved but there is more to do so stand up, be proud and get involved early in the projects!

Author: Dr. Magnus C. Ohlsson, Test Strategist at System Verification.
Phone: +46 73 661 28 60
Email