There have been a few articles over the past few years speculating what the future of testing could be like such as Applabs white paper discussing the changing role of the software tester, Cem Kaner’s interesting set of slides (PDF link) which looks at how the approaches used by software testers are changing from v model and waterfall to exploratory, while Sergey Lesnikov talked about outsourcing and the value testers will add in the future I even did a brief blog on the age range of software tester.
I think it would be a good start to explain a little about myself, I have been in the business of IT for over 24 years a fair few of them in software testing, I have been a employee and a contractor over these years, and I have seen many changes, good and bad.
What would I think as a professional tester if I travelled from 1990 to today?
So if I was armed with my DeLorean (*) and could go “Back to the Future” I wonder what I would find?
* For our younger readers who have no idea what I am on about please have a look at this reference for more information.
In some respects not a great deal has changed with regards to the methods and approaches we follow when creating tests. We still follow the same boundary, edge case routines that have been instilled in us from reading great books such as “The Art of software testing by Glenford Myers” (my testing bible) and more recently “A Practitioner’s Guide to Software Test Design and Lee Copeland”.
I find it amazing when I address fellow professional testers how many of them have not read any books on software testing. It is such a shame, maybe it is an age thing and my generation did not have the web and depended on bookstores to keep us up to date with the latest trends.
In the good/bad old days you would spend months and months writing a test strategy, a test plan and then lots of manual test scripts. If you were really lucky you would get involved in a cutting edge prototype project in which there was no major upfront planning, just a lot of playing, experimenting and tinkering (exploring!) with the rough and ready product. I remember working on a few of these types of projects and having a great time and feeling I was achieving lots. Pity that people looked down on these projects and how they worked, the bean counters needed their numbers.
Everything was extremely regimented and processes must be followed, without question or, in some cases, without thought. Testing was seen as an inferior career choice left to those not good enough to be a programmer or to a junior programmer so they could cut their teeth. Within the companies I worked for there was no such thing as a professional tester. It was only later I would discover that there were great people writing great articles about software testing such as Jerry Weinberg and Cem Kaner to name a couple.
As a tester it could be an easy way to make money, you could, as a contractor, spend months and months writing test plans which you know would never be used and get paid very well. Was I ever guilty of this? I can imagine a lot of people who are reading this are smiling and nodding their head in agreement.
Year 2000 came and went without a bang or major bug, wow, what a way to earn lots of money as a tester.
Then the agile movement started to really take off, or as some testers with closed minds put it, the “The get rid of testers development method”. I saw this method as a great way to become flexible in how we worked and how we could as testers work more closely with programmers and customers – the utopia of software development. As many of us now know, the expectation of agile and how it was implemented frustrated lots of people. The mis-information about the need for testers and that we had no role in software development was rife. It is thanks to the agile manifesto that I started to read about great people such as
which in turn managed to introduce me to the writing of:
Cem Kaner, James Bach and Michael Bolton (12)
along with numerous others.
As I tester I have always tried to keep my skills up to date and fresh by learning bits about programming languages and trying to understand the subtle differences. In the early noughties to do this you had to go to conferences, workshops, or use email, the web was just starting to take off. There was no such thing as social networking or twittering. To do that you had to go and meet people – how scary is that?
I was not a programmer nor did ever want to be one. My mind is not fixed on being creative in a constructive way. I’m more creative in taking things apart and trying to work out how it works. So I have learned lots of different ways to code, badly I must add, in Java, Python, JavaScript, Visual Basic etc
So I then found myself being an automated tester because I could do a bit of code, oh dear, that was such a bad thing. It was one of the most depressing times as a tester I ever had. Someone would come and give me some test cases they wanted automating, I would automate them and they would become part of the regression suite.
This set of ‘tests’ (Yes, I will get on to automation checking and testing but that is in the future.) would then need to be maintained which took up so much of my time since the product was ever changing due to working in an agile way. Any new tests requiring automation would not be done since the old tests kept failing and needed re-coding. I did not stick with this for very long.
I was very interested in the TDD method of working and thinking back now, what I was automating was done far too late in the development cycle. Hindsight is such a wonderful thing.
So agile came and is still here and in my opinion has made one of the greatest impacts on the software development community for good and bad. We now have people arguing on what is agile and what is not agile. I am from the school if the approach you are using works for you then where is the problem?
All of sudden certification started to appear. In the UK they became known as ISEB. This soon became the defacto way to become recognized as a software testing professional. I pondered for a moment at the syllabus and thought what value this would add to my knowledge? Did I need a refresher in static testing, test planning etc?
I considered certifications may be useful to have in the future but they seemed to offer nothing more than I already knew from working in the field. Oh again that wonderful thing called hindsight. A couple of years ago a big debate grew up around certification at the Eurostar conference in Sweden. Michael Bolton decided to give his view on why he did not like certification.
It made me think more and more about this and how as a profession, we can recognize our skills and talent. Answering multi-choice questions is in my opinion, not an effective way of doing that. I would hope that by working with more and more people in the testing community they will be able to vouch for my abilities or lack of them rather than a piece of paper that says I got 90% from a multi-choice exam.
Around this time I started to pick up this buzz called exploratory testing, I started to find articles on it and read as much as I could. I was surprised to see how far back the original ideas have been circulated. Why had such important and groundbreaking concept not got more recognition? This to me has been my ‘eureka’ moment. It only happened a couple of years ago but it threw my thoughts about software testing upside down and is still doing so today. To learn about an approach to software testing that is so natural and so right. To meet so many people who wish to share this knowledge and be helpful and supportive has really opened my eyes to the current state of software testing.
We are now a community where everyone’s thoughts and ideas can be expressed in the digital world via blogs, email and twitter. All of these new communication channels have changed the ways in which we can interact and communicate. Instead of feeling alone and isolated we have a whole world of experts in different fields offering their advice and support.
There are plenty of welcome distracters and people who like to cause a good argument. I never say I am right on anything and I love people to challenge what I say and do.
What do I feel is the future of testing? What is it going to be like in another 20 years? Well I hope I will be about to retire and relax but I think I would still like to have some input and give back to a community to that has been so supportive of me. Maybe I will become a boring lecturer talking about the good old days of testing when you worked for a company rather than this crowd based new fangled way of testing. It is so impersonal.
I wonder if I will be a grumpy old man? I think I could be one already; people say I moan a lot but that is normally because the software they just gave me crashed for the 1000th time.
I think the future of software testing will be more community based with testers working and collaborating together to create the best approaches and ideas for testing. This has already started happening with weekend testing in which groups of like minded individual communicate online and carry out a testing assignment over a weekend. It was started by a group of bright testing indivuduals in Bangalore (Parimala Shankariah, Sharath Byregowda, Manoj Nair, Ajay Balamurugadas) and has now spread beyond India to Europe and hopefully in the future to the rest of the world.
I feel there will be more of a focus of automation that aids exploratory testing rather than “let’s automate all the checks” (14)
There is lots of talk about the future of testing being done by Crowdsourcing. I wonder if this will go the same way as outsourcing. The further away the testers are from the development team the more likely there will be some misunderstandings and wrong assumptions. It has taken years for developers and testers to work closely together and have discussions about design and the way forward for a product. Working as a team and having mutual respect. I do worry that Crowdsourcing will break that bond.
I like how agile has changed the way software development is done, bringing a lot more thoughtful automation and exploratory testing into the mix. Exploratory testing is changing the face of software testing.
You may ask why I think this. Well until recently I did not realize that I have been doing exploratory testing my entire testing career but back in the old days I just called it testing. Now it has a name and has a set of experts wishing to share their knowledge. So if I have done it for the past 20 years why should I change now or in the future?
If it isn’t broken why change it?
About John Stevenson
I have been involved within the testing profession for over 18 years and within the IT industry for more that 22 years. I have had made differing roles within testing, during these periods part of my remit has been to investigate new methods and approaches to testing to enable testing to become more effective and efficient part of this has involved writing internal white papers and presentations. One area I have been very active in is the production of internal workshops on exploratory testing, how to manage exploratory testing and using exploratory testing in an agile environment.
I am married and have three grown up children and one adorable granddaughter. I have many interests including gardening, photography and spending time with my granddaughter
Great article, and many parallels with my own career. I enjoyed your balance view of the developments through your career, never swaying too far into one camp.
A generalist tester! Thanks again
I agree with just about everything you said. It is interesting to note that the term “exploratory testing” started out as a “what if scenario.” Many programmers had never been in user’s environment and didn’t really understand how the user might want to deviate from what the programmer thought should be done.
I agree that a tester should not be a programmer and in fact most programmers make lousy testers. Their methodology of testing is if you find a bug, re-write the software to fix, without understand the impact might have on other programs in the system. Of course we won’t mention regression testing as so few people do it anyway.
I am amazed at what many companies expectations are of a tester. They do not understand the difference between testing and quality assurance. Testing is quality control, whereas qa is “process management and root cause analysis.”
Testers now have to be everything rolled into one. Expectations exceed ability and the pay scale is nowhere near where it should be for the job that is expected.
I think it is changing so rapidly that testers will be absorbed into some management position as yet untitled. Whether they will get the recognition or remuneration is doubtful.
I’m new to Testing stuff, as i’m new to professional software carrer (i’m just a 1 year junior-development), just take some courses in the area and read a lot about it.
What other books (besides “The Art of software testing by Glenford Myers” and more recently “A Practitioner’s Guide to Software Test Design and Lee Copeland”, those commented in the article) would you recommend to a newbie ?
Yet, a phrase of you has confused me: “I feel there will be more of a focus of automation that aids exploratory testing rather than “let’s automate all the checks” (14)”
This last “(14)” is supposed to be a reference ? If yes, can you handle us this ?
Yes, I was nodding my head and have observed many similar things over many years in testing. Thanks for an interesting article.
However, there is one thing I would like to comment on. In common with many other people (including many from the “Tester is for Life” booklet, you seem to be criticizing certification (such as the ISTQB qualifications) because it doesn’t assess tester skills.
I feel this is unfair, for the Foundation certificate in particular. Its basic purpose is to “take away the bottom level of ignorance” about testing and to provide basic concepts and language about what testing is. No it doesn’t assess skills, but it was never intended to! Anyway, I will stop here but I have quite a lot more to say about this which I have put in my blog (http://dorothygraham.blogspot.com/)
Beautiful article, from someone who’s been involved in Testing. I would have to agree with you that you shed a lot light on the field
I suppose you guys need something to make yourselves feel good about having a boring and un creative job. I can’t imagine how you can test software unless you understand its architecture from a programmer’s perspective.
To me, it is indicative of the ever increasing trend to certify everyone. You need to feel important and credentialed I suppose.
You seem to have neglected one thing:
“History tends to repeat itself…”
Back in the 80′s and early 90′s it was still common for a developer to test his own work.
It was mainly small code, meant to run on a very specific platform, with limited complexity.
Than the SW industry became larger, more experienced and wise -
The need for unbiased professional test engineers arise, from limitation in developers abilities to “fully” test their own work.
10 years had passed, and Agile hype started to claim the Death of Testing as a separate profession (sound familiar from somewhere).
Has developers methods changed much in that period?, are there suddenly tools which they use to produce less bugs?, have system became simpler or?, has developers salary been reduced?
Now if history do tend to repeat itself… where will we be in 10-15 years?
Hmm, Thought I would chip in here, maybe I am one of the originals but I started testing software in 1978 approximately, I was an electronics engineer from the armed forces before 1975 and I started out as a CAD software user in a company producing that software.
I was trained formally by Dot Graham (and in case she is reading this – I still have the manual!).
I think that overall the issue is that we have 2 sides (Dev and Test) which are regarded as mutually exclusive, they are not – I was a dev manger in the early 90′s but I love testing software, it is a bit like playing a computer game where you have no idea where the next attack will come from.
The 2 sides are NOT exclusive, we are part of a team to deliver a good customer experience. IMHO a good developer is aware of the limitations of their code and produces code that protects the end user (that is part of my Pareto principle – 80 % of the code protects the a…), the unfortunate problem is that more often than not there is only time to code the remaining 20% which actually processes what is required.
Agile provides the producers with the opportunity to combine test knowledge with development knowledge to the betterment of a delivered product.
Personally I think a developer should learn to test software BEFORE they develop it.
Anyway a good tester can support a good developer regardless of whether they have a certificate or not.
Finbally of curse the company I worked for must be an anomaly since we were testing independently of Dev well before the 90′s!