My manager has said we are doing either automated testing or manual testing. This sounds wrong to me. How can I convince my manager that a mix of both might be good?
Answer: Maik Nog – hanseatictester.info / @maiknog
You have to write up an scientific sounding essay series about the advantages of “Automated Manual Testing (AMT)” and “Manual Automated Testing (MAT)”. Blog about it and make people follow and support your claims.
In the next meeting lecture about the new concepts and confuse your manager, so that he will naturally agree to implement AMT/MAT.
Answer: Rob Lowry – roblowry.wordpress.com
Use automation as a tool, just like you use tools for manual testing when appropriate. Neither, as an exclusive system, are ideal for every project, so why would you want to restrict yourself?
Find a mix that would be good for your team or project and explain the strengths of each when mixed. Show how automation in one area is the right call, but in another area would be weak. Show where manual testing will give the depth of testing you need where appropriate.
Answer: Sakamoto Kazuma – sakamotokazuma.com
Automated testing is a good time-saver, but keep in mind that there will be a lot of things that can’t be found with automated testing. Say your product is expected to return a message A for a certain process, and message B for any showstoppers. Automation alone can’t determine whether or not you receive a showstopper error unless you spend a lot of time developing your automation to recognize what is and what isn’t a showstopper for all error cases. Isn’t it more time and effort efficient to test this manually?
Answer: Christian Baumann – agile-and-testing.chriss-baumann.de
To quote a former manager of mine: “A goal of 100 % test automation is not a strategy, it´s a fantasy.”.
It is just not possible to automate 100% of all tests, be it for technical reasons, be it that automated testing would be too expensive, or whatever…
Meaningful testing always requires to be done by a human, it needs real thinking, instead of just stupid repeating of the same scripts over and over again…
Furthermore automated testing is not possible without manual efforts. E.g. if an automated test fails, a human needs to investigate for what reason: Was it real bug? Who enters this into the bug tracker? Was it a false negative? For what reason? Needs the test script to be updated? □
As a true advocate of Automated Testing , I am sometimes forced to admit not everything can be automated , or to be more accurate , not everything is worth automating (time constrains , costs etc.) …
Normally it would take an expert in that field to show and explain the logic behind unifying Automated and Manual testing.
Many automated testing projects fail to show ROI due to the reason you raised before and are declared a disaster. Many organisations will not even consider trying to automate testing after such an experience.
Although I am not familiar with the product you are testing , I would suggest you present to your manager the benefits of Automation , as being an enabler to allow larger and more in-dept test coverage. Go with him through the pros and cons of every testing method.
A good method to show that would be calculating the number of permutations needed to cover all possible scenarios of a specific window . In that field Automated testing has a big advantage on Manual testing , however it also have a big disadvantage as well. Not all permutations make sense from an application point of view. This will mean a Human Tester must go trough all these 1000′s and sometime 10000′s permutations and determine whether they make sense.
Another method is to raise the testing over night issue. Many Test Automation advocates pray for a solution that will enable an overnight test runs, that would not require Human intervention. I personally was able to implement such a solution in 2 projects I managed, however it takes real in-dept expertise to implement such a system, and sometime would require your development organisation to take part in the effort as well as the Automated testing team. But then who looks at the results after the nightly run ? who would analyse the errors ?
Make sure your manager understand that Manual Testers often become subject matter experts of the product they are testing as they are probably the only team that looks at a product end to end after its being built (developed). Automatic Tests Analysts are programmers , who have a specific knowledge on the correct method to use an automation software (such as QTP). They would expect to get scenarios already prepared for them by a someone that know the application and the possible scenarios. These would be best prepared by the manual testers. Same way that developers would expect to get a product design from the design team , or their equivalent in your organisation.
Hope the above helped you, if not only pointed you in the right direction.
What was signed off in your Test Strategy? If sufficient analysis and time was spent producing that at the required detailed level, not some wishy washy high level generic rubbish, then that’s your answer – easy.