Manual Testing - an inconvenient truth

Why manual testing will always remain despite increased reliance on test automation, and how it can still provide exceptional value

Fraser Powell
Fraser Powell Head of QA

Load it, check it, quick-rewrite it, plug it, play it, automate it. Automation is a trend of today in almost all industries and is quickly spreading into all parts of daily life. And it’s great! Just think how it benefits your daily life; from the moment your alarm wakes you up, the central heating in your home triggered by time or temperature, the map app on your mobile phone that reroutes you based on live traffic reports, the computer backup that has run by the time you reach the office, and Amazon Alexa remembering all the things we can’t! Even my bills are being paid on time throughout without any onus on me to be more organised, thankfully! With technological advances continuing to make our daily lives easier, it would be an opportunity missed if we didn’t utilise the approach in our work. Certainly, the manufacturing industry has done so to great effect, but how about software testing?

Could automated testing replace manual testing?

There are a lot of benefits, but also a lot of misconceptions about those benefits. Let’s look at a few that I’ve seen highlighted in the ongoing debate:

Speed

Machines are certainly faster at running tests, sometimes running a whole regression suite in an hour that may take a couple of weeks to test manually. So, 1 - 0 to automation, right? Well, can machines also identify and write the tests faster? No, they can’t do this at all and require a huge investment of resource to reach a point where they can be considered ‘automated’.

Quality

There’s an argument that machines aren’t prone to human error and thus, produce higher quality outputs. Was that a little chuckle I heard?! The premise is accurate, but in reality, we all know machines fail. Secondly, the automated tests are written by humans initially, so in fact, they’re only as good as the person who wrote them. I would agree however, that automation is more consistent – if you want something tested at the same time, in the same way and repeatedly, automate it.

Thoroughness

Some think that automated tests are more thorough and catch more bugs that humans may miss. Maybe I’m confused, but this is the opposite of my understanding! Automated tests run a limited set of tests and don’t run edge cases, they don’t wonder ‘what if…’ and don’t possess a creative, inquisitive nature.

Re-usability

We also hear that the tests are ‘reusable’ and so once they’re written, their associated costs flatline. Perhaps on a stable, fixed system, but then maybe that’s because they would be redundant after a successful run (if nothing changes, why would you keep running the same tests?). They need to be maintained, expanded as the software is developed, ported to handle new technologies, and/or integrations and extended to cover areas where bugs have been identified. And the skill level required to write automated tests is much higher. So, yes, they are reusable but they will still require further investment.

Is QA dead?

I’ve even heard the question; is QA dead because of test automation? Well, that would be a fundamental misunderstanding of the difference between testing and QA and what they achieve. Testing can be automated but is actually only a measure of quality and feeds into quality assurance. QA is so much more than testing, to assure quality we need to understand so many aspects and variables, most of which are human!

I think one of the greatest demonstrations of automation's strengths and weaknesses is the recent trials of self-driving vehicles. We’ve seen that they can be beneficial and have achieved some success, but they’ve also caused fatal accidents. One accident was caused by an unexpected set of circumstances, which is exactly why the human ability to identify an ‘unhandled exception’ is so important. Automation is fantastic in a controlled environment with a specific objective, but it’s not going to think outside the box, it’s going to test the same thing over and over.

Our approach to regression testing

Automation is also often suggested as being most suitable for regression testing i.e. a long, arduous, time-consuming set of tests to be repeated regularly. Again, I agree with the premise, but I also believe there are better ways. At Ghyston, the developers do write automated tests but we run them as part of our build and continuous integration process, thus building in regression testing every time we make changes and rebuild the software. This approach means we can deliver stable builds into QA that have already passed all the existing tests, leaving our testers to explore the software and focus on usability, discovering new use cases and journeys and provide a comprehensive test approach. We also build confidence by running smoke tests and sanity tests pre-release to ensure there is no sign of fire anywhere in the application and that no unforeseen errors have crept in.

Keeping the balance

Ultimately, it’s about knowing that everything has a place and utilising it appropriately. Automated testing helps us cover the repeated tests and deliver stable builds, and helps us to provide efficiency within the project without sacrificing quality. Manual testing provides the human touch, flexibility, it reaches the areas that scripted tests cannot and provides a platform for evaluation from the perspective of users, which is critically important to understand. As with most things in life, it’s about finding a balance.

Fraser Powell
Fraser Powell
Head of QA

We think you'll also enjoy

The Ghyston 2023 Impact Report

Here is what we got up to in 2023 - we are delighted to share with you our impact report for last year.
Learn more

How to make 2024 the year you tackle the big talent shortage

As part of our series on practical business leadership advice for 2024, we’ve put together four steps you can take this year, to make your organisation more effective in its recruitment and retention, and to tackle another common issue: lack of diversity in the team
Learn more

Our investigation into the LastPass Security Incident

Here is our response to the recent news of a security incident at LastPass and our recommendations of pre-emptive actions to keep your data safe.
Learn more

Subscribe to our newsletter

The latest news and industry insights, straight to your inbox