Software

Understanding Software TestingTesting software is vitally important, but why does it matter, and what does it involve? Erik Boelen, Business Line Manager of Software Testing and Consultancy at Testronic Belgium shares his expert insight

Why should our company consider having our software tested?

Software testing is vitally important and the reasons are simple. Those with software to release want to have the highest quality product on the market, but they don’t typically want to spend a significant proportion of development time and money making sure it offers their customers the highest level of quality. That’s why they pass it to us. We can do it for them quickly and efficiently.

Regardless of which industry or sector you belong to in the digital field, if your product is not hardware, it’s software… and if it is hardware, software is what drives the hardware and gives it a purpose. Whether it’s games, set-top boxes or a movie streaming portal, software is the key component that drives your product and its success or failure is dependent on its error-free operation.

As for functionality specifically, it is important for our users to see whether what they promise their customers and what they are actually providing is equal.

With compatibility testing, which is about making sure software works with different configurations of laptops, tablets, mobile phones and so on, it is vitally important because today so much software – in such competitive markets – needs to be compatible with as many platforms and so on as possible.

What kinds of software needs testing?

The short answer is ‘everything’. We test very simple websites for clients like Google, including web and device applications, testing them against any number of desktop and hardware configurations. We also test firmware - software embedded in hardware - as well as many digital television services, including the menu interfaces and structures. Similarly, we test the whole process of Video On Demand (VOD), testing from an end-to-end point of view, where we are working with disparate technologies across a single service.

Currently, we are working on testing procedures for software that serves the emergency services and for the new generation of Connected Cars. Similarly we can test the software inside a plane’s systems. It really is any kind of software.

I’ve not been involved in the testing process before. What does it broadly involve?

We can come on board at any time of the development process, but ideally we become involved before the software has been completed. Ideally, we’d partner with the developer at the drawing-board phase. This isn’t always the case as QA, in many cases, remains as an afterthought. The advantage to working with a client early is to effectively plan based on their requirements as they develop – release dates can be adhered to; errors are found and dealt with more readily. At the preliminary stages, whether there is a prototype, documentation or just the concept, we can begin to work out what will need testing and how to best test.

Once the developer produces further documentation and the elements that will need testing are identified, a plan on how to best apply that testing can be produced.

Then, when a client agrees to our tailored approach to their software – and starts to deliver the software or parts of that software – we can execute the test scripts and test cases specifically created for each project. And this is what QA boils down to: checking the actual results against expected results. Any discrepancy between those two, then an issue has been identified. It’s that simple.

Compatibility testing seeks to test a client’s application against the most ‘likely’ computing environment the software is likely to be run in. Both the developer and the publisher would like their software to run error-free on every device possible and on PCs and laptops of every configuration of browser, OS, hardware, graphics cards etc. Obviously it is impractical, both financially and time-wise, to test on every device with every configuration. The key is to streamline the process by identifying the core components to running any software in order to return reliable, thorough and representative results.

Once issues are identified, the developer can work to resolve them. Then they re-submit the software and we aggressively retest. That process is repeated until we have a final release version of the software that is of the quality needed.

Software

Does the process end at functionality and compatibility testing?

There are many more considerations to take into account before a piece of software is fit for market. It all depends on the nature of the software and its intended usage. A Web application will need to have its performance monitored against an increasing number of simultaneous users; a real-world load can be simulated of up to hundreds of thousands of simultaneous users with unique behaviour patterns to show how a website or application performs from an End-Users perspective under normal and anticipated peak loads. How many users need to log-on to a site at the same time before it breaks or how many updates or in-app purchases need to be made simultaneously before the system chokes?

Even if the website or application holds up to the demands of seemingly overwhelming numbers of peak-time users, our clients also need to consider how easy the software is to use; in the case of certain types of websites you would need to gauge how easy it is for the End-User to do business with you. What stops a user from making a final transaction through your site or app? Is it the software itself? Usabilty Testing and User Research identify a wide range of possible deterrents in software, websites and apps, such as navigation, emotional response, ease-of-use, etc. We provide a report on improvements from a comprehensive analysis based on user feedback, which once dealt with, can successfully aid customer retention.

Not only can the End-Users’ behaviour be simulated, Usability Testing is sophisticated enough to also simulate the role of the operational teams maintaining and servicing the software. For example, for Digital Television (DTV) content providers or for the team updating Set-Top Box (STB) menus.

The client may also need to carry out tests from within the software itself - Black Box and White Box Testing. With Black Box Testing, a certain interaction has an expected result and we simply monitor and record the process and behaviour. With White Box Testing, the software is checked for elements like component integration where we check what the software is doing at component level rather than at the end result as with Black Box Testing.

This highlights that to effectively check the integrity of software, we have to consider much more than just compatibility and functional issues.

But couldn’t we just test our software internally?

You could try it and see how far you get. People often think that you can perform adequate testing with a normal user or your intern. Some people have ‘purpose-built’ test departments. Do we think having your own test department is a bad idea? No. But operating with no external, objective input definitely is. When testing is left to the developer themselves or even an in-house test team, they are often far too close to the project to see the wood for the trees. Even with thorough training, which is costly, an in-house team will not have the scope of knowledge and experience a dedicated QA company like Testronic has… never. This breadth of knowledge provides peace of mind for every eventuality, many of which would be missed by a dedicated team who may not be conversant with all the potential pitfalls that exist. In this case, supplementing your in-house testing with an impartial, external QA company like Testronic will cover any aspect which might otherwise be missed on release.

Outsourcing testing has a more immediate benefit: you can use us as and when you need… instead of maintaining the financial burden of a full-time internal test team; a full-time team will inevitably mean down or bench time for many of your testers as their utilisation is dependent on continual development, which for many companies is not the case.

In order to test thoroughly with suitable scripts and to deliver meaningful results, with an extensive knowledge of pre-existing issues, you need a professionally trained testing team with a significant amount of on-the-job experience. That is why companies turn to us.

Could I just choose not to test my software?

You could. Good luck. The risks are very wide ranging and the consequences could be grave or financially crippling. With aeroplane or emergency service software people could die. If your software does not work on most of the most popular devices or your checkout system is not secure will ensure shareholder dissatisfaction for sure; an unsatisfied customer and the related negative media attention it could bring can harm a company beyond repair and that can be avoided with quality testing.

Contact usTo discuss how Testronic can help your business, contact Erik Boelen: Erik.Boelen@testroniclabs.com or call: +32 11 560 429

For more infomation visit: www.testroniclabs.com/software/overview