From manual testing to test automation

Testing is a must-have procedure in software development process. The question is what testing you need, what costs you will pay and how to lower these costs with test automation.

Costs associated with software testing

The process of software development is complicated and can last years, there can be and there will be some bugs, mistakes, errors, functions that stop working under certain conditions etc. Sure, there are better testers and users that help to find bugs in software, but the cost of having version of software with bugs is too high.

Software testing might be a minor problem application with a army of free testers and beta-tester, but there also expensive corporate software applications that cannot be released without full testing.

There are different method for software testing - manual testing or automated testing, but the answer about if company should test its software is always "Yes", because costs of having software with bugs available in the market are too high.

Advantages and disadvantages of the black box

There are different concepts about how to test software. Some approaches involves unit tests or functional tests. Software developers usually do these tests in the beginning of the development process. Once the first version of application was released, it is a good idea to start with black box testing.

When tester do a Black Box Testing he don't know about any functions or units inside the application. He only knows how the application should work, e.g. what should user do and what results he will have.

The main advantage of Black Box Testing is that:

• Tester will act exactly as user will do. Tester will click exactly the same buttons and work with exactly the same objects. Thus, the black box testing is a testing from user viewpoint.

The main disadvantage of Black Box Testing is that:

• Black box testing is hard to automated, while functional or unit tests can be completed in just few seconds, black box testing requires tester to click buttons, enter texts and work with software graphical interface (GUI). iTestBot software was designed to make software black box testing faster.

So, how to do black box testing in a case of software? We can talk about three methods: manual testing, "record and playback" testing, automation test.

Manual tests

Manual tests should be performed by human tester. These tests are usually do developers themselves and probably special testers. But as long as these tests require a lot of time, they are expensive. Software developer companies try to automate testing process.

Record and playback tests

Software developer or tester can perform testing once and record the test process, e.g. record the position of the mouse, buttons tester click, text tester enters etc. Then, when it will be needed to test application the recorded test will be played back and results will be checked against expected results. The problem is that tests like this are very sensible to any, even minor changes in program interface or system environment.

Automation testing

There are more flexible tools that allow to automate testing. These tools not just record mouse clicks and moves, but allows testing script to emulate user's behavior. For instance iTestBot allows not just move mouse to point on the screen with specific coordinates, but find on the screen some image pattern and focus on it. Behavior like this makes automation testing more flexible.

When software developers use Test Automation tool it doesn't just move the mouse and click. It does as user would do. The testing software searched for image patterns and do some action according to these image patterns.

How much does automated testing cost?

The total price of test automation is the sum of the test development costs, test using costs and test maintenance costs.

• Test development costs. Depend on software used to create interface automation testing scripts. In a case of iTestBot development won't take much time. The time of developing testing script is comparable to the time of actual testing, as software engineer need click just few more buttons.
• Costs of using automated tests. It will not costs much, you will need a separate computer or a free (night) time on your existent computer. If designed correctly running script will require just few minutes to run test and then analyze results.
• Maintenance costs. There are some costs for script maintenance, because if something will be changed in interface the script might stop working. But there will be needed to update just few images, not whole test. In comparison to "record and playback" test, the maintenance costs of automation testing are much lower.

How to create automation testing script?

It depends much on what automation software you use and what are your goals, but there are some common ideas that you should use:

• Don't rely on mouse move to specific coordinates much. In this way your scripts will not be flexible enough.
• The best idea is to focus mouse pointer on some specific image and the move mouse a little to relative coordinates. In this way your scripts will be more reliable and flexible.
• If possible try to use hot-keys on your keyboard. For instance, when you need to paste some text from the clipboard, use Control + V key, it will save some testing time and make your test more flexible.

Comparison of test automation tools

There are a number of test automation tools available in the market. Here is a short classification of these tools. The more functions and features you are expecting from the software the higher total cost will be.

• Record and playback. This type of software operates only with mouse moves and clicks, it is not much reliable.
• Click buttons, move mouse and playback. Software like this have more functions, for instance, it does not just move mouse to the button, but it can click the button by its internal name or caption.
• Using image templates. Tools like iTestBot can emulate user's behavior by finding certain images. Resulted scripts are more flexible and reliable.

Advantages and disadvantages of test automation

One of the main advantage of test automation is that as a result software developers can run tests more often, find and fix bugs on the early stage of development, before end users will face these bugs.

In comparison to human testing the main disadvantage of test automation are possible errors due to change of tested software. This involve some maintenance costs, but the total costs of owning test automation system will be much lower than owning and managing human-based test system.