GUI Testing Automation

GUI Testing Handbook, check other parts: GUI Testing Introduction | Testing GUI in Windows | GUI of Web Application | Automation of GUI Tests | GUI Test Automation Tips and Tricks

Testing Automation is a process by which the repetitive testing practices are automated using Testing Software Tools like Winrunner, load runner, QTP etc., using input of test cases and predicted result outputs under controlled environment. Testing Automation is recommended in converting a regularized manual testing procedure with formal testing process to automated testing process. A common and recommended usage of automation is in Regression testing. Regression testing is a testing which is repetitive in nature. This testing exposes the bugs in the existing software which is triggered by changes made as a result of fixing the software bugs. Regressive testing is repeated until the test is successful. This process is prone to errors during manual testing due to its repetitive nature as the testing demands checking of the entire functionality repeatedly to avoid any errors in the process of bug fix.  

Is Testing Automation an alternate to Manual Testing?

Manual testing is a process where a functionality of an application is tested under different environments and conditions for its intended functionality, interface, usability, performance, security etc by human testers with required inputs from the application development team and the application documents of various natures. Manual Testing has its advantages and disadvantages. Ad hoc testing is possible in manual testing and it is needed most of the time as the functionality of the application is better understood by a human tester. Practical experience proves that Ad hoc testing exposes more bugs than in an automated testing. The disadvantage in manual testing is the rerun of the tests repeatedly in a build fix which is cumbersome and tiring for any human effort which raises the likelihood of errors slipping unnoticed.

When compared to Manual Testing, Testing automation provides reliability in testing as software tools are not susceptible to tiredness in retests. Testing Automation is a costly process when compared to manual process. Testing Automation requires the working knowledge on the testing tools. Building test cases and building the testing framework take more time than a manual process. Tests like visual reference tests which cannot be identified by code are best tested manually. Testing Automation is a winning recommendation if the application is a large application to manage the costing of testing automation and needs repetitive tests and uses multi languages on multi OS platforms.

GUI Testing Automation

GUI Testing Automation involves the testing of usability aspect of the application. The testing automation should cover all GUI components present in the application. This involves automating a set of actions triggered through mouse or keyboard on the GUI modules of the application and capturing the results to evaluate for its results. Commercially, none of the GUI Testing tools have been accurate in their testing and there are many instances of application usability bugs surfacing on release of the application.

Capture and Playback Method

The most popular GUI Automated Testing tool follows the Capture and playback method. An automated test script is developed and executed to capture and playback low level mouse movements. Very few tools recognize the GUI components as objects while some tools recognize components with mouse coordinates.   More often than not, the insufficiency in the tool generating efficient test script surfaces and the tester inadvertently ends up manipulating the tool, making changes in the recorded script generated and debugging the test scripts in case of failures. This leaves less time for the tester to concentrate on test cases and execution effectively.

Though the word automated is used in the GUI Testing tool, the tool can be best defined as interactive tools. Automation is a term used for non-interruptive processing from end to end. GUI Automated tools fail in this definition as the tools demand frequent interaction as verification points which involve manual entry of data and check points. Once the test script is captured it can rerun many times which is helpful in regression testing. 

Test Monkeys

Test monkeys are automated testing tools used for destructive testing. These tools are called scholastic tools similar to black box testing with no understanding of the application or input of test cases. These tools use random clicks on the GUI using the mouse or the keyboard until the application crashes. These tests simulate the real user and have known to be successful in identifying serious bugs in the application tested.

Test monkeys can be classified into two categories – dumb test monkeys and smart test monkeys.  Dumb test monkeys do not have any input on the nature of the application. They check the GUI by subjecting the User Interface to random activity without any attention to the components or functionality present in the application.  Dumb Test monkey cannot recognize a bug but the applications subjected to testing by these respond to the inputs. This testing has made valuable bug corrections possible. Smart test monkeys are not random testers. They have knowledge on the application and the ability to reflect on the recognition by the application. Smart test monkeys get their knowledge by absorbing the model of the application or by storage and retrieval of state information. They use this knowledge to test the application of the valid values or expected behavior. Smart Test monkeys can also be used to include the functionality of dumb test monkeys for thorough application testing.

Intelligent Automation

Intelligent GUI automation testing tools should have the knowledge employed in different testing methodologies and theories, effective management of software resources, in depth technical intelligence.  These tools should use state table and have the ability to recognize GUI objects and their functionality. The real test of testing does not lie in simple testing but set of complicated sequences and the behavior of the application in that context. This is still the forte of an Expert tester as Automation tools have failed to test sequences of complicated testing tasks effectively.

GUI Test Plan Scripting for GUI Automation testing tool

The systems user interface is analyzed to list out the operations that can be performed on the interface. The operations that can be performed become the backbone for the testing and are called operators of the test plan. The initial state of the GUI components is determined. The target state of the components extending the possible states of the component is listed. The test plan is written taking into consideration the initial states and the target states.

Uses of GUI Automated testing tool

GUI Automated testing is recommended for the following  

• areas of adherence to compliance
• to confirm non presence of broken links
• to ascertain identical user interface across different browsers and multi platforms
• to confirm  User Interface Testing standards compliance
• to identify and address cascading effects of changes in application.
• to confirm functionality
• to ascertain if a build is fit for entering the test phase.
• to track defects and errors repetitively to highlight issues.
• to ensure that each release follows  same functionality in the same way
• to concentrate on areas of test support activities like function specific test data entry
• To identify and detect changes in the user interface when the focus is on aspects of  testing

Application areas of GUI Automated Testing

• In check list testing, in the areas of  checking states of  object, menus and standard window look and feel features.
• Complete automation of Navigation testing
• Large numbers of simple condition based testing with same logical functionality i.e. Decision table based Testing
• Repetitive tests of simple functional transactions
• Testing for multi platform, different operating system and compatibility

Advantages and Disadvantages of Testing Automation Tools 

10 things to remember in GUI Automation Testing

• Automation Tool must be easy to use
• Automation Tool used must support reusability
• Reusable scripts should be capable of being converted to automated test cases
• Automated Testing should start early in the cycle of Application development
• Begin with test scripts creation for simple functionality
• Maintain documentation of reusable scripts and their functionality
• Automation Tool must support storage and retrieval of run time data
• Use Naming standards for scripts, verification points, data storage to ensure uniformity and easy retrieval in the Automated Tool
• Bugs and errors must be properly communicated and closed to avoid redundancy in follow up
• Remember that success in testing using Automated Testing Tool is in identifying the requirements and developing the identified requirements with reasonable goals on testing

Glossary

• Winrunner. A GUI automated testing tool utilizing the record and playback functionality to create test scripts

• QTP. Acronym for Quick Test Professional.  Though a GUI automated Testing tool, it is used primarily for functional regression testing.