A NEW HYBRID SOFTWARE TESTING AUTOMATION FRAMEWORK

Nowadays, with the development of software technologies in all areas of life, software testing, which is an indispensable need of the software life cycle, has become open to development and change. The replacement of manual testing of software products with test automation systems that minimise the human error margin provides the most important example of the transformation of testing processes into a dynamic structure.

This paper details a hybrid approach that utilizes tools such as Java, Maven, Test-NG, etc. used in a traditional test automation. The proposed hybrid approach is based on Java's "Write once, run anywhere" approach. First of all, all HTML elements used in web applications are determined and these elements are saved in a model called PageFactory. PageFactory is a Page object model concept that allows managing elements from a single file. These saved elements are transferred to the next layer, the Test Layer architecture, using the relevant methods in the Page Layer model, and the methods in this layer are invoked and executed. This proposed hybrid approach provides reusability, easy maintenance, cost reduction, increased performance, better quality software products, ease of use and time savings.

Keywords: Software Engineering, Software Testing, Hybrid Test Environments

 The importance of software testing is increasing day by day with the inclusion of software products in all areas of life. In this context, minimising the errors that may occur in the testing of software products will undoubtedly make a positive contribution to increasing the quality of the product.

Bhondokar et al. 2015 proposed a hybrid test automation framework that minimises human intervention and can be easily applied to different organisations with minimal changes in the future [1]. In Wang and Du's study, a Selenium and JMeter based software testing framework was designed for web applications, and it was found that this proposed framework increased the development efficiency of software products [2]. In another framework model proposed by Kim, Na and Ryoo, the architecture of a test automation framework that combines reusable components is presented [3].

In this paper, a new hybrid framework architecture is presented in the field of software test automation. In the second part of the paper, the technologies used, in the third part, the proposed framework design, in the fourth part, the working steps of the proposed design, and in the last part, the results and evaluations obtained from the study are mentioned.

 Java is a widely accepted object-oriented programming language developed for writing portable, operating system independent, portable programs. Java is an open source programming language that is step-by-step executed and compiled [4].

 Selenium is an open source and free browser test automation tool. It supports all browsers and three main operating systems, Windows, Apple and Linux, and runs smoothly. It also allows the use of multiple programming languages such as C#, Python, Java for test scripts [5].

The page object model (POM) used in Selenium is a very common design model

used in test automation projects. Thanks to this model, code repetition is prevented and the maintenance of existing codes is much easier. Figure 1 shows an example of Page Object Model.

 Maven is an automation and build tool used to create and manage Java projects professionally. It enables the elimination of library dependency and IDE dependency in software. Maven also supports various software languages such as C#, Ruby, Scala, and Python [6].

 TestNG is a test framework developed with JUnit and NUnit logic. The most important differences from the other tools mentioned are; It allows all tests to be performed independently of each other, creates test reports, and allows the error messages of the tests that have errors as a result of the test to be added to the report. Some of the advantages of TestNG;

- Multi thread test support,

-Easy grouping of test scenarios,

- Parameter usage

- Flexible test configuration

- Data Driven testing with @DataProvider

- Plugin support for IDEs

- Logging

- Support for dependency testing [7].

 The system architecture developed in this study is shown in Figure 2. The architecture consists of three main parts. These sections are configuration files, data provider and report generator sections. The test automation codes within the architecture are based on the "Object Oriented Programming (OOP)" approach.

 The file specified with the title "Test Config Property File" is the part where the general configuration settings of the tests are made for test automation (Figure 3).

The parameters in the Test Config Property File depend on two Java classes named TestBase and Browser. The TestBase class contains the code that runs the environment and the URL connected to this environment. It also provides the creation of the DataProvider and the enhanced Extend Report. Another task is to define repetitive codes in TestBase using TestNG Annotations. These annotations are presented in Table 1.

Browser class contains the codes that enable the browsers in the Test Config Property File to run. Whichever browser is selected in the Test Config Property File file, that browser runs, and each of them can also be run on a different browser when parallel testing is required. 

 It is an XML file containing the necessary information and configuration details for the build process about the project by Maven. This file is actually a project file that contains and manages information such as project dependencies, plugins or targets that can be run, build profiles, project version, description, developers, mailing lists. Figure 4 shows the contents of the POM file.

 The report generator which is one of the architectural components is an open source, HTML reporting library for Java and .Net programming languages that enables the automatic generation of test reports in Selenium test automation [8]. The advantages of Extent reports can be listed as follows;

 - More customisable than other report types,

 - It can capture screenshots at each test step,

 - The time and date information for the executed test can be displayed in the report,

 - All test scenarios can be run on a single report screen and the results can be easily monitored,

 - It can be easily integrated with frameworks such as JUnit, NUnit and TestNG,

An example of a successfully completed test report generated by Extent report is shown in Figure 6.

An example of an errored test report generated by Extent report is shown in Figure 7.

Another approach to test report generation is open source Cucumber's Behaviour Driven Development (BDD) approach, which prepares test reports in a colloquial language structure. BDD creates test reports using keywords such as "Given", "When", "Then" [9].

 The working steps of the proposed design are shown in Figure 8. The study consists of four main sections. These sections are; Web Application UI, Page Factory, Page Layer, Test Layer.

 Web Application UI section is the area where the application to be tested is located. Each HTML element that we will operate on the relevant page is called an element, these elements are components such as pictures, text, buttons, figures, tables on the page. Firstly, all web elements on the page are determined. They are added as elements to the WebElementi class in Selenium. Figure 9 shows the elements on a sample page.

 It is a Page object model concept that allows to create an object repository in the project, sustainable, reusable and allows to manage elements from a single place. After the elements to be used in the application to be tested are determined, these elements are named and saved with @Findby in Page Factory as the 2nd stage. In this way, the location of the element is determined. Figure 10 shows a sample page factory class.

 Page Layer is the layer where all the pages to be tested in the application are located, the elements named with @FindBy in Page Factory are used in the relevant methods in Page Layer.

 After the elements are used in methods in Page Layer, these methods are called and executed in Test Layer. The method of putting a dot at the end of the method to be executed in Java is also included in the Test Layer here. Thus, other methods can be called without the need for another test page. Figure 11 shows a sample Test Layer code example.

 In this study, a framework is proposed for fast and accurate software testing. Thanks to this framework, it is ensured that the setup of the test environment is created quickly by defining which test will be run in which browser and environment. In addition, changes and updates are easily performed with a single point of control. This ensures ease of adaptation and maintenance.

 Another benefit is that in the existing automation tests, DataProvider must be run repeatedly for each test, while with this proposed hybrid framework, it is enough to run DataProvider once for the desired number of tests. With this automated work, recurring code complexity has been prevented.

 Another advantage of the framework is that screenshots of each step can be taken, test scenario steps can be added, and the duration of the actual test can be seen in the generated reports. In this way, a detailed test report that can be easily analyzed is generated.

 Thanks to the easy management of the proposed framework, new staff to be included in the test teams can be easily adapted to the team. In the future studies, it is aimed to adapt the proposed test automation framework not only for web applications but also for the testing of mobile and desktop software.