Testing with Selenium: Covering an Enterprise Web Application

Regression Testing a Complex UI

Web applications are a mischievous bunch. When they are born they are usually small, clean and orderly, but they may grow up to become complex and error prone monsters. Each feature added to the mix increases the chance of a new bug appearing, a promise that is usually fulfilled. When developing a large and complex web application, we need to be able to continually check regressions and verify that everything that worked until now is still working. So that at least we won’t break more than we need to.
Taboola is are a web company, and must deliver on a very rapid pace. We ship many features on a continuous basis. Under such circumstances no matter how big your QA team is, it will never manage to cover the entire system for every delivered feature.

This means an extensive testing automation framework is a must.

Entering the stage: Selenium

Selenium is a robust browser automation framework, allowing us to write web tests with many different languages: Java, C#, Python, etc. Simply said: it provides an API that allows us to simulate user behavior on browsers. Selenium supports all major browsers, and provides a relatively simple API to test everything UI.

Testing in Taboola

Taboola’s publishers and advertisers use a large enterprise web application called Backstage. Backstage is mainly composed of forms and reports (charts and tables).

Backstage has grown over time. Features are continuously being developed and deployed each week, many of these by different teams that may inadvertently break each other’s functionalities. At each change we need to make sure we haven’t broken anything. Even with QA teams working around the clock, we need to have the best coverage possible. Sometimes a small fix in one report can create havoc on a different form, and we need to catch that in time. So how can we test this behemoth of an application?

To cover as much as possible, we use different test frameworks:

• Mockito for server unit tests
• Karma for JS unit tests
• Wraith for visual (screenshot comparison) testing
• Selenium for client functional and acceptance testing
• REST-assured for API testing

We strive to cover each possible flow that may occur from user interaction with Selenium. Testing the flows actually becomes part of our server testing, since it validates all integration points. Since it’s almost impossible to cover everything, we cover most of the relevant use cases. Testing coverage is then increased over time with these rules:

• As part of fixing each bug, a developer has to add a coverage test verifying the same bug won’t return
• As part of developing a new feature, a developer has to add tests covering the new behavior. The feature is only approved for production after manual QA and all Selenium tests have passed

Over time this has helped us create a dense coverage of the Backstage application.

Taboola’s Selenium Solution

The following diagram shows our solution:

Let’s examine the different parts.

The WebDriver Framework

To be able to add tests easily, we have created a framework that represents the actual pages of the application. Each element is represented by a Selenium component, allowing tests to interact with the website in our tests, as though we were a user.

For Selenium to be able to simulate user behavior, we needed to create a representation of all the components in the site. Each element is represented by a Selenium component, allowing tests to navigate and interact with the UI. We call this the WebDriver Framework.

Application Components

For each web component type we created a Selenium representation that knows how to react and read data from this component.

It receives the WebElement Selenium object that identified it on the web page, and uses it as a base for all behaviors.

For example, a checkbox component:

The Checkbox’ constructor receives the WebElement that wraps the actual element. For example, if a checkbox can be located by its id, it can be passed like this:

@FindBy(id=”myCheckboxId”)
private WebElement myCheckboxElement;
….
Checkbox myCheckbox = new Checkbox(myCheckboxElement);

The actual components are combined in the application structure section.

Structure

After we have all the necessary components, we gather them to the different web pages in our application. A Page class represents a single web page, and its different implementations allow us to represent all the pages we have and their relations.

Writing the Tests

Taboola’s Selenium solution is powered by JUnit, providing the power of assertions and test focused development.

Their structure is in JUnit fashion, with @Before annotation for test preparation, @After for test tear up, and @Test for the different tests.

We have divided the tests into classes, according to the section they cover in Backstage. This is an example of package structure for our tests:

• Campaigns <- All Campaign related tests
  
  • Management <- All Campaign Management related tests
    
    • Form <- All Campaign Create/Edit related tests
      
      • Create <- All Campaign Creation tests
        
      • Edit <- All Campaign Edit tests
        
      • Validate <- All Validation tests (successful and errors)
        
      • Misc
        
    • Inventory <- All Campaign Inventory related tests
      
    • Table <- All Campaign Management Table report tests
      
  • Reports <- Other campaign report tests
    
  • RSS <- RSS Inventories tests
    
  • SelfService <- Self-service Flow tests – approving and rejecting campaigns
    
• Editorial Tools
  
  • BlockContent
    
  • LookBackSettings
    
• …..
  

Each package has a base class that is extended by all classes in subpackages. These base classes provide all the necessary logic to generate entities related to this section.

In order to avoid super long waiting time for tests to conclude, a lot of thought was put into the test preparation stages (the @Before and @BeforeClass methods). Since similar tests are gathered in the same classes, a BeforeClass preparation method generates all needed entities that can be reused. While a Before method takes care of the buildup that can’t be shared (logging in, reaching the page, etc.). To facilitate easy entity generation we’ve created a utility class, DatabaseUtils, which is used for all preparation steps.

Tips for Writing Selenium Tests

After a lot of pain and gain, I’d like to share some important points to be aware of when writing Selenium tests.

• Copy/Paste is your enemy. Since many tests can repeat the same flows with small changes, identify all similar flows and bundle them into utility functions. Test code can become very large very fast, and if you’re not paying attention maintainability can become an impossible task.
  
• Don’t use the same entities and data for different tests, if they are being modified during tests, tests will run in parallel and may break each other’s data. Only generate entities that will remain immutable in BeforeClass methods.
  
• You don’t need to test everything. For component or small page element behavior, you can use Karma tests. Selenium tests are slow – use them for user flows and not for every component aspect.
  
• That being said – assert everything possible. While interaction is slow, assertion is fast. Assert existence of elements, their visibility, their states – whatever comes to mind.
  
• Remember that tests take time to run because they emulate user behavior. Aspire to generate lean and specific flows with most of build up done to the database and not through the test itself.
  
• Don’t panic! It’s not specific for Selenium, I just think it’s a good tip for everything you do.
  

Summary

Selenium is a massive testing framework, and I have only touched the tip of the iceberg in this post. The most important thing to keep in mind when building these tests is: KISS. Keep It Simple, Slugger. You can start by building a partial cover of your web application only, no need to head on with a full blown solution. This will keep some of the bugs at bay, and that’s more than you had before.

I would love to hear what hurdles you’ve come across in your implementation, and of course about the successes as well.

Do you already implement a Selenium testing framework? What are the lessons you’ve learned from your implementation?