Web automation

Save time and let the machine do what it is good at.

Why is it required?

In today's fast moving world, where there are more and more complex systems built with frequent requirement changes and continuous delivery, it becomes difficult to maintain and improve the quality and efficiency of the software product without automated tests. Especially with digital products, there are many browser and device constraints you need to consider. We must continuously measure quality and test the product under all these constraints. Automated tests have the following benefits:

  • Reliable and accurate.
  • Reusability.
  • Quick feedback.
  • Easy maintenance.
  • Increased coverage.
  • Reduction of cost and time on actual testing.

Concept

Our current web automation framework uses Cucumber to define the test scenarios in BDD style specification ("Given, When and Then…"). Cucumber is the main tool which orchestrates the whole framework.

These test scenarios reflect exact actions that an user would perform on the web pages. When the code is deployed on the server, the tests run on a specified browser and environment. The framework integrates function libraries, test data sources, and various reusable modules which act as building blocks to execute test steps that reflect the business process. Tests fail when they do not meet the expected result.

Framework and utilities

Web automation framework includes:

  • BDD Test Specification — Cucumber (Gherkin), plain English sentences.
  • Main Programming Language — Ruby.
  • Web automation tool — Capybara, Ruby gem which simulates how a user would interact with a website.
  • Webdriver — Selenium webdriver for Capybara.
  • Headless gem — The headless gem is a ruby wrapper for Xvfb that makes it super easy to run graphical applications (such as real web browsers) on a headless machine.
  • Env Setup: — Vagrant and Virtual Box using Ubuntu VM.

Maintainability

Selenium acts on webelements with the help of their properties such ID, name, Xpath etc. We use the Page Object Model (POM) design pattern where every web page is a class, and actions performed on those pages are instance methods.

  • There is clean separation between test code and page specific code, such as locators.
  • An object can be accessed by one or more test scripts, hence POM helps us to create an object once and use it multiple times.
  • It is easy to access and update when there are UI code changes.

Setup and teardown

Every test sets up its own set of data, rather than seeding data upfront. This is to avoid any leaky scenarios and dependencies from one test to another. Tests should be independent and self-contained.

Setup and teardown are usually done through built-in hooks in Cucumber like Before, Tagged or At_exit hooks.

Continuous integration

Every test pack contains a config.yaml sitting in the root folder of the project which is used to list specific details of various environments, i.e., server_urls, database IPs etc. Here's an example:

defaults: &defaults_config
default_config: testing

demo:
<<: *defaults_config
url: https://example.demo.url.net

testing:
<<: *defaults_config
url: https://example.testing.url.net

This is used from Jenkins jobs to run tests on various environments. We use these environments on almost every project.

  • Testing — internally-facing testing environment. Code is deployed from existing CI branch which is mostly develop.
  • Demo/staging/UAT — production-like externally-facing environment used for client demos. Code is deployed from existing release branch for a current release.
  • Production — deployed to from the master branch.

Tests are run with an ENV_CONFIG command line parameter which dictates the test run environment. For example:

$ cucumber -f pretty -f html -o <path>/name-of-test-report.html -f junit -o ./	junit_reports ENV_CONFIG='testing' Driver='chrome'

The Driver parameter describes which browser should run the tests (or headless for a headless instance).

Every test run publishes Junit and Cucumber reports.

We set up CI pipeline as below and all test runs from Jenkins jobs are configured in similar fashion.

  1. Build off develop branch
  2. Deploy to testing environment
  3. Run tests against testing environment
  4. Merge to release branch and build
  5. Deploy to demo environment
  6. Run tests against demo environment