A flexible and scalable container based Selenium Grid using with Protractor

I will bring a brief introduction about Zalenium and my experience with it, how I used with Protractor.

About Zalenium

Start a Selenium Grid in seconds, a grid that scales up and down dynamically with this solution based on docker-selenium to run your tests in Firefox and Chrome. If you need a different browser, Zalenium can redirect your tests to a cloud testing provider (Sauce Labs, BrowserStack, TestingBot).

Zalenium works out of the box in Docker and Kubernetes.

Zalenium works conceptually in a simple way:

1. A Selenium Hub starts and listens on port 4444.
2. One custom node for docker-selenium
3. If a cloud testing integration is enabled, a cloud proxy node to support a cloud provider (Sauce Labs, BrowserStack,TestingBot) will register to the grid.
4. A test request is received by the hub and the requested capabilities are verified against each one of the nodes.
5. If docker-selenium can fulfill the requested capabilities, a docker container is created on the run, and the test request is sent back to the hub while the new node registers.
6. The hub acknowledges the new node and routes the test request with to it.
7. The test is executed and the container is disposed after test completion.
8. If docker-selenium cannot fulfill the requested capabilities, it will processed by one of the enabled cloud testing platforms.

Why?

• Have a stable grid to run UI tests with Selenium
• Maintain it over time (keep up with new browser, Selenium and drivers versions)
• Provide capabilities to cover all browsers and platforms

Zalenium’s main goal is: to allow anyone to have a disposable and flexible Selenium Grid infrastructure.

Check the Documentation.

Set up

Prerequisites

• Docker version >= 1.11.1 (probably works with earlier versions, not tested yet).

I'm running the Zalenium with Protractor, if you don't know about it, check my post:

Automation with Protractor, PageObject and, Cucumber

How I used the Zalenium?

I used with docker-compose, but what is docker-compose?

Compose is a tool for defining and running multi-container Docker applications.

docker-compose.yml

# To run the compose
docker-compose up --force-recreate

# To stop the compose
docker-compose stop

Below I will explain some important points on this docker-compose:

The volumes:

volumes:          
  - /Users/tluz/Desktop/videos:/home/seluser/videos          
  - /var/run/docker.sock:/var/run/docker.sock          
  - /Users/tluz/Documents/protractor-zalenium/.htpasswd:/home/seluser/.htpasswd

/Users/tluz/Desktop/videos:/home/seluser/videos

When the compose runs, I'm telling to share the folder on container /home/seluser/videos with my folder in /Users/tluz/Desktop/videos, then the videos generated in the container it will be copied to my folder.

/Users/tluz/Documents/protractor-zalenium/.htpasswd:/home/seluser/.htpasswd

Enabling basic auth

Providing a file with user(s) and password(s)To create a file with that information, please follow the steps for “Creating a Password File” described in the Nginx documentation. After that, map the created file to the container when you start Zalenium.

In this volume, I'm sharing the file .htpasswd to the container with my credentials to log in on the Zalenium stuff(Live and Dashboard).

The commands:

This it was the parameters that I used to my requirements.

command: >          
  start --desiredContainers 2                
        --maxDockerSeleniumContainers 8
        --screenWidth 1360 
        --screenHeight 1024                
        --timeZone "America/Sao_Paulo"
        --videoRecordingEnabled true
        --keepOnlyFailedTests true

desiredContainers: Number of nodes/containers created on startup.

maxDockerSeleniumContainers: Maximum number of docker-selenium containers running at the same time.

screenWidth: Sets the screen width. screenHeight: Sets the screen height.

timeZone: Sets the time zone in the containers.

videoRecordingEnabled: Sets if video is recorded in every test.

keepOnlyFailedTests: Keeps only failed tests on the dashboard (you need to send a cookie with the test result).

In my scenario, I just needed to record the failed tests, to avoid the over storage on the machine responsible to the Zalenium and I’m gonna explain my approach to use with Protractor.

If you need more configuration Parameters you can check on the Documentation about Zalenium.

My config file

protractor.conf.js

• Line 1 I’m setting the address from the container(default: 'http://localhost:4444/wd/hub'), but I set the basic auth and I needed to send the parameter of the user that I set on file .htpasswd, because of that my address has the parameter 'user2:1234@'(user:password).
• Lines 9 to 18 I'm using multiCapabilities, when the test will run it will run with the Firefox(line 11) and Chrome(line 15), the lines 12 and 16 I'm setting the name from the test for every browser to in the dashboard.

Keep Only Failed Tests TRUE

As I using this parameter I have to save the result of the tests in the browser cookies on the variable 'zaleniumTestPassed', let's take a look on the After(lines 37 to 39) in my test file.

spec.spec.js

Description of the parameter keepOnlyFailedTests: Keeps only failed tests on the dashboard (you need to send a cookie with the test result).

In the After (line 37 to 39) I implemented the line responsible to added on the browser cookies the variable zaleniumTestPassed with the status in the value, every time the test run the After it will be called, and if the value is false, the variable zaleniumTestPassed it will be added with the result.

Let's try

Starting Zalenium

Starting with Docker-compose run the command below:

docker-compose up --force-recreate

After the run the docker-compose you should see something like that:

Run tests in this repository:

tuliobluz/protractor-zalenium

In my project, I have three scenarios for my tests, two successful scenarios and one forced to fail.

When I run my tests, you can watch the tests running in real time:

http://localhost:4444/grid/admin/live

If you use the basic auth, you have to log in with the user that you setting up before.

When the running finished the terminal will show this log:

Chrome:

Firefox:

Like I was waiting just one scenario failed, and below has my dashboard with the scenario recorded in both browsers, let's check it out the dashboard.

Dashboard:

http://localhost:4444/grid/admin/live

1- List of the test were running.

2 - Test name

3 - Information about test(you can get the information about the build was running when the test broken and set in this field)

4 - You can check the logs about from Selenium and Browser Driver Log.

5 - Cleanup: The dashboard has a retention period(default 3 days), when you want to clean every test after 3 days just use the clean up. Reset: It will remove all information about the dashboard.

Conclusion

Zalenium bring a good feedback about my tests, I can watch the running in real time, besides the zalenium record the failed test(or every status from the tests). The dashboard gives me a way to check what is wrong in my test, as opposed to a running in headless.

Now you can check the best way to implement on your pipeline, and how you will use this tools to deliver more quality to your team.

Thank you guys!!!

Reference:

• Zalenium - A flexible and scalable Selenium Grid.
• zalando/zalenium

The problem:

I was suffering with a problem when I was running the tests in Protactor, in my structure containing pageobejct and cucumber. But the problem was that I using the tag “restartBrowserBetweenTests: true” on protractor.conf.js. When you had using this, the browser will be restarted every time that it will run and it lost the reference with test files, then it couldn’t interact with the page.

Just to remember you, if you use the tag “restartBrowserBetweenTests: true ”, this will cause your tests to slow down drastically.

The error:

org.openqa.selenium.NoSuchSessionException: Session ID is null. Using WebDriver after calling quit()?

The solution:

To solve this problem you have to do one function in an object to initialize the locators, that function must be called always you will run the test.

I created the module exports to object, this object has an one arrow function to initialize(lines 5~9) the locators to be used on the test.

Inside the Before(lines 8~10) I put the specPage.init(), this will call the initialize always the test will run.

That's works for me, thanks to read my post.

I wrote one post about how to run Protractor with PageObejct and Cucumber, if you need something about structure or some tips, just read.

Post: Automation with Protractor, PageObejct and Cucumber

My repository:

tuliobluz/protractor-cucumber-pageobject

I will show how to set up, which practice that I used to run protractor with the beautiful Page Object and Cucumber.

What is Protractor?

Protractor is an end-to-end test framework for Angular and AngularJS applications. Protractor runs tests against your application running in a real browser, interacting with it as a user would.

How does it work?

Protractor is an end-to-end test framework for AngularJS applications. Protractor is a Node.js program that supports the Jasmine and Mocha test frameworks.

Selenium is a browser automation framework. Selenium includes the Selenium Server, the WebDriver APIs, and the WebDriver browser drivers.

Protractor works in conjunction with Selenium to provide an automated test infrastructure that can simulate a user’s interaction with an Angular application running in a browser or mobile device.

Process Communication

The Protractor communication with Selenium Server it is by HTTP, the local server(selenium-server-standalone) or remotely(example: Sauces Labs) send the request by JsonWireProtocol to run interacting in the browser(Chorme, Firefox ou outros).

Why use Protractor?

• Protractor is built on top of WebDriverJS, which uses native events and browser-specific drivers to interact with your application as a user would;
• Protractor supports Angular-specific locator strategies, which allows you to test Angular-specific elements without any setup effort on your part;
• You no longer need to add waits and sleeps to your test. Protractor can automatically execute the next step in your test the moment the webpage finishes pending tasks, so you don’t have to worry about waiting for your test and webpage to sync.

Hands-on

Set up

• Prerequisite Node.js
• Install Protractor globally: npm install -g protractor
• Install the helper that download the browsers and instance of Selenium Server: webdriver-manager update
• Start the server: webdriver-manager start should be print this log.

Obs.: Remember start the server when you will run the tests.

Tip: Good practice to name files

I like to let the name files with one default in the end like .po to PageObjects and .spec to specifications. This lets more easy to identify my files.

Example:

login.po.js 
login.spec.js

Another example:

loginPage.js 
loginSpec.js

Choose one to use and enjoy. :)

Running the first test

I created the following structure of folders to the first test:

- tests
  |- e2e
    |- specs
- protractor.conf.js

• I created a file calls protractor.conf.js, this file will be responsible for my properties of the project.

tests/e2e/protractor.conf.js

exports.config = {
  framework: 'jasmine',
  seleniumAddress: 'http://localhost:4444/wd/hub',
  specs: ['./tests/e2e/specs/*.js']
}

Only this tags on my config file will invoke the default configuration to others tags.

• I created a file inside my folder specs with name spec.js

tests/e2e/specs/spec.js

1. describe('Protractor Demo App', function() {
2.   it('should have a title', function() {
3.     browser.get('http://juliemr.github.io/protractor-demo/');

4.     expect(browser.getTitle()).toEqual('Super Calculator');
5.   });
6. });

• Line 3 calling the method get(‘http://www.angularjs.org'), the browser will open with the link.
• Line 4 I’m waiting for the title of the page is (‘Super Calculator’), below is a screenshot about the page

• To run our test you will run the commands calling the config file(protractor.conf.js), remember to run on terminal inside the folder.

protractor protractor.conf.js

• Obs.: If your config file calls protractor.conf.js, you just need to run protractor in your terminal to run the tests.

Result:

1 tests, 1 assertion, 0 failures.

Can I use Protractor in application non angular?

When you put the tag below on my protractor.conf.js, I’m telling to run the tests on applications non-angular.

ignoreSynchronization: true,

BUT, using this property, my tests will become delayed, because of that, I must put the waits on my tests.

Tip: “If my application that it is Angular there is a part non-angular, what I need to do”?

Using this tag on my tests to turn on or turn off:

browser.waitForAngularEnabled(false);

Remember to use waits on your tests, because it will be delayed

Page Object

When writing end-to-end tests, a common pattern is to use Page Objects. Page Objects help you write cleaner tests by encapsulating information about the elements on your application page. A Page Object can be reused across multiple tests, and if the template of your application changes, you only need to update the Page Object.

• Create the following structure of folders to use PageObject:

- tests
  |- e2e
    |- pages
       |- specs.po.js
    |- specs
       |- spec.spec.js
- protractor.conf.js

Below it is the file structure to use PageObejct.

tests/e2e/pages/spec.po.js

Above it is the specification file to use the Page Object

tests/e2e/specs/spec.spec.js

• Line 1 spec.spec.js, instancing the PageObject that I will use in my tests.
• Line 4 spec.spec.js, always that test be will call it will initialize the PageObject.
• Line 6 spec.spec.js, the test calls the method of getting in PageObject in line 10 of spec.po.js, triggering get(‘http://www.angularjs.org'), the browser it will open in the link.
• Line 7 spec.spec.js, the test calls the method setName of PageObject spec.po.js to send one parameter a string(‘Julie’), line 14 of spec.po.js the field is filled with received string.
• Line 9 spec.spec.js, the test expects that the return text on the method specPage.getGreeintText() of spec.po.js, it be equal (‘Hello Julie!’).

Run the test again:

protractor protractor.conf.js

• Obs.: If your config file calls protractor.conf.js, you just need to run protractor in your terminal to run the tests.

Result:

1 spec, 0 failures

What is Cucumber?

Cucumber is a tool for running automated tests written in plain language. Because they’re written in plain language, they can be read by anyone on your team. Because they can be read by anyone, you can use them to help improve communication, collaboration and trust on your team.

Set up to Cucumber:

How I used a Custom Framework to Protractor, so I must use Chai library, to perform the expects in the tests:

To install:

npm install — save-dev cucumber

npm install — save-dev protractor-cucumber-framework

npm install chai

npm install chai-as-promised

• I created the following folder structure to cucumber:

- tests
  |- e2e
    |- features
       |- spec.feature
    |- pages
       |- specs.po.js
    |- specs
       |- spec.spec.js
- protractor.conf.js

Config file to Cucumber:

protractor.conf.js

• Lines 06 to 17, I put the preference about the browser, in my case I’m using the Chrome, but There are many options to use.
• Lines 42 and 43, It’s where I choose the framework that I using on my project, in this case, I’m using Cucumber, the new version of Protractor don’t have it, so I had to informed there is a custom framework if you want you can customize your profile.
• Lines 45 to 47, this properties is to set up the feature files.
• Line 54, I put this property with true, at the end of every test, the browser will be restart(It become your tests delayed, I used this tag because, in my scenario I needed, I recommend don’t use this.).
• Lines 56 to 64, I informed the path to specification files.
• Lines 93 to 99, onPrepare is the pre-condition that it will be running before the tests, the lines 96 to 99 I declared the cucumber global variables to use on the spec files.

I didn’t need changes on PageObject file.

tests/e2e/pages/spec.po.js

Using cucumber

• I created the feature file spec.feature.

tests/e2e/features/spec.feature

• I run the protractor: protractor protractor.conf.js

When I run the Protractor, it showed this log on the terminal.

Resultado:

tests/e2e/specs/spec.spec.js

• Then I needed to change the spec file, putting all functions inside the file spec.spec.js.

Example:

Given('The user go to {string}', function (string) {
           // Write code here that turns the phrase above into concrete actions
           return 'pending';
});

• So, the final spec file using the PageObject must be similar that:

• Line 1 spec.spec.js, I’m instancing the PageObject that I will use in my tests.
• Lines 3 to 6 instancing the library of chai, because I used this to assertions on tests.
• Lines 8 to 10 I put the Before, that’s mean every time the test will run the function init() will initialize the object of PageObject.
• Line 15 spec.spec.js, I called the function the get(string) of spec.po.js with the parameter a string (URL) received by spec.feature. Line 6 spec.po.js, triggering the function get(string), the browser is open with the link.
• Line 19 spec.spec.js, I called the function setName(string) of spec.po.js with the parameter a string(name) received by spec.feature.
• Line 23 to 24 spec.spec.js, I expect that function specPage.getGreeintText() it will return one string and that string it will equal (‘Hello Julie’).

Running the Protractor: protractor protractor.conf.js

• The test will fail because the string I expect it is “Hello Julie!”, but the getGreeting() returned “Hello Tulio!”, bellow there is an example like the protractor showing the fail.

To this test works I have to change the expect from(‘Hello Julie!’) to (‘Hello Tulio!’).

Thank you so much, guys!!! I hope that you enjoy.

Conclusion

I hope this guide increase your knowledge base, I tried to do something easy and dynamic. I let some tips and good practices in this guide.

I was suffering a problem when I was doing this project, that I’m telling in this post: Protractor — Session ID is null. Using WebDriver after calling quit()?

Feel free to read too.

My repository that I used on this guide.

tuliobluz/protractor-cucumber-pageobject

You can check another structure to use Protractor in this repository:

tuliobluz/protractor-supertest

Reference:

• Protractor - end-to-end testing for AngularJS
• Getting Started with Protractor and Cucumber
• BDD Testing & Collaboration Tools for Teams | Cucumber
• GitHub - cucumber/cucumber-js: Cucumber for JavaScript
• Chai