protractor | gherkin | cucumber | automated acceptance test | automated testing | Technology
This is a follow up to my previous post Automated Testing and ATDD with Gherkin, Cucumber and Protractor: Getting Started where I discussed the path taken on my first project using BDD and how manual testing time and the # of bugs generated were both significantly reduced. With this post we'll be setting up a project to run automated acceptance tests with Cucumber and Protractor using human readable acceptance criteria written in Gherkin. Our feature tests will be hitting the ToDo MVC AngularJS example site.
Installation of NodeJS and NPM is necessary. Quick setup here.
A few terms to get started
Gherkin: "Gherkin is the language that Cucumber understands. It is a Business Readable, Domain Specific Language that lets you describe software’s behaviour without detailing how that behaviour is implemented." - Gherkin Github page
Cucumber: Behavior Driven Development framework to connect Acceptance Criteria written in Gherkin to the code that actually exercises the application under test.
Protractor: Implementation of the Selenium API that manipulates the browser from code. Initially written to be used with AngularJS but can be used with non-Angular applications.
The automated test application
We'll start with a clean directory, something like atdd-todo-app
Inside atdd-todo-app, we create a package.json (how NPM is configured) with the following contents:
{% highlight json linenos%} { "name": "cucumber-protractor-example", "version": "0.0.1", "description": "Jumping off point for CucumberJS and Protractor projects", "scripts": { "install": "node node_modules/protractor/bin/webdriver-manager update", "start" : "protractor protractorConfig.js" }, "devDependencies": { "chai": "3.5.0", "chai-as-promised": "6.0.0", "cucumber": "1.3.1", "protractor": "5.1.0", "protractor-cucumber-framework": "1.0.1" } } {% endhighlight %}
At this point we run npm install to gather all the necessary libraries.
The next step is configuring Protractor. This is a file that configures protractor and tells it where to find your tests, how to run them, what site to run them against, and what browser to run them in. More advanced options would be running multiple browsers simultaneously, telling Chrome to run in 'headless' mode, or setting up a remote server like SauceLabs or BrowserStack. For our purposes, we're going to just use Chrome and tell Protractor we want to use the Cucumber.js framework.
Let's create our protractorConfig.js file with the following contents:{% highlight javascript linenos%} 'use strict';
exports.config = { directConnect: true, capabilities: { browserName: 'chrome' }, framework: 'custom', frameworkPath: require.resolve('protractor-cucumber-framework'), specs: ['features//*.feature'], cucumberOpts: { require: ['features//step_definitions/**/*Steps.js'] }, baseUrl = 'http://todomvc.com/' };
{% endhighlight %}
Explanation:
- Line 4:
directConnectcan be used to avoid the use of a Selenium server. For more information here is a good explanation on stackoverflow. Unfortunately it will NOT work with firefox given the current libraries. I find myself using it when I first get started with a project because the drivers are regularly updated to match the version of the browsers.directConnectallows me to not deal with that until I want to start using non-Chrome browsers. - Line 6: This is how we tell protractor what browser we'll be using.
- Line 8: In order to use Cucumber with Protractor we must tell protractor to use a 'custom' framework.
- Line 9:
frameworkPathis how we tell protractor what custom library to use. - Line 10:
specsis how we identify what files hold our tests. - Line 12:
cucumberOpts.requireidentifies what files contain the code that can/will be executed by Cucumber
Add a todo.feature Gherkin file in a features directory with the following contents:
{% highlight gherkin linenos%} Feature: Creating a new todo task As a person with multiple things to do I'd like a way to manage my tasks So that I see what I'm getting done
Scenario: Adding a new task
Given I have gone to the angular todo mvc page And I have entered "mow the lawn" into the todo entry box When I hit enter Then I should see "mow the lawn" in the todo list {% endhighlight %} Explanation:
- Line 1: Gherkin files must have one and only one
Feature:identifier and it must have a ":", followed by a name of the feature. - Line 2-4: Some description of the feature. Usually matching the User Story details.
- Line 6:
Scenario:must also be followed by a ":" in order for it to be properly formatted. - Line 7-10: Each step must begin with either Given, When, Then, And, or But. The content following this prefix is what is used to match against the defined steps using Cucumber.
- Find more details here.
Finally Create todoSteps.js in features/step_definitions/ directory using this Javascript:
{%highlight javascript linenos %} 'use strict';
var chai = require('chai'); var chaiAs Promised = require('chai-as-promised'); chai.use.(chaiAsPromised); var expect =chai.expect;
varfirstExampleSteps = function () { this.Given(/^I have gone to the angular todo mvc page/, function () { return browser.get('/examples/angularjs/'); });
this.Given(/^I have entered "([^"]*)" into the todo entry box$/, function (newTodoText) {
var newTodoInput = element(by.id('new-todo'));
return newTodoInput.sendKeys(newTodoText);
});
this.When(/^I hit enter$/, function () {
var input = element(by.id('new-todo'));
return input.sendKeys(protractor.Key.ENTER);
});
this.Then(/^I should see "([^"]*)" in the todo list$/, function (todoText, callback) {
var todoLabel = element(by.cssContainingText('li label', todoText));
expect(todoLabel.isDisplayed()).to.eventually.equal(true).and.notify(callback);
});
}; module.exports = firstExampleSteps; {% endhighlight %}
Explanation:
- Line 3-6: Pulling in the assertion libraries and setting
expectto be used for assertions (see line 25). - Line 9-11: Our first step definition. The
this.Given, function is the Cucumber function that takes a regular expression and a function. That function will be passed any captured variables (explained later in this post) and a callback function. This callback function is always the last argument. For example, if n is the number of captured arguments then the callback will be the n + 1 argument. We use the callback in the case of asynchronous code execution. If the asynchronous code returns a promise then that promise can be returned from the Cucumber function without calling the callback. Most Protractor functions return promises so we can usually just return those. I hope to go into more detail on the promises in a future post. Note:this.Given,this.When, etc... is all just syntactic sugar (represent the same underlying function) which is why that part of the Gherkin step is ignored. - Line 10: Using protractor's
browser.getfunction to navigate to the/examples/angularjs/route of our application. Notice that we're not calling a callback function. We just return the result of the call tobrowser.get('/examples/angularjs/'). - Line 13-16: Another step definition this time with a captured argument. That captured field
([^"]*)is passed in asnewTodoText. - Line 14: Using Protractor's
elementfunction to look up the target element (akaElementFinder) by the idnew-todo. Could have also donevar newTodoInput = $('#new-todo');. More on that later. - Line 15: ElementFinder objects have a number of functions and in this case we call
sendKeys(newTodoText). This actually inputs the text into the input box represented by newTodoInput.
For more info see the Protractor API
At this time you should have the following directories and files in your project:
features/
step_definitions/
todoSteps.js
todo.feature
package.json
protractorConfig.js
Now we should be able to run our feature tests.
In a command line interface shell, cd to the project directory and run: protractor protractorConfig.js
I have prepared a video to support the [running & debugging of an application in IntelliJ IDEA/WebStorm] run_debug_video.
I haven't verified that Cucumber/Protractor combination runs in any IDE other than IntelliJ IDEA/WebStorm. Here is a link to other supported IDEs
The onPrepare() setup function
In the protractorConfig.js you can add an onPrepare() function to handle some set up. Quite handy to setup your configuration based on command line arguments or environment variables. I hope to explain more in a future post.
{% highlight javascript linenos %} cucumberOpts: { require: ['features//step_definitions//*Steps.js'], }, onPrepare: function() { browser.manage().window().maximize();
browser.baseUrl = 'http://todomvc.com/'; } {% endhighlight %}
- Line 5: Using Protractor to maximize the window size. We are not expected to set the window size but I wanted to provide an example. We can also call
setSize(width, height). Example:browser.manage().window().setSize(1200, 800);. This is a good idea if you have different window sizes you'd like to test. - Line 6: Setting the baseUrl so our tests know where to go.
How does the Cucumber know which step definition to run? Regular Expressions.
Considering the following snippet from protractorConfig.js: {% highlight javascript %} specs: ['features//*.feature'], cucumberOpts: { require: ['features//step_definitions/**/*Steps.js'] }, {% endhighlight %}
Each step in a .feature file needs a step definition (in this case a Javascript function) that matches it based on the regular expression (Ignoring the Given/When/Then/And/But prefixes in the Gherkin). Steps in any file identified by the specs attribute (feature files) will be tested against all step definitions in any of the files identified in the cucumberOpts.require array in the protractorConfig.js file. You will get a warning if a step matches multiple definitions.
Capturing Group
There will be times when you have steps that can be simplified down to one step using a regex to capture a variable.
Before
{% highlight gherkin %} #todo.feature
Then I should see mow the lawn in the todo list Then I should see clean the gutters in the todo list
{% endhighlight %}
{% highlight javascript %} //todoSteps.js this.Then(/^I should see mow the lawn in the todo list$/, function () { var todoLabel = element(by.cssContainingText('li label', 'mow the lawn')); return expect(todoLabel.isDisplayed()).to.eventually.equal(true); }); this.Then(/^I should see clean the gutters in the todo list$/, function () { var todoLabel = element(by.cssContainingText('li label', 'clean the gutters')); return expect(todoLabel.isDisplayed()).to.eventually.equal(true); });
{% endhighlight %}
The ([^"]*) can be added to capture some text that then gets passed into callback as todoText.
After
{% highlight gherkin %} #todo.feature Then I should see "mow the lawn" in the todo list Then I should see "clean the gutters" in the todo list {% endhighlight %}
{% highlight javascript %} //todoSteps.js this.Then(/^I should see "([^"]*)" in the todo list$/, function (todoText) { var todoLabel = element(by.cssContainingText('li label', todoText)); return expect(todoLabel.isDisplayed()).to.eventually.equal(true); });
{% endhighlight %}
Non-Capturing Group
Quite often we'll have Givens and Whens that do the same thing that really just have a different 'tense'.
By adding in the (?:have clicked|click) you can allow your step definitions to match more feature file steps without passing that value to the callback.
Before
{% highlight gherkin %} #some.feature Given I have clicked the button {% endhighlight %}
{% highlight gherkin %} #some.other.feature When I click the button {% endhighlight %}
{% highlight javascript %} //commonSteps.js this.Given(/^I have clicked the "([^"]*)" button$/, function (buttonText) { var buttonElement = element(by.buttonText(buttonText)); return buttonElement.click(); });
this.When(/^I click the "([^"]*)" button$/, function (buttonText) { var buttonElement = element(by.buttonText(buttonText)); return buttonElement.click(); });
{% endhighlight %}
After
{% highlight gherkin %} #some.feature Given I have clicked the button
{% endhighlight %}
{% highlight gherkin %} #some.other.feature When I click the button
{% endhighlight %}
{% highlight javascript %}
//commonSteps.js //You can use a non-capturing group by adding a question mark just after the first paren. this.Given(/^I (?:have clicked|click) the "([^"]*)" button$/, function (buttonText) {//notice there is only one parameter, buttonText var buttonElement = element(by.buttonText(buttonText)); return buttonElement.click(); }); //Again we see that the Given/When/Then/And/But prefix of the step is irrelevant (Syntactic sugar).
{% endhighlight %}
Protractor selectors
The element finding is done by selectors. There are some shortcuts: by.tagName, by.className, by.id but really we're just talking css selectors. The function by.cssContainingText('label', labelText) will come in handy. There's also by.xpath that can be pretty powerful but tough to maintain. XPath lookups are useful when looking for siblings, first-child, last-child, etc... Read more about different by options from Protractor.
{% highlight javascript %}
// A few ways to find the same ElementFinder object var newTodoInput = element(by.id('new-todo)); var newTodoInput = element(by.css('#new-todo)); var newTodoInput = $('#new-todo');//This is a super cool option {% endhighlight %}
Non-Angular applications
Set browser.ignoreSynchronization = true; in the onPrepare() function of protractorConfig.js. Further explanation.
Basic Terminology
GWT: "Given, When, Then" format for describing your Acceptance Tests.
Feature file: File laying out Acceptance Criteria for a particular feature, organized by Scenarios and Steps, and written in Gherkin.
Scenario: Collection of Steps that will describe some situation.
Step: Line in a feature file that will be matched to a Step Definition using regular expression matching. One of Given, When, Then, And, and But.
Step Definition: Cucumber function that performs some action when a Step is hit with a matching regular expression.
Promises: A Javascript concept that allows for asynchronous code to be executed without blocking your application. That you will need to understand.
Conclusion
I hope this helps get you started. There is a lot you can do with Gherkin/Cucumber/Protractor and this post just barely scratches the surface. I hope to see a smaller time gap between this post and the next. Also, please be aware that the Gherkin provided does not represent the best-practices for BDD. If you want to have questions or would like to discuss this or any other post, please hit me up on Twitter.
