Using Gherkin To Write Accessibility Tests

Accessibility acceptance tests can come in many different forms. Some are technical and require the tester to look within the source code itself (traditionally known as white box testing). Others are purely related to how a user interacts with the product (traditionally known as black box testing).

While many automated accessibility tools exist for white box testing, finding efficient ways to do black box accessibility testing is often harder to achieve.

Behaviour Driven Development (BDD) is a software development methodology already used by lots of Agile teams. It allows both technical and business stakeholders to easily understand product requirements. Gherkin is a business-readable, domain-specific language that lets you describe software’s behaviour without detailing how that behaviour is implemented. It is a popular choice for defining user acceptance test scenarios, and integrates easily with many automation frameworks including Ruby/Cucumber, PHP/Behat and .NET/SpecFlow (amongst others).

This blog briefly examines the benefits of using Gherkin syntax as a way of writing accessibility tests, and how and when to best do that.

Prior knowledge

This blog isn’t going to explain what Gherkin or BDD is, as there are many resources online which can give you a basic understanding of them. Some of my favourites are:

• Github Cucumber Wiki
• Cucumber.io reference document
• Automation Panda Guide To BDD

Practical guidance

When is Gherkin an appropriate tool?

As a rule of thumb, Gherkin is most appropriate for scenarios which relate to user behaviour, and so don’t require someone to look at the actual code behind the scenes.

Here are some examples where it would be appropriate and easy to use Gherkin for your accessibility scenarios:

Scenario: Skip to content link
Given I am on the homepage
When I select the ‘Skip to content’ link
Then the main content has keyboard focus

Scenario: Video captions
Given I am on the video page
When I select the ‘play’ button
And I turn on captions
Then the correct closed captions display throughout the video

If you do choose to use this approach, there are several benefits for the team:

1. It creates a uniform approach to writing acceptance tests – your accessibility scenarios are written in exactly the same way as all of your other functional scenarios.
2. If you already have an automation framework or a manual test case suite in place, Gherkin scenarios are easily integrated into your current framework and approach. This brings with it the possibility that your accessibility tests could form part of your regression test suite, and test runs can be recorded in your test case management system.

When is Gherkin not an appropriate tool?

However, as I have previously mentioned, not all accessibility tests are suitable for Gherkin, notably the more technical ones. Here are some examples of when Gherkin may not be an appropriate tool to use:

Scenario: Anchor tags
Given I am on the home page
When I inspect the source code
Then each anchor tag has the href attribute

Scenario Outline: ARIA landmark role
Given I am on the [type] page
When I inspect the source code
Then the main content has the landmark=main ARIA markup
Examples: of pages
| type        |
| sign in   |
| account |

Scenarios written using Gherkin describe user behaviour. The previous example is not describing user behaviour, therefore Gherkin is not an appropriate tool in this specific instance.

Another important thing to consider, is that there are numerous automation tools, libraries and APIs already available (including aXe, Pa11y, Google Accessibility Tools) which can be used to check whether code has been marked up to meet accessibility standards.

In the teams I’ve worked with, we’ve often used Gherkin for the tests which aren’t already covered by such tools, and utilise the automated tools to cover the tests which are easily automated.

Conclusion

If your team is already using Gherkin as a tool to write your user acceptance tests, then there’s absolutely no reason why you can’t use it to write your accessibility acceptance tests as well.

However, if you aren’t using Gherkin already, and are using a different tool which works for your team, don’t feel like Gherkin is your only option. The key thing to remember is that when you write your accessibility acceptance tests, try and use the same approach and tools that you are already using within the team. Accessibility tests are no less important than any other type of test.

What do you think?

We hope these insights are useful to you, and would love to hear your experiences around implementing accessibility tests in Gherkin. Please share your comments below.

Want more?

If this blog has been useful, you might like to sign-up for the Hassell Inclusion newsletter to get more insights like this in your email every month.