Sub-second Acceptance Testing is an interesting test paradigm that we have adopted for our DevOps pipeline.

It challanges the concept of the test-pyramid, and the view that unit tests should be written with extremely high test coverage, with a lesser number of integration tests, and an even smaller number of e2e tests. Aslak Hellesøy says this has come more from the understanding the e2e tests are slower than integration which are slower than unit. So, to keep testing times useful the pyramid is needed.

There is another point-of-view that many unit tests can be viewed as meaningless boilerplate. If you are mocking a database, and writing unit tests for a CRUD repository fronting that database, what are you really gaining?

Finally, in DevOps, the true thing that matters is that you are delivering value to the customer. Tests should be focussed on customer value. OK, we often have more than one customer - internal users, support teams, etc. So, BDD is a reasonable way of defining tests that are customer-facing.

Sub-second BDD takes the view that the BDD tests should be run as acceptance tests in all cases. If I am a frontend developer I should run the BDD tests before committing. BUT, it can be horrifying to have the full stack constantly deployed and updated to my local machine, and those tests can be SLOW as. Likewise for the backend guy. And, yuck, do I really want to contaminate my machine with Node?!?

The take-away we took from Aslak is that we would run the same BDD tests in three scenarios:

• Backend, with a mocked database. For the backend devs.
• Frontend, without a browser, headless or otherwise, and mocked services. For the frontend devs.
• Fullstack, using headless Chrome, databases, the full full-stack. And this is used in the delivery pipeline gate before the production deploy.

The interesting thing is that using tests as acceptance tests in this way makes you think about the language of the tests. Instead of saying “When the user clicks on button 4”, we say “When the user selects race 4”. This style is adopted from screenplay testing

In the following diagram, only the green boxes are written for testing. All the other boxes are part of the normal application stack. And, the tests themselves are common.

We use CucumberJS for the frontend and full-stack tests, and godog for the backend as the test drivers. Then Nightwatch API as the web-driver layer in the full stack tests driving headless Chrome.

We also did some stub work in go-micro for the backend tests so that all of the micro-services domain logic are loaded into a single executable and are configured to talk directly with each other rather than over network connections.

Future topics in this space:

• Fast backend BDD tests using mocks and stubs
• Cucumber-JS with Vue and avoiding a webpack prebuild stage
• E2E tests leveraging Kubernetes and gitlab-ci

Image by Free-Photos from Pixabay