Do you feel confident your team is shipping code that won’t break production?
Building a product is tough: ensuring the promised value of your product matches the value that is delivered can be difficult. Deploying possibly untested code to your application has the potential to break features in unanticipated, and often unseen, ways. Fortunately, your teams can combat this cycle by implementing different forms of manual and automated testing.
At Dyspatch, we use a testing tool called Cypress to validate the quality of our software code, ensuring the product experience is maintained while we continuously ship and deploy code to production daily.
The Value of Code and UI Testing
Testing a product and putting it through its paces is incredibly valuable to both the team and the organization. It provides reassurance that the developers you trust to build the software aren’t delivering breaking changes in ways that can impact your customers.
Testing your code:
- Saves time in the long run, for you, your team, and the company
- Is industry best-practice — the majority of professional developers test their code
- Reduces the potential side effects a new feature can have on the rest of your codebase
- Mitigates regressions between deployments of your software and products
- Is, most of all, fun!
As a developer, testing the code you write is strongly encouraged, and making sure you don’t break said code when you deliver updates in the future is paramount. Different forms of testing add various levels of both cost and value to a company, and it’s always wise to choose the level that suits your team and the needs of the product. Unit testing is the most basic form and covers the very fine-grained bits of your codebase. Integration tests combine pieces of your application and test them in slightly larger chunks. End-to-end tests are added as a final step to cap off the entire test suite and add value by testing the application from front to back, testing the whole picture.
Manual user interface (UI) testing can be arduous and cumbersome, however, and very expensive in terms of financial cost and people hours. For these reasons, many teams don’t see the need to have many end-to-end tests, if any at all, in their development cycle. Historically, UI testing has been difficult to automate consistently and accurately and is very slow in comparison to other forms of testing, such as unit and integration tests. But in the last 10 years or so, tools have emerged from the open source community to help alleviate this pain, and the rise of web application languages has made it even easier to get up and running with reliable, automated end-to-end (E2E) tests.
There are many tools out there, but none with the representation in the testing community like Selenium. Selenium is the de facto tool for automated UI testing, has been around for over 10 years, and has an impressively large community surrounding it. You can automate different UI tests, such as form submissions and ensuring interactive elements on a web page end up in the correct state. The problem with automation tools like this, however, is that they don’t have any knowledge of the state of your application while running in a web browser. That said, there are hacks to get this functionality, at least partially, but they’re not very user-friendly.
The Shining Beacon Of Hope
Selenium is beginning to show its age in 2018, and anyone in the web development community who’s used it for a period of time will tell you it’s hard to learn to love it. A good reason it’s falling out of favour with developers new to testing web applications is that it isn’t as web-friendly as more modern testing utilities. There are newer and more directly-scoped unit and integration testing tools that have improved in the areas where Javascript developers tend to spend their time. This is exactly where Cypress starts to fill the gap for E2E testing; it’s a modern tool, written for web developers by web developers.
There are a few key areas that differentiate Cypress from Selenium, one of which is the fact that Cypress is built to be integrated into the development cycle while Selenium is a standalone testing tool. Another is that Cypress tests are written in web-native languages (anything that can transpile to Javascript) and the test runner executes your tests directly inside the browser, not from the outside using a remote protocol. This also gives your tests the ability to have automatic DOM-element retries while your tests run and there is no need to add explicit waiting or timeouts for elements to be visible in the browser window.
Because the actual test code is being executed inside of a browser, there no object serialization and no remote protocol to generate test flakiness — you have access to everything you would find in a web application environment. What this means for web application developers on your team is that they can participate in writing the Cypress tests in a familiar language, just like they would when writing unit and integration tests for their own code.
QA and UI testing at Dyspatch
At Dyspatch there is a dedicated QA Developer who maintains and iterates on the suite of Cypress UI tests and the infrastructure surrounding it. This leaves room for the developers actually building the application code to care about their own unit and integration tests, but also to write ‘happy path’ UI tests into the suite as well. Because Cypress UI tests live alongside the code they’re testing, it becomes very straightforward for developers to maintain those tests while also feeling confident updating features in the code.
Here’s an example of Cypress running a very simple test, with a setup and a few assertions. It’s that simple!
If you have a Node.js environment in your project, with npm available, you can install Cypress to your dev-dependencies by going to your terminal and typing npm install -d cypress. After installation, opening the Cypress test runner can be invoked with ./node_modules/.bin/cypress open, again from your terminal.
Creating your first test spec is equally as simple as the two steps above. Go to your terminal again and type touch cypress/integration/first_test_spec.js, then navigate to the cypress/integration folder in your file explore or terminal. Open the first_test_spec.js file you just created in your text editor of choice.
With this newly created test spec file, you can run the simple test below as-is:
The test reads:
- Describe the behaviour of the test
- Describe what the test should be doing
- The test content:
- Visits a URL and searches for an element on the page containing the word ‘type,’ then clicks on it (this is likely a button)
- Asserts that the URL of the current page includes a predetermined URL segment
- Gets the element on the page with the class name ‘action-email’ (likely an input), types the value ‘fake@email.com‘ into the input field
- Asserts that the typed value has been bound to the input value attribute
Tests such as this are easy to write and reason about, and even though it may seem too simple compared to some unit and integration tests, that’s the point; E2E tests should never test too much and should have a fairly light footprint relative to your overall testing stack.
As of this writing, Dyspatch, our Enterprise email template creation product, has over 70 happy-path and UI tests automated and running against a production-like environment. The tests notify the development team and set off fire alarms (not literally) when they fail. This lets the rest of our organization know that we care about the quality of the application experience and that they can rest assured customers won’t be affected when we deploy new features. These tests execute within a heavily monitored, several-minute boundary, and when the average time starts to creep up, we investigate ways to improve the run-time, so as not to block application developers from deploying.
Adding Cypress to our continuous integration checks on code check-in’s has not only become an invaluable piece of tooling in our development team’s quiver, but it has also even prevented production-breaking bugs from bringing down our application. Because of this, we’re investing more time into finding ways we can introduce Cypress to other levels of testing in our application stack.
