Drupal Testing Strategies

Creating a Drupal site can be a complex process. Some people put together Drupal sites using a collection of different modules whilst others use Drupal as a framework and build the site using code.

No matter what sort of Drupal site you have, you'll need to have some testing in place to make sure that it works correctly. This is especially important when applying updates to your sites as the updated code can create unwanted side effects (or contain bugs) that might cause the site to malfunction. Generally, Drupal modules are very good with updates, but there are sometimes interactions between modules can cause problems that weren't taken into account.

When it comes to testing, there are a number of different options available to a Drupal developer. Each of which have their own uses and depend on the type of Drupal site you are trying to test.

In this article, I will go through a number of testing strategies that you can use to test a Drupal site. I won't delve too deep into each topic as this is more of an overview of the different technologies and strategies available.

Static Analysis

If you are writing custom code for your Drupal site then the first stage of any Drupal testing solution should be inspecting the codebase itself. This means looking at the code written and ensuring that it meets with certain coding standards and best practices.

Static analysis should only be done on custom code you write for a Drupal site. Due to the Drupal coding standards Drupal core itself and contributed modules tend to be well written.

When writing modules that intended to be contributed modules it is very important to adhere to the same coding standards. 

PHP Codesniffer

PHP Codesniffer (or phpcs) is a tool that will scan PHP files to look for patterns of code called "sniffs". Drupal has a couple of standards that check for different things within your codebase.

• Drupal - This is the Drupal coding standards and includes rules like "open braces will be on the same line as the function declaration".
• DrupalPractice - In addition to the core coding standards there are a list of best practices that a Drupal site should follow. This includes things like using dependency injection or passing arguments to the translation functions.

There are some good instructions about how to install phpcs onto your system on the drupal.org website.

It is usually a good idea to run both of these standards at the same time as this will flag everything.

The following command will run phpcs against the custom modules directory using the Drupal and DrupalPractice standards. This includes instructions to add some of the custom file extensions that Drupal has, and to also ignore any vendor directories.

phpcs --standard=Drupal,DrupalPractice --extensions=php,module,inc,install,test,profile,theme,css,info,txt,md,yml --ignore=node_modules,bower_components,vendor web/modules/custom

This will produce no output if your code is fine, but will produce output showing what is wrong with the code.

Note that quite a few of the Drupal coding standards issues can be solved by using automated tools like PHP Code Beautifier and Fixer. Many coding standards problems will be due to whitespace, so getting something to automatically fix that means you can concentrate on the more important standards issues.

Whilst phpcs is capable of inspecting CSS files for coding standards issues you can also use CSScomb to format and order your CSS files if you want more control over how your CSS if formatted or sorted. There's a page on the drupal.org documentation site detailing how to do use CSScomb if you are interested.

The phpcs tool is also able to inspect (and understand) yml files and so can inform about problems in things like routes.

PHPStan And DrupalStan

PHPStan (or PHP Static Analysis) is similar to phpcs but will look beyond the scope of the single file. This means you can get a good picture of how your code will act when run.

PHPStan is a great tool, but because it relies on knowing things about the application involved it has a few issues when inspecting Drupal. That's where DrupalStan comes in.

DrupalStan is an extension of PHPStan and informs the tool about how Drupal is structured so that it can make a more informed analysis of the code. It takes into account how Drupal registers services and plugins and so won't flag unnecessary things.

One note of warning when you start using PHPStan is not to be too disheartened by the amount of output the tool generates. The tool consists of a number of levels (0 - 9) and it is quite difficult to create code that will pass a level 9 analysis. By default, PHPStan will use level 0 (the lowest level) and even at this level it will generate a number of issues for apparently correctly put together code. You should look at addressing those issues before moving onto level 1.

Instead of blindly installing this and starting to fix issues you should agree with your team about what level would be appropriate for your project.

What DrupalStan is also excellent for is deprecation testing your code against different versions of Drupal. It does take a little bit of setting up as you need to install the phpstan/phpstan-deprecation-rules package and add some configuration files. Once done though you can easily get a report on any code you need to change to allow it to work with future versions of the system.

ESLint

One thing we haven't looked at yet is JavaScript and we can use the ESLint tool to inspect JavaScript code for coding standards and best practice as well. Drupal uses the Airbnb JavaScript Style Guide and has lots of best practice approaches to writing JavaScript in a maintainable way.

If you have written any JavaScript for your Drupal module then it's a good idea to run it through ESLint to make sure that you are adhering to the correct Drupal standards.

You can look at the drupal.org documentation page for more information about how to install ESLint and tweak the settings to suit Drupal.

External Package Testing

Rather than embed lots of code within Drupal you should be considering creating external packages that you can then add as dependencies of your custom modules. By creating an external package you can not only ensure that it can be re-used on other (non-Drupal) projects but you can write unit tests for the package before it even reaches Drupal. 

A good example of this approach is with the addressing package, written by the Commerce Guys. When writing the new version of the commerce module for Drupal 8, Commerce Guys abstracted a lot of the address code they wrote for Drupal 7 into a separate package. This package was then used as the central address code for the Drupal Address module, which is pretty much the standard way to add address functionality to a Drupal site.

The addressing package contains extensive unit tests for how to create and manipulate address information and this meant that the Drupal module then didn't have to test the underlying addressing code. They could just add tests for the Drupal address fields and assume that he underlying code was tested (which it was).

So, the next time you are integrating with an API, or writing lots of code to perform a specific task, consider splitting that code out into a separate package and writing unit tests for it. I personally find writing unit tests for single packages easier than writing them for Drupal sites. It's also easier to integrate code metrics like percent coverage with a separate package.

Drupal Testing

Finally, we get into testing Drupal from within.

This is done using PHPUnit and Drupal has really good support for tests with three different levels of testing functionality. These levels are unit, kernel and functional testing, and you make use of these levels depending on what sort of test you want to run.

Before you can run your unit tests you need to install the "drupal/core-dev" composer library, preferably as a dev-dependency. This package is just a composer.json file that includes a number of other dependencies that are required to run unit tests within Drupal.

With that in place you'll need to copy the core phpunit.xml file from core/phpunit.xml.dist into your project root. Once there you'll need to change some of the settings to point to your development site.

There are extensive instructions on how to get Drupal set up correctly so that you can run your PHPUnit tests, so I won't dive into too much detail here.

./vendor/bin/phpunit web/modules/custom

There is plenty of documentation on drupal.org about what tests are and how to run them, so I will just concentrate in this section on what to test.

In order to write Drupal tests you need to have structured your code according to the Drupal coding standards. Without using dependency injection or well structured code you will find writing tests a difficult thing to do. The tests will error in all kinds of interesting ways unless you have set things up correctly.

Let's have a look at the different levels of tests you can get Drupal to run.

Unit Testing

Unit tests are the smallest thing you can test in an application, hence the name "unit". At this level of testing Drupal is not bootstrapped and so you do not have access to the database or any services. In fact, if you do need to create service classes you must first mock all of their dependencies manually.

Mocking is a way of creating objects without creating them. It's a really useful technique that allows you to generate an object and all of it's dependencies without having to actually create the object. You can then expose all or part of the original class to your unit testing framework and use it like you normally would.

In Drupal, mocking is accomplished by using the Prophecy package and is injected into Drupal using the Prophecy PHPUnit package.

These tests extend the class Drupal\Tests\UnitTestCase and should live in a "test/src/Unit" directory within your module.

Use this type of testing when:

• Testing to see if strings are treated correctly.
• Testing arithmetic of any complex (or not!) calculation.
• Testing any object manipulation.
• Testing file generation code, especially non-binary files.

Basically, test anything that won't touch Drupal's internal API's (unless you are prepared to mock them). Unit tests also take very little time to run as Drupal isn't bootstrapped at all.

Kernel Testing

Kernel testing is a level higher than Unit testing, and at this level we have access to things like the database and some internal services. You will probably need to add some code to get Drupal up to a level that you can test with, but this is more useful than unit testing if you want to test how things work within Drupal.

These tests extend the class Drupal\KernelTests\KernelTestBase and should live in a "test/src/Kernel" directory within your module.

Use this type of testing when:

• Testing that custom entities work as intended.
• Testing custom field handlers.
• Testing your custom migration configuration and classes.

Basically, anything low level enough that unit tests don't have access to, but where a full page request is not required.

Functional testing

Functional testing is where you have a fully installed version of Drupal to use a testing environment. This essentially means that your tests should revolve around opening pages and clicking buttons. Using the site as any normal user would.

These tests extend the class Drupal\Tests\BrowserTestBase and should live in a "test/src/Functional" directory within your module.

By default, the functional tests will run through curl requests, and so won't understand JavaScript elements on the page. If the page you want to test contains (and requires) JavaScript then you need to instead extend the class Drupal\FunctionalJavascriptTests\WebDriverTestBase. It is also best practice to put these tests in a "test/src/FunctionalJavaScript" directory within your module. This does require that you test with a JavaScript enabled browser, so you need to have set that up in your configuration settings before running the tests.

Use this type of testing when:

• Testing access permissions to routes.
• Testing forms, especially forms that have ajax functionality.
• Testing the outcome of hooks added to a Drupal site.
• Testing that configuration forms save their configuration correctly.

Basically, you can test anything that a user can do on the site through the interfaces that the module creates.

For each test, Drupal is installed and then uninstalled once the test is completed. For this reason the tests take a little longer than other types of tests, but they are essential when testing how your users interact with the site.

Testing Modules

One final note in terms of testing Drupal is creating testing modules that will only be used for the duration of your tests. This helps you test things like custom plugin or entity types without having to create 'dummy' versions of them in your codebase.

All you need to do is add them to your tests module dependencies so that they are installed as the tests are run. You then have access to ensure that the custom plugin type is working or that your events are triggering correctly.

If you do add testing modules to your module then they should exist in the 'test' directory of your modules, preferably within a 'modules' directory. You should also add "hidden: true" to the test module *.info file to prevent it from being installed as a normal module.

External Testing

We have looked at testing Drupal from within, but if you are creating a Drupal site you will probably want to test the entire site as a single application. This is where external testing comes into play and there are actually a number of different libraries available.

Behat

Behat is a PHP package that allows an "outside in" approach to testing. You essentially spin up a browser and interact with your Drupal site as a normal user would. This approach contrasts to unit testing in that you are testing how the site works as a whole, rather than small parts of a bigger picture.

Behat uses a package called Mink, which is essentially a wrapper around any browser you want to plug into the system. This means that you can support simple curl based tests or spin up a browser and use Chrome to test the site.

Tests in Behat are written in a syntax called Gherkin, which is intended to be written in sentence structure.

  Scenario: Load the homepage
    Given I am on "/"
    Then the response status code should be 200

You can pair Behat with the Drupal extension package that adds a number of Drupal specific functions and features to Behat. This allows you to do things like generate users with specific roles or create pages of content that you can then use as part of your site testing. Critically, the extension will keep track of everything that it created during the testing process and can delete them automatically afterwards. This prevents your testing environment from being bloated by mess from past test runs.

Behat is great for testing access permissions, especially with complex site setups that contain lots of user roles and content types.

This is a powerful tool, but I would advise you to use an IDE that understands how to get from Gherkin to the PHP code behind that rule. It makes life a lot easier when you can dig into the Gherkin when learning how to write tests to see how those tests interact with your site.

Nightwatch / Cypress

Nightwatch and Cypress are end-to-end testing frameworks written in JavaScript. They are quite similar in terms of functionality, but differ in some aspects.

The main difference is that Cypress can intelligently wait for the page to finish loading before running tests. This is contrast to Nightwatch, which needs extra code to wait for a certain amount of time or for elements to be shown on the screen before it can run tests.

In my experience, Nightwatch can be a little fiddly to get working correctly. Especially if you are attempting to test a site that has just has the caches cleared. The initial delay in the caches being rebuilt or other random variations in the page speeds can cause tests to fail and it takes time to "dial in" how much delay is required to allow tests to pass correctly.

Integrating your tests with Drupal is also important as it allows you to create a known environment for the tests to run with. There are a couple of ways to do this, but perhaps the most reliable way I have found is to issue Drush commands to the site. You can then just generate users or call a custom Drush command to setup the needed test environment for the tests to run.

Drupal has some internal unit tests written in Nightwatch, and also has a page on how to get Nightwatch tests running for your module.

Backstop

Backstop is a JavaScript tool that provides visual regression testing and I still class it as an external test since it doesn't have any Drupal integration. The tool allows you to see the before and after effects of styling changes made to the site. You need to run the tool once at the start in order to have a "base" that you can then use as a comparison to your changes.

Visual regression testing can be a great thing to add to your site. It will help reassure you when you make changes to the styles of your site. It can even be used against the live site and is a great way of checking important pages.

The downside is that it can create a lot of false positives. If you make lots of styling changes then Backstop will flag a lot of the site as having failed tests.

Load Testing

Finally, there is load testing, where you deliberately throw lots of traffic at a site to gauge how much traffic your site can handle.

In terms of load testing Drupal, remember that there's a big difference between anonymous and authenticated traffic. Part of your load testing should be logging into the site and navigating around some pages.

Also remember that Drupal's cache system is heavily reliant on users interacting with the site. For this reason you should probably run a simple script to "warm" the caches of the pages you intend to test before actually throwing traffic at them. Without this step in place you'll find an obvious delay in your first block of traffic before the caches are created and the site can start responding normally.

There are a number of local tools like JMeter that provide this sort of functionality. You can add a user journey to JMeter so that your user will log in and navigate around the site a little. Because they are local tools, however, they have limited effectiveness.

Services like loader.io or BlazeMeter can be used to throw a lot of traffic at your site at the same time and are great for stress testing your site. They can even accept JMeter scripts, which means you can perfect your user journey locally and then upload it to the site.

One word of warning is that if you have a CDN in front of your site you should probably avoid running load tests as they will might annoy the CDN or just trigger DDOS protection systems.

Conclusion

How you test will largely depend on what your requirements are. If you are running a simple site without any custom code then you'll probably want to use external testing only. If you have a lot of custom code then you should have static analysis and unit testing strategies in place.

If you have an existing site with no testing at all (which often happens in my experience) then you should write tests for anything you change or fix on the site. Adding tests to an untested site can be a bit of a daunting task, but a site with one test has 100% more tests than a site with zero tests. Don't try to add lots of unit tests after the fact unless you have a large budget and the willingness to do so.

As you add more tests to your site you should look at combining this with tools like Jenkins and Sonar to create test coverage reports. This can then build up a picture of your test coverage, which will show improvement over time. 

Fundamentally, if you set up any test you should automate the running of that test. There are lots of options available to do this, but it's important that all tests should have been run before the new code reaches the production environment.

I have lightly touched upon several topics here, but if you want me to dive into anything more than leave a comment and let me know. Also, you want to know about some mistakes to avoid when writing tests then I have previously written an article about 7 mistakes to avoid when writing tests.