Drupal 8: Custom Cache Bins

Drupal's cache system is robust, feature full and extensible. It can be used to cache small blocks of rendered output or the results of more complex calculations. The cache can be stored in a for the page request, in a database or in a different cache system.

I recently needed to store a decent amount of data that was pulled from an API. Getting the data from the API took a few seconds so it was important to cache this within the site for quick retrieval. Even just storing the cache in the database was many times quicker than the API so it made sense to cache the results within the site. Having the user sat there waiting for the page to load every time they do anything isn't great.

To this end I created a small Drupal service that would manage this cache for me. Here is the record from the services YAML file within the module.

services:
  my_module.cache:
    class: Drupal\my_module\Cache\MyCache
    arguments: ['@cache.default']

The class itself was basically a set and a get method for the specific cache bin with some static caching being used to prevent the same database request being made over and over again. The caches were time based so that they would expire in 24 hours and also tagged in order to allow them to be specifically cleared at a later time.

<?php

namespace Drupal\my_module\Cache;

use Drupal\Core\Cache\Cache;
use Drupal\Core\Cache\CacheBackendInterface;

/**
 * Class MyCache.
 *
 * This class combines a static cache and an 'active' cache, loaded from a the
 * default Drupal cache location.
 *
 * @package Drupal\my_module\Cache
 */
class MyCache {

  /**
   * The default cache bin.
   *
   * @var \Drupal\Core\Cache\CacheBackendInterface
   */
  protected $cache;

  /**
   * MyCache constructor.
   *
   * @param \Drupal\Core\Cache\CacheBackendInterface $cache
   *   The default cache bin.
   */
  public function __construct(CacheBackendInterface $cache) {
    $this->cache = $cache;
  }

  /**
   * Sets a cache with a specific id, data and type.
   *
   * @param string $id
   *   The cache id.
   * @param mixed $data
   *   The data to be cached.
   * @param array $type
   *   The type of data being cached. This is used to set up the cache tags.
   */
  public function setCache($id, $data, $type): void {
    $cid = 'my_custom_cache:' . $type . ':' . $id;

    $tags = [
      'my_custom_cache:' . $type . ':' . $id,
      'my_custom_cache:' . $type,
      'my_custom_cache',
    ];

    $tags = Cache::mergeTags($tags, [$cid]);

    // Set the database cache.
    $this->cache->set($cid, $data, (new \DateTime('+1 day'))->getTimestamp(), $tags);

    // Set the static cache.
    $staticCache = &drupal_static(__FUNCTION__ . $cid, NULL);
    $staticCache = $data;
  }

  /**
   * Get a specific cache.
   *
   * @param string $id
   *   The cache ID.
   * @param string $type
   *   The cache type.
   *
   * @return mixed
   *   The cache or false if the cache was not found.
   */
  public function getCache($id, $type) {
    $cid = 'my_custom_cache:' . $type . ':' . $id;

    $staticCache = &drupal_static(__FUNCTION__ . $cid, NULL);

    if ($staticCache) {
      // If the static cache exists, then return it.
      return $staticCache;
    }

    // Get the cache out of the database and return the data component.
    $result = $this->cache->get($cid);
    return $result->data ?? NULL;
  }

}

The finished class in the project has a few methods for invalidating the cache when needed, which is why I'm adding in the cache tags. These are used to identify cache buckets that I needed to clear.

By default, this will add anything you cache to a cache_default table which contains a collection of other caches not connected to this data. In order to change this and use a different table we need to set up a custom cache bin. This is done using an entry in the services YAML file. This custom cache bin is then passed to the cache service set up earlier.

services:
  # Custom cache bin.
  cache.my_custom_cache:
    class: Drupal\Core\Cache\CacheBackendInterface
    tags:
      - { name: cache.bin }
    factory: cache_factory:get
    arguments: [my_custom_cache]
  # Cache service.
  my_module.cache:
    class: Drupal\my_module\Cache\MyCache
    arguments: ['@cache.my_custom_cache']

Now, because we have injected our custom cache bin into the MyCache class, any data we store in the cache is then stored in a table called cache_my_custom_cache.

What I needed to do was keep this cache entirely separate from the normal Drupal cache mechanisms so that if the caches were cleared then I didn't have to rebuild the caches again. This would allow things like the CSS/JavaScript cache to be flushed and not also flush our API cache. In order to do this I needed to allow a cache clear event to take place, without purging this custom cache from the system. This is possible, but a few things needed to be overridden in order for it to work correctly.

The cache_factory class being used to load the current cache backend will always load in the default Drupal cache systems, so we need to override that in the services to be a custom service.

cache.my_custom_cache:
  class: Drupal\Core\Cache\CacheBackendInterface
  tags:
    - { name: cache.bin }
  factory: my_module.cache_factory:get
  arguments: [my_custom_cache]

With that in place we need to set up the factory we just defined. This is first done by setting up another service called my_module.cache_factory.

# Custom cache factory, used to stipulate the loading of cache.backend.database.my_module.
my_module.cache_factory:
  class: Drupal\my_module\Cache\CustomCacheFactory
  arguments: ['@settings', '%cache_default_bin_backends%']
  calls:
    - [setContainer, ['@service_container']]

With the service set up we need to create a concrete cache factory class. This class is pretty simple, all we are doing is returning a service called cache.backend.database.my_module, which doesn't exist (yet)

<?php

namespace Drupal\my_module\Cache;

use Drupal\Core\Cache\CacheFactory;

class CustomCacheFactory extends CacheFactory {

  public function get($bin) {
    $service_name = 'cache.backend.database.my_module';
    return $this->container->get($service_name)->get($bin);
  }
}

To create this service we first need to define the service in the module services YAML file.

# Custom cache backend database factory, used to load in the CacheDatabaseBackend object.
cache.backend.database.my_module:
  class: Drupal\my_module\Cache\CacheDatabaseBackendFactory
  arguments: ['@database', '@cache_tags.invalidator.checksum', '@settings']

Next we define the CacheDatabaseBackendFactory class that we just defined in the services file above. This factory is used to return an object called CacheDatabaseBackend, which is a custom defined class. The core Drupal database caches also keep hold of a setting called DEFAULT_MAX_ROWS that is used to prevent the cache system becoming too large. I intend to increase this setting in the custom class, so I also included an override for the getMaxRowsForBin method that would allow this custom value to be used by Drupal's upstream garbage collection.

<?php

namespace Drupal\my_module\Cache;

use Drupal\Core\Cache\CacheDatabaseBackendFactory;
use Drupal\my_module\Cache\CacheDatabaseBackend;

class CacheDatabaseBackendFactory extends DatabaseBackendFactory {

  /**
   * {@inheritDoc}
   */
  public function get($bin) {
    $max_rows = $this->getMaxRowsForBin($bin);
    return new CacheDatabaseBackend($this->connection, $this->checksumProvider, $bin, $max_rows);
  }

  /**
   * {@inheritDoc}
   */
  protected function getMaxRowsForBin($bin) {
    return CacheDatabaseBackend::DEFAULT_MAX_ROWS;
  }
}

Now we can finally create out custom cache backend and override the base cache clearing mechanism. This class extends the DatabaseBackend class but allows us to specially override the cache clearing mechanisms. Essentially, we set the deleteAll() method to do nothing, whilst also adding our own deleteCustomCaches() method that will delete anything in the cache using the same overridden deleteAll() method from the parent class.

<?php

namespace Drupal\my_module\Cache;

use Drupal\Core\Cache\DatabaseBackend;

class CacheDatabaseBackend extends DatabaseBackend {

  /**
   * {@inheritdoc}
   *
   * We deliberately set this higher as we have a lot of data to store.
   */
  const DEFAULT_MAX_ROWS = 50000;

  /**
   * {@inheritdoc}
   */
  public function deleteAll() {
    // This method is called during a normal Drupal cache clear. We absolutely
    // do not want to flush the caches so we do nothing.
  }

  /**
   * This method will call the original deleteAll() method.
   *
   * Calling this method will permanently delete the caches for this cache bin.
   *
   * @throws \Exception
   */
  public function deleteCustomCaches() {
    parent::deleteAll();
  }
}

There are quite a few different pieces in play here, but we now have a custom cache bin that has been isolated from the main cache clearing mechanisms. This does mean that if we do need to clear the custom cache when we need to implement that functionality. For the project I was working on I created a form and a custom Drush command so that the cache could be cleared and rebuilt easily. That is beyond the scope of this post, but there really wasn't a lot of code written to do that. You just need to create a form where the submit handler runs the deleteCustomCaches() method in the CacheDatabaseBackend service.

There is also the issue that I have built all of this with a database cache system in mind. It's perfectly possible to create different implementations of cache bins (memcache for example) by adding some detection to the CustomCacheFactory get method and returning a different cache bin. This would, however, also mean writing a lot of code to override the different cache mechanisms. With the implementation above I am certain that this cache is being stored in the database, which is still a much quicker way of fetching the data I need and ultimately reduces the complexity of the problem. Overriding these classes to create custom interfaces is fine, but when I revisit the same code again in 6 months I need to remember how it all works. Documentation can only go so far.