Drupal 9: Using Lazy Builders

The lazy builder API in Drupal allows the creation of highly dynamic content within a render array without having to disable the cache for the entire render array or the page the content is attached to. What this means in real terms is that the initial render array can be cached quite heavily, but lazy builders allows for additional rendering to be done in after the initial rendering pass to generate content.

There are several reasons why lazy builders might be useful. The most useful reasons are to present some highly personalised or dynamic content to a user, but you might also want to offset the rendering of some slow code to give it some more fine grained caching that stores the outcome in a cache bucket.

Lazy builders work by using a normal render array, but instead of rendering the content you inject a lazy builder placeholder into the content. A callback method is used to tell Drupal how to inject the content into the placeholder. Then, at a later stage of the rendering process, Drupal will call the callback function and replace the placeholder with the actual content.

If the Big Pipe module is installed then you can also benefit from the page being streamed to the user in parts. This means that if your render function is slow then it won't cause the rest of the page to be slow. The Big Pipe module will add in placeholders to the markup as the render array is generated that are then sent to the browser first. The footer of the page is then rendered and sent separately and JavaScript is used to replace the placeholders with the actual content.

Whilst lazy building doesn't provide you with performance benefits, it can be used in conjunction with Big Pipe to mask those problems so that your page will appear to load quickly.

Creating Lazy Builders

Lazy builders are created using the #lazy_builder type in the same way as other render elements. The render array must contain a callback as the first element, and an array of arguments to that callback as the second element. Lazy builder render elements must contain only #cache, #weight, and #create_placeholder, and should also contain no children.

For a full example of how to implement lazy builders let's take a Drupal block that has been placed onto a page. This block just prints out the current time and has a low cache max-age setting so that the time gets re-created on every page request. The cache max-age setting also means that the block is causing the rest of the page to be uncached since the that setting is bubbling up the rendering tree.

This is obviously an arbitrary example as you wouldn't print the time out like this, but it's simple and doesn't introduce any other elements.

<?php

namespace Drupal\mymodule\Plugin\Block;

use Drupal\Core\Block\BlockBase;

/**
 * Provides a block to show lazy building in action.
 *
 * @Block(
 *   id = "lazyblock",
 *   admin_label = @Translation("Lazy block"),
 * )
 */
class LazyBlock extends BlockBase

  public function build() {
    $build = [];

    $build['time'] = [
      '#markup' => date('r', time()),
    ];

    return $build;
  }

  public function getCacheMaxAge() {
    return 0;
  }

}

Since the time is a very dynamic piece of content we should  convert this into a lazy builder. The first thing to do is replace the #markup render element with a #lazy_builder render element. The callback here references a method in the same block class called lazyBuilder().

  public function build() {
    $build = [];

    $build['time'] = [
      '#lazy_builder' => [
        static::class . '::lazyBuilder',
      ],
    ];

    return $build;
  }

Before we go any further we need to ensure that our lazy builder implementation can call the lazyBuilder() method. To do this we need to implement the TrustedCallbackInterface interface so that we can tell Drupal that our lazy builder callback is allowed to be called.

When implementing this interface we need to add a method called trustedCallbacks(), which will be called automatically by Drupal through the detection of the interface. The return value of this method must be any methods within this class that are allowed to be used as callbacks.

Here is the basic implementation of this in our block class.

namespace Drupal\mymodule\Plugin\Block;

use Drupal\Core\Block\BlockBase;
use Drupal\Core\Security\TrustedCallbackInterface;

class LazyBlock extends BlockBase implements TrustedCallbackInterface {

// ...

  public static function trustedCallbacks() {
    return [
      'lazyBuilder'
    ];
  }

}

If you try to implement a lazy builder without the interface you will see the error \Drupal\Core\Security\UntrustedCallbackException being generated (with a corresponding log entry). If you have tried to implement a lazy builder and the page is crashing, then make sure this interface is correctly added and that your implementation of trustedCallbacks() is returning the correct method names.

As a side note, it is possible to have multiple lazy build methods and multiple callbacks being defined. Your implementation of trustedCallbacks() just needs to return multiple items in the return array.

With that in place we can now add out lazyBuilder() static method. This just returns a render array that re-creates the time readout we had before but rather than have the entire block with a low cache max-age. As the max-age cache has been applied to the output of the lazy builder we can remove the max-age from the block class.

  public static function lazyBuilder() {
    $build = [];
    $build['lazy_builder_time'] = [
      '#markup' => date('r', time()),
      '#cache' => [
        'max-age' => 0,
      ],
    ];

    return $build;
  }

Using max-age in this way means that the block output will be cached but lazy builder output will not be cached. Meaning that the block cache settings will not cause the page to be un-cacheable. Here is the full code of the new block with the lazy builder implementation.

<?php

namespace Drupal\mymodule\Plugin\Block;

use Drupal\Core\Block\BlockBase;
use Drupal\Core\Security\TrustedCallbackInterface;

/**
 * Provides a block to show lazy building in action.
 *
 * @Block(
 *   id = "lazyblock",
 *   admin_label = @Translation("Lazy block"),
 * )
 */
class LazyBlock extends BlockBase implements TrustedCallbackInterface {

  public function build() {
    $build = [];

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        self::class . '::lazyBuilder',
      ],
    ];

    return $build;
  }

  public static function lazyBuilder($format) {
    $build = [];
    $build['lazy_builder_time'] = [
      '#markup' => date($format, time()) . rand(100,999),
      '#cache' => [
        'max-age' => 0,
      ],
    ];

    return $build;
  }

  public static function trustedCallbacks() {
    return [
      'lazyBuilder'
    ];
  }

}

With these changes in place the output is very much the same. Although what actually happens with the markup and cache depends on the presence of the Big Pipe module.

It's worth looking at how this happens through Drupal's use of placeholders.

Placeholders

As the lazy builder is rendered a placeholder will be generated.

If Big Pipe is not installed this will be an internal drupal-render-placeholder that will be replaced as the page is rendered. The markup created through the normal rendering pipeline will look like the following.

<div id="block-lazyblock" class="contextual-region block block-mymodule block-lazyblock">
  <h2 class="block__title">Lazy block</h2>
  <drupal-render-placeholder callback="Drupal\block\BlockViewBuilder::lazyBuilder" arguments="0=lazyblock&amp;1=full&amp;2" token="u2Ri_CUbZXRAmoDOKhVB62KiXKM3Wg_4xjzWhfZIhR4"></drupal-render-placeholder>
</div>

Note that this never actually reaches the page markup as it is replaced by the content received from the lazy builder callback. You might have seen similar placeholders when dealing with media embedding inside Drupal.

If the Big Pipe is installed then things are done a little differently. The placeholder generated will reach the output of the page and will look something like the following.

<div id="block-lazyblock" class="contextual-region block block-mymodule block-lazyblock">
  <h2 class="block__title">Lazy block</h2>
  <span data-big-pipe-placeholder-id="callback=Drupal%5Cmymodule%5CPlugin%5CBlock%5CLazyBlock%3A%3AlazyBuilder&amp;args%5B0%5D=1&amp;token=MpqvUbAYFbejeQ8mSMHZYztGHD5lZLbEGt3y-LsNKi0"></span>
</div>

Big Pipe will then add another block of code to the footer of the page that includes a JavaScript element. This is used to replace the markup produced by the Big Pipe module with the markup produced by the lazy builder callback method.

This process is how Big Pipe works. The top of the page is rendered out with the placeholder elements within the page and then sent to the browser. At a later time, the footer of the page is then rendered out and sent to the page, which contains the scripts that perform an in-place replacement of the element.

If you are using a CDN or other static page cache then you might find that Big Pipe doesn't behave correctly due to the fact that the entire page is cached and so the streaming of placeholder elements and footer scripts doesn't happen. As the streaming doesn't take place the entire page is rendered out in full and the placeholders are replaced immediately, without running the lazy build callback method.

In both instances, the final output of the markup will look something like this.

<div id="block-lazyblock" class="block block-mymodule block-lazyblock">
  <h2 class="block__title">Lazy block</h2>    
  Sun, 01 May 2022 11:19:01 +0000
</div>

It is also possible to force the use of a placeholder using the #create_placeholder element, which should exist along side the #lazy_builder element in the render array.

    $build['time'] = [
      '#lazy_builder' => [
        static::class . '::lazyBuilder',
      ],
      '#create_placeholder' => TRUE,
    ];

Drupal will attempt to detect if a placeholder is required and so most of the time you won't need to include this element. If you find that your placeholder is missing for whatever reason then include this setting and everything should work as intended.

If you attempt to use the #create_placeholder element without a #lazy_builder element in place Drupal will throw an error.

Passing Arguments

It's also possible to pass arguments to lazy builder functions.

To do this just pass an additional array, containing your arguments, with your lazy_builder definition as the second element after the callback element.

We can use arguments to flip the lazy builder implementation we created above around a little and pass in the date formatter to the callback via the lazy builder definition.

  public function build() {
    $build = [];

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        $this::class . '::lazyBuilder',
        [
          'r'
        ],
      ],
    ];

    return $build;
  }

We can then rewrite our lazyBuilder() callback method to accept this parameter.

  public static function lazyBuilder($format) {
    $build = [];
    $build['lazy_builder_time'] = [
      '#markup' => date($format, time()),
    ];

    return $build;
  }

Note that all arguments passed must be in the form of a primitive data type; they can't be complex objects or arrays. You are essentially restricted to passing string, bool, int, float, or null values. This means that if you want to pass in an entity to load some data you need to pass in the ID of that entity and load it from within the callback.

Callback Styles

As there are a number of different styles of callback that can be used with lazy builders so it is worth talking about them as it's a good way of showing how to include different classes as part of the callback.

Self Referencing Method

This references a callback method from within the current class and can take the form of using the PHP "self" keyword and the "::class" special class constant, which prints out the fully qualified namespace of the class.

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        self::class . '::lazyBuilder',
      ],
    ];

The same can be done using the $this variable.

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        $this::class . '::lazyBuilder',
      ],
    ];

Static Class Method

Instead of referencing the class using self or $this it is also possible to reference the class using its name. This can be done using the ::class special constant.

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        LazyBlock::class . '::lazyBuilder',
      ],
    ];

Or by referencing the class name as a fully qualified string.

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        '\Drupal\mymodule\Plugin\Block\LazyBlock::lazyBuilder',
      ],
    ];

Using this method it is possible to reference other classes in the Drupal codebase, you just need to ensure that those classes also implement the TrustedCallbackInterface interface.

Service Method

It is also possible to reference a method within a Drupal service using the following syntax.

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        'some.service:someMethod',
      ],
    ];

Using this method we can allow our lazy builder callback method to have some dependency injection. The other styles here all reference the callback method statically and so must contain static calls to the various things that need to be used or loaded.

Again, in this case the service class must implement the TrustedCallbackInterface interface. Take a look at the service route_processor_csrf and method renderPlaceholderCsrfToken in Drupal core for a similar example of this in action.

A Better Example Of Lazy Builders In Action

Instead of printing out the time, I thought it would be better to look at a better example of how lazy builders could be used in a real world situation. I mentioned before that lazy builders are great for highly dynamic content, and one of the better examples of this is to print user details back to the user.

Here is a block that accepts the current user as a dependency injected parameter that I'll be using for the rest of the example.

<?php

namespace Drupal\mymodule\Plugin\Block;

use Drupal\Core\Block\BlockBase;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use Drupal\Core\Session\AccountInterface;

/**
 * Provides a block to show lazy building in action.
 *
 * @Block(
 *   id = "lazyblock",
 *   admin_label = @Translation("Lazy block"),
 * )
 */
class LazyBlock extends BlockBase implements ContainerFactoryPluginInterface {

  protected $currentUser;

  public function __construct(array $configuration, $plugin_id, $plugin_definition, AccountInterface $current_user) {
    parent::__construct($configuration, $plugin_id, $plugin_definition);
    $this->currentUser = $current_user;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $container->get('current_user')
    );
  }

  public function build() {
    $build = [];

    return $build;
  }

}

We first need to set up our block build() method to return the lazy builder implementation, which will call the printUserDetails() callback method. What we want to do is print out the user details in the lazy builder callback, but because we can't pass the user object upstream we need to just transfer the user ID.

  public function build() {
    $build = [];

    $build['dynamic_user_content'] = [
      '#lazy_builder' => [
        self::class . '::printUserDetails',
        [
          $this->currentUser->id(),
        ]
      ],
    ];

    return $build;
  }

Before adding the callback we also need to add in our TrustedCallbackInterface so that we can register the callback with Drupal.

use Drupal\Core\Security\TrustedCallbackInterface;

class LazyBlock extends BlockBase implements ContainerFactoryPluginInterface, TrustedCallbackInterface {

  //... removed for brevity.

  public static function trustedCallbacks() {
    return [
      'printUserDetails'
    ];
  }

}

Finally, we add in the printUserDetails() method, which accepts the single parameter of a user ID. With this in hand we then load the user account and use this to generate some content. In this case we are printing out the user's name and showing them how long they have been a member for.

Remember that we can't dependency inject in this situation since it's a static method, so we need to use Drupal service and entities directly. Think of a static method in much the same way as we would do in a hook function.

  public static function printUserDetails($uid) {
    $build = [];

    $account = User::load($uid);
    $t = \Drupal::translation();

    $build['lazy_builder_username'] = [
      '#markup' => '<p>' . $t->translate('Hi @name', ['@name' => $account->getDisplayName()]) . '</p>',
    ];

    $build['lazy_builder_memberfor'] = [
      '#markup' => '<p>' . $t->translate('Member for: @time', ['@time' => \Drupal::service('date.formatter')->formatTimeDiffSince($account->getCreatedTime())]) . '</p>',
    ];

    $build['#cache'] = [
      'contexts' => [
        'user',
      ],
    ];

    return $build;
  }

The response from the printUserDetails() is still cached within Drupal's rendering process and so we need to include a cache setting in the response from the lazy builder callback. In this instance we are setting the cache context to be for the "user", which means that only the user in question sees their version of the response. We could also set the max-age value to 0 if we didn't want the lazy builder output to be cached at all.

If you want to see how lazy builders and the Big Pipe module can speed up your page load try adding a sleep() to the printUserDetails() method above. You'll see the page 'pop' in with the placeholder markup in place, followed a few seconds later by the output from the lazy builder method

This example is adapted and simplified from an example in the Drupal Examples module that shows how to use lazy builders and a number of other rendering elements. The example shows a lot more than just lazy builders, which is why I have simplified it slightly.

When To Use Lazy Builders?

There are some situations when the use of lazy builders would be beneficial.

• When creating highly dynamic content such as printing the current time or rendering statistics a lazy builder should be used. This allows the dynamic content to be dynamically generated every page load without interfering with the rest of the page cache.
• Related to this is content that can be cached but would create a high number of values in the cache tables. For example, if you created a block that printed out the user's name you would have to include the users ID in the cache context. If you have many users then this will add lots of data your cache tables, most of which will have a very low hit rate. You could instead use the max-age cache setting to reduce the cache age to 0 and prevent caching without adding lots of data to the cache tables.
• By using a service as the lazy build callback it's possible to abstract away the rendering of certain elements in your render array without effecting the render array itself. This allows you to move some of that complexity into other services so that they can be re-used by other rendering functions. Useful when you have an ajax callback that is also used in the lazy load render pipeline and you want to use the same method in multiple places.
• If you have some code that is slow to generate content then using a lazy builder can offset the slowness. This might just be some slow to generate code on your site, but might also include callbacks to APIs or anything that would otherwise slow down the page. By using lazy builders as well as Big Pipe you can prevent the code from slowing down the entire page since the rendering will be offset from the page generation.

Troubleshooting Lazy Builders

If you have implemented a lazy builder and it isn't speeding up your page load or just isn't working as expected then there are some things you can try.

• Check the cache settings of your lazy builder callback method. If you are not explicitly setting a cache setting for that output then Drupal will make some assumptions about how it should be cached, which isn't always right for your use case. Adding in a #cache element to one or more of your return elements will help you out here.
• Look in the cache_render table for the outcome of your rendering process. This should help you find what has been rendered and what cache tags have been assigned to that cache.
• Check your Drupal site cache settings. My local environment normally has a very 'cache-free' setup by default so that I can develop and test things quickly without having to flush caches a lot. I will normally turn this off when testing cache settings on a module before pushing it upstream. This interface will mean that your render pipeline and your lazy builders are re-created on every page load.
• The Big Pipe module might not be active. This means that your placeholders would be held internally and you will not benefit from the streamed page load.
• An upstream CDN or Varnish layer might be caching your entire page and so all of the output of the Big Pipe rendering process will be served at the same time. You'll need to find another mechanism to work around this.

Conclusion

Lazy builders provide some convenience for rendering pathways, but when coupled with Big Pipe the benefits become obvious. Highly dynamic or slow to generate content can be made to cause less of an impact to the page load times using the Big Pipe rendering process. It is therefore recommended to think about how you can use lazy builders when building out renderable elements in Drupal.

This is not a substitute for fixing performance problems. If you have some slow code on your page then lazy builders will only mask the apparent slowdown of those elements. You should also be aware that some platforms do not support Big Pipe at all and so using this approach will not provide any benefits.