Drupal 8: Running A Batch Through An AJAX Request

This problem came out of a recent project I was working on. I had to perform a bunch of API lookups on behalf of a user that could take a minute or so to complete. The results of the API lookups were cached and so once it was done the site would be very quick, unfortunately the initial API lookup slowed down the page load quite considerably so this created a problem.

Rather than just doing the API loading in the page load process and making the user sit through it I created a batch process to load the API results in a more manageable manner. This created another problem as although the batch runner in Drupal is really good, it is perhaps a little too much just to show to users and expect them to understand what is going on. This led me to think if I could run a batch process via an AJAX callback from the page they were trying to load.

I have to say that I searched for a solution to this problem for quite a while. Turns out that no one had solved this problem before (that I could see).

The first step in this was to create a library that would control the AJAX callback to the batch process.

loader:
  js:
    js/loader.js: {}
  dependencies:
    - core/jquery
    - core/drupalSettings

Along with the associated JavaScript file called loader.js. I didn't know what I needed to fill in here so just a stub, so I first created it with a minimal AJAX callback to a route that would trigger the batch process.

(function ($, Drupal) {
  'use strict';

  Drupal.behaviors.account = {
    attach: function attach(context, settings) {
      $.ajax({
        url: Drupal.url('loading/ajax'),
        type: 'POST',
        contentType: 'application/json; charset=utf-8',
        dataType: 'json',
        success: function success(value) {
          // Do stuff...
        }
      });
    }
  };
})(jQuery, Drupal);

This was loaded onto the page by attaching it to the output of the page. In this case I was just using a static interstitial page to run the AJAX request.

  public function myLoading() {
    return [
      '#theme' => 'my_loading',
      '#attached' => [
        'library' => [
          'my_module/loader',
        ],
      ],
    ];
  }

This was a standard Drupal controller so it's not doing anything special. This could potentially be in a block or something but we wanted to take the user to a 'loading' page whilst we grabbed things from the API and then send them back to the page they were trying to access.

The final step was to set up the AJAX endpoint so that it could trigger the batch process.

  public function ajaxBatchProcess() {
    // Setup batch process.
    $this->batchService->setupLengthyBatchProcess();

    // Get the batch that we just created.
    $batch =& batch_get();

    // Ensure that the finished response doesn't produce any messages.
    $batch['sets'][0]['finished'] = NULL;

    // Create the batch_process(), and feed it a URL that it will go to.
    $url = Url::fromRoute('user.page');
    $response = batch_process($url);

    // Return the response to the ajax output.
    $ajaxResponse = new AjaxResponse();
    return $ajaxResponse->addCommand(new BaseCommand('batchcustomer', $response->getTargetUrl()));
  }

I have missed out some of the complexity here, but essentially I had wrapped the batch creation in a service. This service was essentially a class that setup and managed the batch run. The function setupLengthyBatchProcess() essentially just wraps the batch_set($batch); call and can be used in a submit handler or something similar. The batch finish function also reported on what it had just completed so this was removed in order to prevent these messages being shown to the user.

After some investigation I found that the result of the batch_process() function is to return a path to the batch runner (e.g. batch?id=12345&op=start). I didn't want to show this to the user so I couldn't just return a redirect response to the AJAX request.

One way around this was to set the 'progressive' option in the batch settings. What this does is essentially process the entire batch in one go, which isn't really what I wanted to do. What we lose (aside from being able to handle the amount of data being processed in little chunks) is the ability to tell the user how long they have to wait as we can't report on how far through the batch process we are. A simple progress indicator is really useful, even if the user only has to wait 10 seconds.

If you do want to go down this progressive route then just change a few files in the above code and everything I have shown so far will work. Note that I'm no longer sending a URL to the batch_process() function as we aren't going to be doing any redirecting. You also need to make sure that when the AJAX request has finished that it redirects correctly as it currently does nothing.

$batch =& batch_get();

$batch['progressive'] = TRUE;

$response = batch_process();

What I ended up doing was reverse engineering the batch runner that Drupal 8 has. This uses a plugin called ProgressBar that basically watches an endpoint and reports on the progress. the updateCallback() function is used to either update the user as to progress or perform a redirect. This effectively runs the batch process in the same way that Drupal would normally run it.

(function ($, Drupal) {
  'use strict';

  Drupal.behaviors.account = {
    attach: function attach(context, settings) {
      var progressBar = void 0;

      function updateCallback(progress, status, pb) {
        $('#updateprogress').html(progress + '%');
        if (progress === '100') {
          pb.stopMonitoring();
          window.location = '/user';
        }
      }

      function errorCallback(pb) {
      }

      $.ajax({
        url: Drupal.url('account/loading/ajax'),
        type: 'POST',
        contentType: 'application/json; charset=utf-8',
        dataType: 'json',
        success: function success(value) {
          progressBar = new Drupal.ProgressBar('updateprogress', updateCallback, 'POST', errorCallback);
          progressBar.startMonitoring(value[0].data + '&op=do', 10);
        }
      });
    }
  };
})(jQuery, Drupal);

Ultimately, this works very well. When the users login (and we detect that they have no cache yet) they are sent to this intermediate page and the batch process is run behind the scenes whilst they wait for the API calls to complete. The process reports back how far it has got to finish and once complete the users are sent back to their account page.