Drupal

LocalGov Drupal Camp 2025 was held in The Abbey Community Centre (near Westminster Abbey in London) on July 3rd 2025. I travelled down (despite the English rail network's attempts to the contrary) for the day as an attendee.

It was a hot day in London on the 3rd July, and thankfully the Abbey Community Centre had lots of doors and windows that we could open to let some air through. Including a few fans dotted around to try and keep the rooms cool.

After a small icebreaker, where people stood up if this was their first LocalGov Camp or what sort of role they have, we started the camp with an introductory session.

Creating queues using the core queue classes in Drupal is fairly straightforward. You just need a mechanism of adding data to the queue and a worker to process that data.

As the data you add to the queue is serialised you can add pretty much any data you want to the queue, so the only limitation is rebuilding the data once you pull it out of the queue.

There are some situations where the core Drupal queue system needs to be altered in some way. You might want to separate the data into different tables, or have a different logic for creating or storing the queue items, or even integrate with a third party queue system for manage the queues.

When writing data to the queue database system Drupal will convert the information to a string using the PHP serialize() function. When the information is pulled out of the database the queue system will unserialize() the data to convert it back into the original information.

When you first start using the queue system you will probably use an array or the PHP stdClass object to store the information on your queue. Whilst this works, the information they contain is pretty free form and makes testing or working with the data a little cumbersome.

A much better way of storing data is by creating an object of a known type and using that as the storage medium for the queue.

I've talked a lot about the Batch API in Drupal recently, and I've mentioned that it is built upon the Queue API, but I haven't gone any deeper than that. I wrote about the Queues API in Drupal 7, but thought I would bring my understanding up to date.

A queue is a data construct that uses a "first in, first out" (or FIFO) flow where items are processed in the order that they were added to the queue with the first item added being processed first. This system has a lot of different uses, but is most important when it comes to asynchronous data processing. Drupal and many modules make use of the queue system to process information behind the scenes.

The difference between a queue and a batch is that the batch is for time sensitive things where the user is expecting something to happen. A queue, on the other hand, is more for data processing that needs to happen behind the scenes or without any user triggering the process.

This is the sixth article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

So far in this series we have looked at creating a batch process using a form, followed by creating a batch class so that batches can be run through Drush, using the finished state to control batch processing, processing CSV files through a batch process and finally adding to running batch processes. These articles give a good grounding of how to use the Drupal Batch API.

In this article we will look at how the Batch API is used within Drupal. The Batch API in Drupal is either used to perform a task, which I will call "direct", or to pass on the batch operations to a hook, which I will call "indirect". These aren't official terms you understand, I'm just using them here to separate how Drupal uses the Batch API.

This is the fifth article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

So far in this series we have looked at creating a batch process using a form, followed by creating a batch class so that batches can be run through Drush, using the finished state to control batch processing and then processing CSV files through a batch process. All of these articles give a good grounding of how to use the Drupal Batch API.

In this article we will take a closer look at how the batch system processes items by creating a batch run inside an already running batch process. This will show how batch systems run and what happens when you try to add additional operations to a running batch.

This is the fourth article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

So far in this series we have looked at creating a batch process using a form, followed by creating a batch class so that batches can be run through Drush and then using the finished state to control batch processing. All of these articles go together to form a basis of batch processing in Drupal.

In this article we will look at bringing these concepts together to perform a task that is quite common on websites, processing Comma Separated Value or CSV files, which we will do using the Drupal Batch API.

This is the third article in a series of articles about the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

So far in this series we have looked at creating a batch process using a form and then creating a batch class so that batches can be run through Drush. Both of these examples used the Batch API to run a set number of items through a set number of process function callbacks. When setting up the batch run we created a list of items that we wanted to process and then split this list up into chunks, each chunk being sent to a batch process callback.

There is another way to set up the Batch API that will run the same number of operations without defining how many times we want to run them first. This is possible by using the "finished" setting in the batch context.

This is the second part of a series of articles looking at the Batch API in Drupal. The Batch API is a system in Drupal that allows data to be processed in small chunks in order to prevent timeout errors or memory problems.

In the previous article we looked at how to setup the batch process using a form, with the batch methods being contained within the structure of the form class. When the form was submitted the batch process ran through 1,000 items and then printed out a result at the end.

Whilst there is nothing wrong with running the Batch API with everything in a form class, it is normally better to abstract the batch processing code into a separate class.

The Batch API is a powerful feature in Drupal that allows complex or time consuming tasks to be split into smaller parts.

For example, let's say you wanted to run a function that would go through every page on you Drupal site and perform an action. This might be removing specific authors or removing links in text. You might create a small loop that just loads all pages and performs the action on those pages.

This is normally fine on sites that have a small number of pages, like under 100 pages. But what happens when the site has 10,000 pages, or a million? Your little loop will soon hit the limits of PHP execution times or memory limits and cause the page to be terminated. How do you know how far your loop progressed through the data? What happens if you tried to restart the loop?