Too Many Open Files Error In PHP SoapClient

Whilst working on a project recently I hit upon a PHP error that I've never seen before. During a process where a soap service was calling an API the connection would fail and the program would fatal error and stop.

I had protection mechanisms in place to catch this kind of connection error, but the fatal error was caused when the program tried to throw the exception I had put in place to indicate a failed connection.

Here is the error message (with some of the detail removed). This is a Drupal site but that detail is irrelevant to the problem.

PHP message: PHP Warning:  require(/docroot/modules/custom/api_integration/src/Exception/UnableToConnectException.php): failed to open stream: Too many open files in /vendor/symfony/class-loader/ApcClassLoader.php on line 112
PHP message: PHP Fatal error:  require(): Failed opening required '/docroot/modules/custom/api_integration/src/Exception/UnableToConnectException.php' (include_path='') in /vendor/symfony/class-loader/ApcClassLoader.php on line 112

What this was telling me was that the program couldn't connect to the soap service but that it couldn't open the exception to handle this because there were 'too many open files'.

After some digging around I found people with similar problems, but nothing specifically related to the PHP SoapClient object. It turns out that this open files limit is under control from within PHP, but is part of the underlying operating system limits. Indeed, I tried out this process on a couple of different PHP versions and received the same result.

This limit (on unix systems) can be seen by using the command ulimit with the -n flag to look at the maximum number of open file descriptors. On my current system this shows that the limit is 256 files.

$ ulimit -n
256

In order to artificially hit this limit I created a little script that would open the same file over and over again, but never close the connection.

<?php

$handles = [];

for ($i = 1; $i <= 254; $i++) {
  $handles[] = fopen('/dev/null', 'w');
}

Running this script produced the following error, the final time that the script tries to open the file causes this warning. Note that I set the limit of the loop to 254. This is because the number of open files includes the PHP process and the file running the script.

PHP Warning:  fopen(/dev/null): failed to open stream: Too many open files in test.php on line 6

Looking back at the soap process I was running I could see that the soap client was connecting to the soap service and after a certain number of connections it simply couldn't connect any more and caused an error. The error regarding the inclusion of an exception file is an upstream red herring that was masking the problem.

After hours of looking through support forums talking about how to change the operating system settings (something I wasn't that happy doing) I found the solution. It turns out that the PHP SoapClient will connect to the soap service and keep the connection alive. Unfortunately, the next connection does not use this currently active connection and creates another one when the next soap request is made. During large operations this limit is quickly reached and will cause an error.

To prevent soap keeping the connections alive you need to pass in the keep_alive option when creating the SoapClient object.

$options['keep_alive'] = FALSE;
$soapClient = new \SoapClient($wsdl, $options);

This allowed me to go from a couple of hundred connections to many thousands and it allowed the soap process to complete successfully. It also meant that I had solved this problem within the code, rather than tinkering with the operating system to solve it.

Honestly, I hate using soap.