PHP Streams

Note: This post is over two years old and so the information contained here might be out of date. If you do spot something please leave a comment and we will endeavour to correct.

13th December 2019 - 14 minutes read time

Streams are a way of generalising file, network, compression resources and a few other things in a way that allows them to share a common set of features. I stream is a resource object that has streamable behaviour. It can be read or written to in a linear fashion, but not necessarily from the beginning of the stream.

Streams have been available in PHP for quite a while (at least since version 4.3.0) and are used pretty transparently by most PHP programmers. They can be used to access files, network resources, command line arguments, pretty much anything that goes through the input/output stream in PHP.

I was recently looking at ReactPHP and found that the use of streams was a requirement in order to prevent blocking the input/output stream. Although, I had seen streams being used in PHP applications, I wasn't entirely certain how to use them myself. As a result I thought I'd put together a post about them.

Streams come in especially handy when accessing large files. Let's say we needed to access a log file that was a few hundred megabytes in size. We could just read it the file into memory with file_get_contents() and then run through the contents until we find what we need. The problem with this approach is that we would quickly run out of system resources and the program would just error before we had a chance to access the data.

A much better solution to this is to use PHP streams. We can use the function fopen() to get a file handle and then stream the file a little bit at a time, thus reducing the amount of memory needed to read the file.

$handle = fopen('logfile.log', 'r');
while (false !== ($line = fgets($handle))) {
  // Do something.
}

This is the transparent stream usage I mentioned above. Whilst we haven't explicitly used any stream functions or syntax we are still using PHP streams to read in chunks of the file.

Stream Syntax

You can access streams directly using the syntax [scheme]://[target]. The scheme is the name of the wrapper and the target is what is being read or written, which largely depends on the thing being read. PHP has a number of different built in wrappers, transports and stream filters. You can find out what they are using the following three functions.

<?php
print_r(stream_get_transports());
print_r(stream_get_wrappers());
print_r(stream_get_filters());

On my current system this outputs this.

Array
(
    [0] => tcp
    [1] => udp
    [2] => unix
    [3] => udg
    [4] => ssl
    [5] => tls
    [6] => tlsv1.0
    [7] => tlsv1.1
    [8] => tlsv1.2
)
Array
(
    [0] => https
    [1] => ftps
    [2] => compress.zlib
    [3] => compress.bzip2
    [4] => php
    [5] => file
    [6] => glob
    [7] => data
    [8] => http
    [9] => ftp
    [10] => phar
    [11] => zip
)
Array
(
    [0] => zlib.*
    [1] => bzip2.*
    [2] => convert.iconv.*
    [3] => string.rot13
    [4] => string.toupper
    [5] => string.tolower
    [6] => string.strip_tags
    [7] => convert.*
    [8] => consumed
    [9] => dechunk
)

This gives plenty of options to connect to all sorts of streams.

The php:// Stream

The built in php:// stream is perhaps the most versatile stream wrapper and a good starting point to see how to use streams. An example of the php:// wrapper being used is with the stdin target. This is a read only stream that will read the input from the standard input and create it as a stream in PHP. We can then read the contents of the stream through the fgets() function. This reads one line in at a time, but is fine for what we are doing. We could also use fread() to do this as well.

$stdin = fopen('php://stdin', 'r');
while (false !== ($line = fgets($stdin))) {
    echo $line;
}

This script can be used by piping the output of a file into the script on the command line.

cat somelog.log | php stream.php

The output of the script is the contents of the file printed onto the command line, but the core idea of piping into a PHP script is outlined here.

Another example is using php://input to read the input to the script. This is another read only stream that can be used to read the raw body of POST requests to the script. Let's say we called a script with the following curl request.

curl -X POST -d "data1=thing" -d "data2=anotherthing" "http://localhost:8000/stream.php"

We can access the POST data by using the $_POST super-global variable.

print_r($_POST);

This will print out the following.

Array
(
    [data1] => thing
    [data2] => anotherthing
)

By using php://input we can read out the contents of the POST data directly as an input stream.

$input = fopen('php://input', 'r');
while (false !== ($line = fgets($input))) {
    echo $line;
}

This outputs the following.

data1=thing&data2=anotherthing

php://memory and php://temp

PHP 5.1 introduced php://memory and php://temp which allow reading and writing to either memory or a temporary file.

The php://memory can be used in much the same way as the php://input stream, but in this case we can both read and write to it. Here is an example of opening a memory stream and then writing some data to it.

<?php

// Open memory stream for reading and writing.
$memoryStream = fopen('php://memory', 'rw+');

// Write to the stream.
$text = 'sometext' . time();
fwrite($memoryStream, $text);

No file was created during this code, we are just writing to memory. We can also read from the memory stream as we normally would with files and streams. Note that you can't do this straight away though, we must first rewind the pointer to the start of the stream before we can read the contents of the stream. This can be done using the rewind() function.

// Rewind the stream back to the beginning.
rewind($memoryStream);

// Read the data from the stream.
while (false !== ($line = fgets($memoryStream))) {
    echo $line;
}

The php://temp stream acts like php://memory, but the crucial difference is that if you write more than a certain amount of data (by default 2MB) then a file is created to store this information. The location of this file depends on your system, but you can find this information out by using the sys_get_temp_dir() function. This file will be automatically deleted when the PHP script ends.

It is also possible to control how much data you can add to the php://temp stream before a file is created. This is done by appending "/maxmemory:n", where 'n' is the maximum amount of data to keep in memory before using a temporary file, in bytes.

// Write to memory, or a file if more than 1028 bytes is written.
$tempStream = fopen('php://temp/maxmemory:1028', 'rw+');

With that in place, if the data we write exceeds more than 1028 bytes then a file is created in the temporary directory.

Streaming A Web Page

You might have seen that http and https are available as stream wrappers. This means that we can open up a web page as a stream resource and stream it in as data. Here is an example of this in action.

$stream = fopen('https://www.hashbangcode.com/', 'r');
echo fread($stream, 50);

This returns the following output.

<!DOCTYPE html>
<html lang="en" dir="ltr" prefix="

Note that we are cutting off the end of the html tag, but this is because we are only reading in the first 50 bytes of data, which is where the data stops.

Filters

Filters are a form of meta-stream that allow you to manipulate data as it passes through a stream. This means you can add filters to function calls like readfile() or fopen() without having to change data before (or after) it passes through the stream. The syntax of filters is php://filter/read=[filter]/resource=[stream] with the filter being from stream_get_filters() above and the stream being any usable stream.

For example, we can expand on the php://stdin example above by passing the input through a filter. In this case we are changing all of the text to be uppercase.

$input = fopen('php://filter/read=string.toupper/resource=php://stdin', 'r');
while (false !== ($line = fgets($input))) {
    echo $line;
}

Another example is to swap out the resource part with a web address. This will open the web address as a stream.

$input = fopen('php://filter/read=string.toupper/resource=https://www.hashbangcode.com', 'r');

When we read the contents of the filtered stream it will be in uppercase.

We can do the opposite of this by using the file_put_contents() with a similar filter setup. The snippet below will write the text 'hashbangcode', in uppercase, to a file called uppertext.txt. Note the use of the write filter instead of the read filter.

file_put_contents("php://filter/write=string.toupper/resource=uppertext.txt", "hashbangcode");

Stream Contexts

Another powerful part of streams is creating contexts. This allows us to change the way in which the streams are used. For example, when we use the file_get_contents() function to access a URL it will always do this using a GET request.

echo file_get_contents("https://www.hashbangcode.com");

In order to change the type of request being made we create a stream context and pass this to the function. Stream context are created using the stream_context_create() function, which takes an array that details the context that needs to be created.

The following is an example of creating the needed context to create a POST request (including the content of the request) using stream_context_create() and then using file_get_contents(). I'm using https://postman-echo.com/post/ to test this post request as it will return whatever you sent it and so is a good way of testing that everything worked.

$data = [
    'data1' => 'hello world',
];

$context = stream_context_create(
    [
        'http' => [
            'method' => 'POST',
            'header' => [
                'Accept: application/json',
                'Content-Type: application/x-www-form-urlencoded'
            ],
            'content' => http_build_query($data),
        ],
    ]
);

echo file_get_contents("https://postman-echo.com/post/", null, $context);

Running this returns the following.

{"args":{},"data":"","files":{},"form":{"data1":"hello world"},"headers":{"x-forwarded-proto":"https","host":"postman-echo.com","content-length":"17","accept":"application/json","content-type":"application/x-www-form-urlencoded","x-forwarded-port":"443"},"json":{"data1":"hello world"},"url":"https://postman-echo.com/post/"}%

Another good example of using stream contexts is when bypassing the strict SSL rules that PHP has. Let's say we were trying to connect to a local server that doesn't have a fully signed SSL certificate. We can create a stream context that allows the SSL verification step to be bypassed.

$context = stream_context_create(
    [
        'ssl' => [
            'verify_peer' => false,
            'verify_depth' => 5,
            'allow_self_signed' => true,
            'verify_peer_name' => false,
        ]
    ]
);

echo file_get_contents("https://localhost", null, $context);

Conclusion

There are a lot more to streams in PHP than what I have detailed here. PHP includes a bunch of functions that allow you to interact with streams in lots of different ways. It is also possible to create custom stream wrappers and filters, which I might detail in later posts.

PHP