Playing With ReactPHP

I recently saw an implementation of a Twitter wall that used node.js to run searches on Twitter and post the results on a webpage. I had been wanting to create something using ReactPHP so I thought this was a good opportunity to have a go. ReactPHP, if you haven't heard of it, is an event-driven, non-blocking I/O that is essentially the PHP equivalent of node.js. The major difference is that ReactPHP is written in pure PHP with no extra components, whereas node.js is a collection of different programs, interfaces and languages. As a first attempt I wanted to create something simple so it needed to use simple JavaScript to load in the latest tweets for a given hashtag from a ReactPHP server. I have to warn that this is a simplistic implementation of ReactPHP, but it shows the basics of how to get started.

Building The Frontend

The first thing to do is create a HTML page that will display the tweets. When creating a simple ReactPHP server a port is normally assigned, in which case the first thing I had to do is figure out how to do a request to a certain port number in JavaScript (not port 80). Normally this would violate the same origin policy that JavaScript enforces but we can get around this using a technique called JSONP. This involved a cross domain request that returns a JSON string wrapped in a function callback. This is possible in jQuery using the getJSON() function.

The first parameter of this function is the full URL to our ReactPHP server (including the port), which must contain a parameter similar to 'jsoncallback'. This is what is used on the server side to send back the correct data format. The second parameter isn't used but it allows you to pass additional arguments. The third parameter is the callback function that does something with the returned data. We will return a JSON object with a property of 'content' that will contain the tweets.

function loadtweets() {
    $.getJSON("http://www.tweetstream.local:4000?jsoncallback=?", {},
      function(data) {
        $('#stream').html(data.content);
    }
  );
}

When we run this code it will send a request to the following URL structure.

http://www.tweetstream.local:4000?jsoncallback=jQuery18205617587673477829_1353020048954&_=1353020058958

What we do on the server side is return a string that wraps some JSON output inside a function with the same name as the jsoncallback parameter. The function is expanded, and the JSON it contains is then available to the return function. The only thing we do here is pass the entire content string into an element with an ID of 'stream'.

jQuery18205617587673477829_1353020048954('content':'text')

This function is basically wrapped in a call to setInterval(), which is used to poll the ReactPHP server every 5 seconds to see if there are any tweets. We could expand on this to try and keep the tweets already on the screen and just append the new ones to the top, but this will work for the purposes of this demonstration.

<div id="stream">
</div>
<script src="jquery-1.8.2.min.js"></script><script>
<!--//--><![CDATA[// ><!--
 
$(document).ready(function() {
 
  // Run the loadtweets() function every 5 seconds.
  window.setInterval(function() {
    loadtweets();
  }, 5000);
 
  function loadtweets() {
    $.getJSON("http://www.tweetstream.local:4000?jsoncallback=?", {},
      function(data) {
        $('#stream').html(data.content);
      }
    );
  }
});
 
//--><!]]>
</script>

Composer

With that in place we can now build our ReactPHP components. This is done using a tool called Composer, which is a dependency manager built in PHP. I won't go into Composer too much, but to get a copy just run the following command in the directory you want to work from.

curl -s https://getcomposer.org/installer | php

This will download a file called composer.phar, which is used to download various packages. All we need to do is tell Composer what packages we need to download. This is done using a file called composer.json where we list (in JSON format) the components we want for this project. Obviously we need to include ReactPHP but I have also included Zend Framework 1 so that I can use the Zend_Service_Twitter_Search class. Here is the composer.json file for this project. I could use a different Twitter class, but I am familiar with the Zend Framework implementation so I went with that one out of convenience.

{
    "require": {
        "react/http": "0.2.*",
	"zendframework/zendframework1": "dev-release-1.12"
    }
}

To get Composer to do something with this just call the composer file with the flag install like this.

composer install

This will download ReactPHP and Zend Framework 1 as well as a few other dependencies. You will now have a directory called vendor that will contain the downloaded packages as well as an autoloader.php file. To use anything from these directories just include the autoload.php file and start using the classes you need.

autoload.php
composer
evenement
guzzle
react
symfony
zendframework

ReactPHP

We are finally ready to start using ReactPHP. The first step (aside from including the autoload.php file) is to instantiate a couple of objects. These are the event loop, socket and http objects. ReactPHP works by creating an internal event loop, which other systems can then plug into. The loop is then used to create a socket server, which is then in turn used to create a HTTP server. The HTTP server is essentially a wrapper around the socket object with some extra stuff to take care of headers and other HTTP related things.

The HTTP object is then set to fire an event when it receives and incoming request using the on() method. Inside this method we setup a closure to receive the incoming request and send corresponding output. This closure is where we find the jsoncallback wrapper name and return a json string containing all of the tweets we have loaded.

Finally, we set up the socket server to listen to port 4000 and run it.

<?php

require 'vendor/autoload.php';
 
// load objects
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$http = new React\Http\Server($socket, $loop);
 
// set up a request event
$http-?>on('request', function ($request, $response) use() {
  $query = $request->getQuery();
 
  if (!isset($query['jsoncallback'])) {
    // no jsoncallback parameter passes, so we quit.
    $response->writeHead(200, array('Content-Type' => 'text/plain; charset=utf-8'));
    $response->end('finish' . PHP_EOL);
    return;
  }
 
  // Set up the correct headers
  $response->writeHead(200, array(
    'Content-Type' => 'application/x-javascript; charset=utf-8',
    'Expires' => 'Mon, 26 Jul 1997 05:00:00 GMT',
    'Last-Modified' => gmdate("D, d M Y H:i:s") . ' GMT', 
    'Cache-Control' => 'no-store, no-cache, must-revalidate', 
    'Cache-Control' => 'post-check=0, pre-check=0',
    'Pragma' => 'no-cache',
  ));
 
  // Load in the tweets
  $searchResults = loadtweets();
 
  // Generate the output
  $output = '';
  if ($searchResults !== FALSE) {
    foreach ($searchResults as $result) {
      $text = clean_tweet($result['text']);
      $output .= '<div class="tweet">';
      $output .= '<div class="screen_name">' . $result['from_user_name'] . '</div>';
      $output .= '<div class="profile_picture"><img src="' . $result['profile_image_url'] . '" /></div>';
      $output .= '<div class="tweet_contents">';
      $output .= '<p>' . $text . ' <span class="timestamp">on ' . $result['created_at'] . '</span></p>';
      $output .= '</div>';
      $output .= '</div>';
    }
  }
 
  // JSON encode and wrap the output in the jsoncallback parameter
  $output = $query['jsoncallback'] . '(' . json_encode(array('content' => $output)) . ')' . PHP_EOL;
 
  // End the response
  $response->end($output);
});
 
// Listen to socket 4000
$socket->listen(4000);
 
// Run the server
$loop->run();

The clean_tweet() function above is taken from fiveminuteargument's PHP implementation of Remy Sharp's JavaScript Twitter parsing functions. These functions will add HTML links to the things like usernames and hashtags and means that the Twitter stream itself is a little more user friendly.

Of course we are missing the vital component of actually getting the tweets. We do this using the Zend_Service_Twitter_Search class which is part of Zend Framework. We already have access to this class thanks to our earlier actions with Composer so we don't need to include anything else. Here is the function in full, which will search for tweets in english using the hashtag #drupal.

function loadtweets() {
  // Get twitter search object
  $twitterSearch  = new Zend_Service_Twitter_Search('json');
 
  // Set the correct language for the search
  $searchQuery = array(
    'lang' => 'en'
  );
 
  // Run the search
  $searchResults  = $twitterSearch->search('#drupal', $searchQuery);
 
  // Return the results
  return $searchResults;
}

After saving this script to a file you can run it just like any other PHP script.

php TweetStream.php

You can check that this script is running by running it through curl via the following command. Without the jsoncallback parameter the script will just return 'finished' and quit but it's a good way of making sure that the right port is being listened to.

curl www.tweetstream.local:4000

Anything you echo out within the script will be output to the terminal instead of to the user awaiting a connection. This is a good idea to see that things are firing as expected. To stop the program again just Ctrl+c.

What we are doing here is no better than just creating a web based script that will pull the latest tweets down and return them to the browser. The power of ReactPHP is that we can separate searching for tweets from the user requesting the list of tweets. We can do this by using the addPeriodicTimer() method of the event loop object to create a timer that executes every 10 seconds. With this we can pull down the latest tweets in a controlled manner without relying on user interactions to run the script.

$loop->addPeriodicTimer(10, function() use() {
  gettweets();
});

The gettweets() function runs the twitter search and stores each of the results in a file.

function gettweets() {
  // Get twitter search object
  $twitterSearch  = new Zend_Service_Twitter_Search('json');
 
  // Set the correct language for the search
  $searchQuery = array(
    'lang' => 'en'
  );
 
  // Run the search
  $searchResults  = $twitterSearch->search('#drupal', $searchQuery);
 
  // Store the results
  if ($searchResults !== FALSE && count($searchResults['results']) > 0) {
    foreach ($searchResults['results'] as $result) {
      if (!file_exists($cache_directory . $result['id'])) {
	    $fp = fopen($cache_directory . $result['id'], 'w+');
	    fwrite($fp, serialize($result));
	    fclose($fp);
      }
    }
  }
}

We can then change the loadtweets() function to pull the data out of the saved files and return it to the request event when needed. As we used the id of the tweet for the filenames we can reverse sort them to get a sorted list of tweets with the last one first. This means that searching for tweets and showing those tweets to the user are now two separate events.

function loadtweets() {
  // Get a list of the files in the cache directory
  $files = array();
  foreach (new DirectoryIterator('cache') as $fileInfo) {
    if ($fileInfo->isDot()) {
      continue;
    }
    $files[] = $fileInfo->getFilename();
  }
 
  // Reverse sort the files we have found
  rsort($files);
 
  // Uncompress each of the files
  foreach ($files as $key => $file) {
    $files[$key] = unserialize(file_get_contents('cache/' . $file));
  }
 
  // Return the result
  return $files;
}

Future Plans

The next step in this is to create a socket server and connect to it using either the HTML web socket interface or something like socket.io.js. This would do away with the need to a timing interval to pull the latest results. As new items are found they would be pushed directly to the browser and then just pushed to the top of the list rather than re-loading the entire tweet steam all over again.

One thing that I found interesting whilst playing with this stuff is the paradigm shift it required. I have been using PHP in a web server environment for so long that I kept thinking about the application in a stateless state. One main revelation was that any variable or function I created in the global scope persisted in that scope until the program was stooped. This meant that anything I set during one request was still there when the next request was made.

I can see lots of potential for ReactPHP, even after only a few minutes messing about with simple sockets and ports. What I have accomplished here was done in a couple of hours, although I spent a while trying to figure out how to contact the ReactPHP server in JavaScript. Take a look at the ReactPHP website for more information and some more (better) implementations of ReactPHP. I am currently familiarising myself with the package and will definitely be revisiting it again in the future.