Storing Automated YSlow Tests With Show Slow

I was at a meeting of the Manchester Web Performance Group the other day where Tom Taylor gave a talk about some of the performance testing tool he uses at Laterooms.com. He used a ruby script to set up some preferences in Firefox which then ran Selenium to open some web pages and test them with YSlow. The results of the YSlow inspection are then sent to a Show Slow server where the results can be graphed over time. I realise that I've just mentioned a whole stack of technologies there, so let me pick out the important ones:

Selenium is a remote control agent for web browsers, although it is most stable in Firefox. I have written about this tool before but it allows us to automate interaction with a website via a series of selenium scripts. These scripts can be exported into different code formats, including PHP.

YSlow is a plugin for Firebug created by Yahoo that attempts to grade sites based on a set of performance criteria. These are things like how the HTML is structured and how many HTTP requests are made.

Show Slow is a PHP tool that accepts parameters from YSlow and stores them in a database. These results are graphed and summarized which allows the inspection of Show Slow results over time. It doesn't have the ability to add data to it directly, instead this is done via an API. There is a public facing service located at www.showslow.com where you can see results of other websites.

This automation of YSlow tests got me to thinking about how to accomplish this using PHPUnit to handle the Selenium tests. All I would need to do is open up Firefox and configure it to auto inspect a given page with YSlow before posting the results to Show Slow.

First I had to install Show Slow. This is quite a straightforward process, but will probably require a LAMP based system in order to work correctly. There are more instructions on how to install the software on the Show Slow site. After a few minutes I had a Show Server on my network that I could send YSlow data to.

After looking through the PHPUnit Selenium extension source code for a bit I realised that I needed to approach it from a different angle. There was nothing that allowed me to configure Firefox from within PHPUnit. The solution came with Firefox profiles. It turns out that you can get Firefox to load itself with a different profile instance which will contain a different set of plugins and configuration options. This can be done by opening up a terminal any typing in:

firefox -ProfileManager -no-remote

Windows users will need to use firefox.exe and OSx users might need to reference the Firefox application absolutely.

What this will do is open up a window where you can manage Firefox extensions. Just create a new one and save it to a specific directory and remember the path that you put it in.

This will open a default version of Firefox with no extensions included and all of the config options set to default. The first step is to install Firebug and YSlow so that you actually have the tools you need to run the tests.

Next, you need to configure Firebug and YSlow to automatically run the tests on the page and send the results to your Show Slow server. Go to the Firefox about:config page (enter about:config into the address bar) and alter some of the options.

First we need to set up the YSlow metrics to push the test results to Show Slow.

extensions.yslow.beaconUrl = http://my.showslow.server.com/beacon/yslow/
extensions.yslow.beaconInfo = basic,grade,stats
extensions.yslow.optinBeacon = true

We also need YSlow to automatically run when the page loads.

extensions.yslow.autorun = true

There are also some Firebug options to set as well. What we need to do is ensure that Firebug is loaded and that the YSlow pane is shown when Firebug is loaded. If YSlow isn't shown on screen when the page is loaded then the tests will not run so this is important.

extensions.firebug.addonBarOpened = true
extensions.firebug.allPagesActivation = on
extensions.firebug.defaultPanelName = YSlow

Once done you can shut down Firefox. You can optionally disable some of the other features like the built in spell checker as they aren't really needed and will speed up the time it takes to start up Firefox.

Now we need to run the Selenium Server in much the same way as usual, but here we feed in the Firefox template we created earlier using the -firefoxProfileTemplate parameter. If you are using Windows then you might need to surround the argument with double quotes.

java -jar selenium-server-standalone-2.24.1.jar -firefoxProfileTemplate /path/to/selenium/firefox_profile

When you use this Selenium server you will load up Firefox with the profile created, effectively running YSlow on every page request. All we have to do now is tie it together by create some PHPUnit tests. These tests don't need to be complex, all we are doing is loading up the web page and getting YSlow to post the results it has to Show Slow. You can do other things within this test if you like but it's important to add some time at the end to allow YSlow to properly complete all tests and send the results. If you don't put this time in there is a good chance that your tests will produce no output.

<?php
class SeleniumHashbangcodeTest extends PHPUnit_Extensions_SeleniumTestCase 
{
  protected function setUp()
  {
    $this->setBrowser("firefox");
    $this->setBrowserUrl("http://www.hashbangcode.com/");
  }

  public function testMyTestCase()
  {
    $this->open("/");
    sleep(10);
  }
}

You can run this in the same way as you would normally run unit tests with PHPUnit. You can also take a screenshot of the page being tested by including a call to the takeScreenshot() method. The $screenshotPath and $screenshotUrl variables need to be set before this is done, but this is a good way of seeing that things are working as expected.

public function testMyTestCase()
{
  $path = '/';
  $this->open($path);
  $this->screenshotPath = __DIR__;
  $this->screenshotUrl = $path;
  $this->takeScreenshot();
  sleep(10);
}

The best way to structure the tests is by creating a separate test class for each site, which might contain more than one page inspection. You can run a directory full of test classes in PHPUnit if you end each file name with "Test.php" and then just pass the directory into the PHPUnit command. This means that all of the Selenium/YSlow tests can be run at once in an automated fashion either through a cron job or via a CI server like Jenkins.