curl

Timing Page Responses With Curl

27th March 2022 - 8 minutes read time

Timing web requests is possible in curl using the -w or --write-out flag. This flag takes a number of different options, including several time based options.

These timing options are useful for testing the raw speed of requests from a web server and can be an important tool when improving performance and quickly getting feedback on the response.

The -w or --write-out flag in curl has a number of different options, far more than I can add here. You can use these options by surrounding them in a "%{parameter}" structure and passing this as a string to the -w flag.

PHP Streams

13th December 2019 - 14 minutes read time

Streams are a way of generalising file, network, compression resources and a few other things in a way that allows them to share a common set of features. I stream is a resource object that has streamable behaviour. It can be read or written to in a linear fashion, but not necessarily from the beginning of the stream.

Streams have been available in PHP for quite a while (at least since version 4.3.0) and are used pretty transparently by most PHP programmers. They can be used to access files, network resources, command line arguments, pretty much anything that goes through the input/output stream in PHP.

I was recently looking at ReactPHP and found that the use of streams was a requirement in order to prevent blocking the input/output stream. Although, I had seen streams being used in PHP applications, I wasn't entirely certain how to use them myself. As a result I thought I'd put together a post about them.

Some Useful Curl Snippets

14th July 2015 - 11 minutes read time

Curl is an incredibly useful tool and has all sorts of flags and options available for every situation. I tend to use curl quite a lot for all kinds of stuff, and not just downloading large files. So I thought I would post a few of the most common things that I use the tool for. Note that most of the following URLs don't really exist, they are just for demo purposes. I have also left out the output of these commands as they vary from a few lines to many pages of output.

Pretty Print JSON With Python

23rd December 2014 - 5 minutes read time

JSON is a very common data format, but reading it can be a little difficult, especially if the JSON contains very little white space. If you have Python 2.6 or above you have use the json.tool to format the JSON so that you can read it correctly. This is also a good way to validate JSON strings that you have had to hand edit before they cause errors upstream.

If you have a file called file.json, which contains a bunch of JSON output, you can use Python to format this into a readable structure in the following way.

python -m json.tool file.json

You can also write this output directly into a file, in this case called formatted.json.

python -m json.tool file.json > formatted.json

The most common use of this tool is to pipe JSON to it from the command line (either directly or via a script).

Netscape HTTP Cooke File Parser In PHP

30th June 2011 - 6 minutes read time

I recently needed to create a function that would read and extract cookies from a Netscape HTTP cookie file. This file is generated by PHP when it runs CURL (with the appropriate options enabled) and can be used in subsequent CURL calls. This file can be read to see what cookies where created after CURL has finished running. As an example, this is the sort of file that might be created during a typical CURL call.

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

www.example.com        FALSE        /        FALSE        1338534278        cookiename        value

The first few lines are comments and can therefore be ignored. The cookie data consists of the following items (in the order they appear in the file.