Timing web requests is possible in curl using the -w or --write-out flag. This flag takes a number of different options, including several time based options.
These timing options are useful for testing the raw speed of requests from a web server and can be an important tool when improving performance and quickly getting feedback on the response.
The -w or --write-out flag in curl has a number of different options, far more than I can add here. You can use these options by surrounding them in a "%{parameter}" structure and passing this as a string to the -w flag.
For example, if we wanted to return just the HTTP status code of the response we would use the following curl command.
curl -w "%{http_code}" -o /dev/null -sL https://www.hashbangcode.com
This returns the string "200" and nothing else.
To explain the parameters used above:
- -w is the write out flag and we have passed in the http_code option, which means that the output will contain this information.
- -o will send the output of the response to /dev/null. In other words we are just throwing this away.
- -sL this is a dual flag, with the "s" running curl in silent mode without any error messages or progress bars and the "L" will let curl following any redirects so we are actually measuring the final destination and not the initial response.
More importantly for the purposes of measuring the time taken for the response to complete are the time parameters available to the write out flag.
To reference the documentation for the time based variables is as follows.
- time_appconnect - The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed.
- time_connect - The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
- time_namelookup - The time, in seconds, it took from the start until the name resolving was completed.
- time_pretransfer - The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the particular protocol(s) involved.
- time_redirect - The time, in seconds, it took for all redirection steps including name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete execution time for multiple redirections.
- time_starttransfer - The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the result.
- time_total - The total time, in seconds, that the full operation lasted.
Any of these variables can be added to the request to get an indication of how long a request took to run.
curl -w "%{time_total}" -o /dev/null -sL http://www.hashbangcode.com
This will return a single value of how long the request took to complete, in seconds.
You can combine these variables together in order to create more detailed information about the request.
curl -w "Connect time: %{time_connect}\nTime to first byte: %{time_starttransfer}\nTotal: %{time_total}\n" -o /dev/null -sL http://www.hashbangcode.com
The time_starttransfer option gives us access to the "time to first byte" metric, which the time it takes between the request being received and the server sending a response. An important consideration when benchmarking server response times.
Running this command produces the following output.
$ curl -w "Connect time: %{time_connect}\nTime to first byte: %{time_starttransfer}\nTotal: %{time_total}\n" -o /dev/null -sL http://www.hashbangcode.com
Connect time: 0.033286
Time to first byte: 0.147526
Total: 0.157203
This does, however, become a little unwieldy as most of the command is now taken up with the -w argument.
The good news is that instead of supplying a string to this flag we can create a template file and supply the filename to the flag using the @ symbol. This is called a readFile macro, and will inject the contents of the file into the flag arguments.
To recreate the above we can create a template file called "curl-firstbyte.txt" and add the following contents.
Connect time: %{time_connect}\n
Time to first byte: %{time_starttransfer}\n
Total: %{time_total}
The .txt file extension here just makes the file easy to edit; it isn't actually required.
We can then change the command to reference the file using the readFile macro syntax. This means that the curl command is simplified to the following.
$ curl -w "@curl-firstbyte.txt" -o /dev/null -sL http://www.hashbangcode.com
This works in exactly the same way and is much easier to type into the command line.
For completeness, we can create a template file that contains all of the available time based parameters in one go. Create a file called curl-timings.txt and add the following content to it.
time_appconnect: %{time_appconnect}\n
time_connect: %{time_connect}\n
time_namelookup: %{time_namelookup}\n
time_pretransfer: %{time_pretransfer}\n
time_redirect: %{time_redirect}\n
time_starttransfer: %{time_starttransfer}\n
time_total: %{time_total}\n
And then reference this template file in the same way as before.
curl -w "@curl-timings.txt" -o /dev/null -sSLIXGET https://www.hashbangcode.com
Running this command produces the following output.
curl -w "@curl-timings.txt" -o /dev/null -sL http://www.hashbangcode.com/
time_appconnect: 0.061876
time_connect: 0.033015
time_namelookup: 0.002912
time_pretransfer: 0.080220
time_redirect: 0.045696
time_starttransfer: 0.149438
time_total: 0.158410
This makes it clear that most of the time taken is the web server responding to the request.
Make sure you check out the curl man page on the write out flag for a detailed breakdown of every option that you can pass to this template.
There are better tools out there to test performance of your site, but this technique can be a useful tool for getting quick feedback on how much you have changed the performance of a single request and nicely complements full stack testing solutions.
Add new comment