Managing Website Rescue Projects, From Audit To Onboarding

I have been a web developer for quite a few years, and one constant that is an unfortunate story in the industry is the rescue project. This is a web project that has been mismanaged by an agency or developer in some way so that the site is at a critical point. Either, the site is unable to launch or it has already been launched and suffers from serious issues. Projects in this state are caused by a variety of factors, and can be challenging and difficult to turn around.

Some agencies, quite understandably, won't go near rescue projects as they can be extremely tricky to manage. The client will have already invested money in getting the site in the first place, so spending money to fix the site can be a little painful for them. This is especially the case for small business sites, clubs or charities without any extra funds.

In this article, I hope to show a plan that you can use to onboard problem sites like this to make the projects a little less painful to manage.

Very rarely you can be asked to rescue websites within your own agency. I'll be focusing more on projects created by external developers, but some of the steps I will go through can also be applied to internal projects.

First Step

The first step in any rescue project is to just listen. Often, the client will want to off load what happened to their site, or how much time and resources they put into it. Listen to the client. By listening to them you can start to figure out where the problems lie.

Invite the client to a meeting to discover their current situation. Have someone technical in that meeting so they can take notes on what appears to be wrong with the site so they can ensure that these aspects are covered when you take the project on.

Sometimes the client will know where the problems stem from, and this will help you structure your work so that you are not repeating the mistakes. For example, if the client said that there was a lack of communication with the last agency, ensuring your communication is regular with the client throughout the process will establish trust. Make sure you use the same domain language as the client as you will find that breaks down communication barriers.

During this process it is very important to ask questions, you need to discover the original intention of the site was. Find out explicitly why the client is not happy.

Finally, you can give some guidance and advice on how to fix their most critical issues. You can give a very rough estimate of how long it will take to fix things, but don’t commit to anything until you’ve done an audit.

Auditing The Site

Before you commit to doing any work on the site you should perform an audit to see what the site looks like. This should be a separate item of paid work and you should produce a report from this audit.

Auditing a site can be time consuming and complex, but try to time-box this into two or three days. You will probably see the major problems in the first day so anything after this has diminishing returns. The audit process isn't looking at solutions, you can talk about best practice and what the errors are, but don't try to debug the more complex problems. Don't try to find absolutely everything, the purpose of the audit is to find the most important issues so that you know what the project looks like before working on it.

There are a number of steps you can take during the audit process.

1. Getting Access

Your first task is to gain access to the site. Sometimes this can be tricky, but you need to have access to one or more of the following before starting out on the audit.

• Git/code access - You will need this to look at the code. Not just in terms of the code written, but any versions of dependencies need to be checked. It's often a red flag when the source code of the site is not stored in a source control system. You might be given a simple zip file containing the code, but you should make sure that the site code itself is governed by source control.
• Database - A copy of the database is a good idea as the code and database often go hand in hand. You will want to run the site somewhere to look closely at what is going on. This can be anonymised by removing any personal information as that is not needed for the audit.
• Access to the site - Administration access to site is preferred, but if not then a high level administrator account is a good idea. You will generally not be able to view the content structure of the site without this.
• Server access - This isn't a requirement, but access to the server (or access to the hosting dashboard if this is a PaaS solution) can provide lots of good information about the setup of the systems involved. A development environment is fine for this purpose, providing that the server has the same setup as the production environment.

Whilst you can make do without one or more of these components, your audit may be missing key information without it. If you don't get access to something then you need to make the client aware that you may not discover everything about the site.

2. Investigation

Once you have the site you should look at setting things up locally, see what everything looks like, spot any errors or problems. If you have problems setting up the site due to errors then these should form part of your audit report. Rather than take notes and write them up later you should just add your findings into the final audit document and then tidy up the report later.

This investigation should be structured in such a way that you find out about the site and then look at where problems might come from. The information you gather here might form part of the audit, but can be included in an appendix in the document. For example, you might generate a list of how many users or pages the site has, and whilst this is useful information, it might not actually be a problem.

Keep in mind the issues described to you by the client and focus in on those problems to see if you can spot anything.

Another tip is not to only focus on the negative aspects of the site. If you spot something that is good then add this to the audit . Having a few points that state "this is done well" can make a rescue site look less like a disaster. It also means giving your client some confidence the site can be rescued.

Potential avenues of investigation:

• Server setup - What web server is used to host the site? How is it configured? Look for things like testing files or database dumps left in the publicly available web root and other careless mistakes.
• APIs - Many sites push or pull data from an API. Make a note of what these APIs are and how they are used. Look at the authentication of these services and how performant they are. Keep in mind the resiliency of the site without access to the API, watch for disruptions caused by the API being unavailable.
• Installed modules or dependencies - See what kind of dependencies the site has and what sort of modules or plugins are installed. This is important for systems like Drupal and WordPress as some issues may come from third party code.
• Out of date dependencies - As well as looking at the dependencies installed, make sure you look at the current versions of them. Out of date modules can contain bugs or be unsupported and will probably need to be migrated.
• Content structure - CMS sites will have a structure of content. Make sure you look at this structure to see how the site is put together. For example, Drupal sites can set up with content types and taxonomy terms in a poor way, causing frustrations for users trying to edit content.
• Status screens - If it's a CMS it may have a status screen. Many CMS sites will show obvious problems on this status screen so you should take note of them.
• Custom code - Many of the problems will stem from custom code. I would suggest you not only look at the code using a static analysis tool but take time to simply read through it. If the site is written in PHP then there are lots of tools you can use to scan the codebase like phpmd or CodeSnifer. Looking for simple coding standard problems or functions with high cyclomatic complexity can lead you towards the problem areas of the codebase.
• Security problems - Make sure you look out for any security problems. Out of date dependencies can often be the cause of security problems, but may also be from custom code so make sure you audit with that in mind.
• Configuration files - Some CMS sites come with configuration files. For example, Drupal has a configuration management system built in and getting this wrong can lead to really obvious problems in the structure of the configuration files. Some systems will keep this configuration in a database, so check the structure of that configuration as well.
• Database best practice - Looking at the structure of the database can reveal issues. The lack of field indexes can lead to large performance problems and is a good avenue for improvement.
• Site Performance - Use tools to measure how fast the site is. Something as simple as Google's Lighthouse tool can provide lots of useful information on the speed of the site. You can also use tools like XHprof or New Relic to profile the site performance. Keep an eye out for slow page loads, especially when saving content. Try to get a picture on where the performance problems are coming from.
• SEO - Spend some time checking the site for best practice SEO. Simple things like having a sitemap.xml file or decent content setup can really improve the amount of traffic coming to the site.
• Accessibility - If you ran Google's Lighthouse on the site then you will have seen any accessibility problems. Many countries have a legal requirement for sites to be accessible so making sure the site using best practice here is important.
• Tests - Does the site have an active testing suite? Can you run the tests without error? The presence of a testing framework is a good sign, but if it has been left behind and doesn't run then it is next to useless.

I should note here that if you do find any serious security problems on the site you should tell the client straight away. Problems like downloadable databases in the web root, SQL injection errors, or user data being leaked are very serious concerns. Even before you start writing the audit itself you need to disclose these issues as soon as possible. Doing so will build on the trust between you and the client and show that you know what you are doing.

You should try to automate some of this process as well. Things like accessibility checks or CMS best practice tests are often easy to automate and doing so will give you more time to concentrate on the important things.

Once you have finished your investigation you will probably have discovered a number of things. It is now time to write the audit.

3. Writing The Audit

From the investigation you will have produced a document filled with problems and any other findings you have noted down. You now need to flesh out each finding to explain what you have found and why it is a problem. When writing about the problems, also write about potential solutions to those problems and how the site would benefit.

When I have the audit written out I will then traffic light each issue.

• Critical or security findings in RED.
• Actionable, non-critical findings in AMBER.
• Satisfactory or cautionary audit findings in GREEN.

Collecting these items together is a good idea as well. This means that the document will have the most critical issues at the top, followed by the rest of the issues found. Having this traffic light system in place allows you to show the seriousness of each problem. This is very useful when going back to the client with the final document.

Remember that things that are fine can go into the audit as green items, meaning that they don't need anything done to correct them. Don't be too fine grained with the green items though. For example, if the site passes your SEO checks then write that as a single item saying what you checked.

The finished audit document should be professionally written in an unbiased, non-technical manner. State facts and write about best practice approaches to solving the issues found. Your audit document will be read by a number of different people, and not just those working on the site. Management will read the audit document at some point so having it professionally written in an approachable manner will benefit everyone involved.

As an example, I once wrote an audit on a site that was then used to take a previous agency to court over the non-delivery of work agreed in the contract. The fact that I had written the document in an unbiased manner allowed it to be used in this way. Incidentally, you might also be asked to remove your company branding from the document, so be prepared to create a white labelled audit document.

Instead of handing over the audit, and letting the client read it on their own, I have always found it beneficial to talk the client through your findings. A couple of hours going through the audit findings really helps everyone involved understand what you have found and how serious those problems are.

Once you have written a few audit documents you will probably have a few items that are commonly found on sites. Keeping a  copy of your old audit documents can help you build up a bank of findings that you can pull into your new audit documents.

From the audit you can start to create a quote or schedule of work estimating how long it will take to correct each error.

What Happened?

At this point you should have a pretty good idea about what happened in the project. Which means you can start seeing where the problems arose. This will either be the client or the developer who built the site.

Let's look at some of the reasons why projects end up in this state.

1. Problems caused by the developer

• Lack coding abilities will often cause lots of problems on the site. Issues like bugs, errors, accessibility issues, or poor performance problems stem from inability to write code correctly.
• No knowledge of the framework the site was built in. This is especially the case for sites built in WordPress or Drupal where code hacks and bodged code is often the norm. inexperienced developers can also configure the site incorrectly so creating content is difficult. If the developer has configured themselves into a mess they tend to never take time to fix that mistake. This is especially the case in Drupal where changing the configuration of fields can be a little complex.
• Making mistakes. Some problems were caused by a one off event that still manifests itself in the current site. This is quite likely to be a migration, but can be due to other factors.
• Not understanding the brief. Even with the best will in the world, sometimes the project fails because the developer delivered something that the client didn't want. Processed like agile can help to prevent this happening, but this isn't a guarantee.
• Lying to the client. This will be a case where the project is going very wrong, but the developer will not acknowledge this to the client.

2. Problems caused by the client

• Design by committee. If lots of people are in charge of creating the website you tend to find the focus being pulled in lots of directions. This is usually a bad thing as it creates a messy and unfocused result that no-one is happy with.
• Unable to make decisions. I have worked with clients that are unable to make decisions which leads to time consumption, meaning important parts are being a left until the end and rushed. Clients can also have a slow response time to questions and finalising approvals.
• No focus or goals. Some clients will want a website just to have a website, whereas others will have a massive collection of items that they class as MVP. A lack of focus in the initial project or a key goal which they are trying to achieve can mean a lack of decent foundations for future development. Having a large amount of items as the MVP means that the budget is stretched which can mean that nothing is implemented correctly.
• No knowledge of how the site works. A good client to work with is often engaged with the developers and has a decent knowledge of how the site is put together. A client who has no idea how to use their own site is often not connected to the project and so it has developed into an unfocused state.

Onboarding The Site

Once you have created the audit that might be the end of your interaction with the client. There is no guarantee that after the audit process you will then go on to fix the problems you find. This is why you should charge the creation of the audit as a separate item of work.

If you are going to take on the project then you will need to think about moving the site from the current host. Many sites are hosted in house by the agency that built them and so it might be necessary to migrate the site to another platform. If the site is hosted on a PaaS platform then it's just a case of changing ownership on the account.

Through your investigations of the site, and the server it is hosted on, you will know the versions of various servers being used so you can use this as the basis of configuring the new platform. If an out of date version of the hosting environment was causing problems then you can fix that issue as you prepare for the migration.

As you have made a note of the API integrations used on the site you can make sure that they are still able to be used on the new environment. Some API's require VPN services or IP whitelisting and so moving servers will cause these services to fail if not accounted for.

You should ask up front if moving away from the current host is needed. If the site is hosted with the client or on a third party hosting service then you can probably keep the site where it is. It's generally safer in the long run to keep things where they are, assuming the server isn't the cause of the problems.

In summary, ask the following questions:

• Where is the site hosted?
• Can it be moved?
• Does it need to be moved?
• Are automated deployments being used?
• How reliable are deployments?
• Are any server specific API security measures in place?

Once you have full control over the site then you can start to plan you next actions.

Plan Your Next Actions

The audit document is essentially your plan, so you should use it to make a plan of action. I normally take each red and amber finding from the audit and create tickets for each. This gives you a base that you can use to undertake the work needed to turn the site around. The traffic light system will feed directly into the priority of these tickets so any red item can be created as high or critical.

Once the tickets have been created then go through each and add an estimate to them. Estimates should not be done during the audit, but you should be aware of how much time each item will take to solve as you find them in the system. You absolutely need to know how long each item will take before you start work on the site. As a general rule, if you have a task that goes over two days then look at splitting that into sub tickets. Some tickets might need some investigative work before estimation can be made properly.

With the estimates in place you need to make a decision about the project. If the site is a total disaster then it might be a good idea to just start again rather than attempt to fix the problems found. I haven't seen a site in such a poor state that it couldn't be turned around, but there have been a couple of close calls. It's at least worth talking over the situation with the client.

If, during your audit process or whilst talking to the client, you found problems with the deployment of the site then you should look into that as well. There is no point in fixing aspects of the code only to be unable to deploy them correctly.

Sometimes you will find that the client will come to you with a list of outstanding bugs from their previous agency. These should be incorporated into your tickets and worked on along side your audit tickets. Whilst you can look at implementing new features, this should generally be avoided until you have no critical findings left over from the audit. It is also absolutely essential that you fix any security problems first. The client really doesn't want a hacked site to add to their problems.

Your goals for sorting tickets should be to solve any errors and bugs, simplify complexity, and improve the user experience. While estimating and creating a plan for smaller challenges you also need to include the over arching work that needs to be done.

It's a good idea to create a maintenance agreement with the client so that you can work through the tickets and go beyond that into new work. Some problems can mean many hours of work to correct, a maintenance agreement will help work to be carried out  along side other tickets. Remember that clients can just want their immediate concerns to be worked through, so a maintenance agreement isn't always viable.

Stories from the front line

As I said at the start of this article, I have done this process a few times. I thought, therefore, that I would try to tell a few stories about how these audits have gone in the past. All names have been entirely removed here to protect both parties involved.

1. The Coding Disaster

The client had enough of errors and things not working on their site. They had been working with the current agency for over a year and although the site was finished it often was slow and produced errors at vital points. One of these errors was during the checkout process of the shopping cart system, which was ultimately costing them money. The site also suffered from performance problems and would often fail to load entirely. Users were tired of the errors and nothing ever being fixed.

Looking at the site it was found that it should not have been as complex as it was. The site was created in Drupal and it had way too many content types for what needed to be shown. There was also an issue with creating content on the site as editors needed to create content in several areas in order for it to display correctly. Because of this, removing content was also an issue and needed careful searching to find all the places content existed.

The developers who built the site didn't know Drupal, and that was clear from the amount of bad practice that was going on. The audit found that Drupal core had been altered in several places. The site had lots of custom code, most of which was to do with controlling how the content types worked. Most page requests produced PHP warnings due to missing array elements and large sections of code were commented out or simply inactive.

I was approached to help fix the project and turn it around. Some time was spent clearing up the codebase and fixing some of the more critical bugs. The client brought a list of outstanding issues with them that needed to be fixed first. Once the site was working well I looked at removing some of the extra content types and tweaking the listing pages on the site to streamline the editorial process.

After this, the site was working perfectly and I was involved in the maintenance of the site for quite a while afterwards.

2. Never got started

In this case, the client was looking for a new agency to take on their Drupal site. After I was given access to the administration of the site I found that it was a bit of a mess with clear indications of known problems, but problems that could be easily solved. There were just some obvious poor choices from their previous agency that had left the editor experience a bit of a mess.

The client wanted to integrate with a CRM system, so I would also have been building out that integration. At the time they were still shopping around for the right system, so that wasn't set in stone. An audit of the codebase was still needed before work could begin on onboarding them and starting new features.

Although I was given access to the site I never got access to the codebase or database. After a few months of negotiating with a very draconian and abrasive IT manager we eventually parted ways before the project got started.

This might seem like a strange example to include, but I wanted to add it as sometimes it's best to part ways rather than continue to work with someone who really doesn't trust you.

3. The incomplete project

After many months of development with their current agency the site was still not finished. What was there didn't really work that well. It had numerous errors, bugs and even some major performance problems.

The client approached me to finish the project and turn around the site. This was a member driven Drupal site and they were put in the awkward position of spending all of their membership income to fix the site, despite already having done this to build it in the first place. The members weren't happy.

During the audit it was found that the site had several major problems, mostly coming from custom code. One particularly odd design pattern was used in quite a few places around the site that caused large performance problems. The main function of the site was to run events so there was an event content type held in the system. In order to display one event, all of the events were loaded from the database and compared with an ID to find the relevant event, which was then displayed. With one or two events on the site this didn't cause any problems, but when hundreds of events were present it caused a major problem. Hunting down and sorting this odd design pattern sped the site up considerably.

I worked with the client to fix the major issues found from the audit, which often joined up with their current bug list. We eventually parted ways, but the site was in a working state and they were very grateful for the work done to rectify the issues.

4. Fumbled and mismanaged

A large, international development firm with multiple different offices around the world built the site, but the relationship with the client had broken down. One part of this agency had built the site, but it was now being managed by another part due to the soured relationship with the original build team who had utterly fumbled the site.

The audit of the site showed a history of poor decisions, fumbled implementations and an awful deployment system. It was a Drupal site and some very strange decisions were made in terms of the site structure. Many of the image fields were created as file upload fields, which meant that the site had no image processing in place at all so the client had to upload files of a very specific size.

The deployments were a particular sticking point as they would bring the site down for up to 20 minutes and caused quite a bit of stress and revenue loss for the client. The current work around for the deployment was to "keep flushing caches" during the deployment until the problem fixed itself. Deployments were therefore a very stressful experience for everyone involved.

Looking at the current issues the client brought with them, and the result of the audit, it was clear that the developer had been lying to the client. This was especially apparent in the final months when the developer was being paid to do work but not actually completing it. Items that had apparently been worked on had no evidence of any changes in the site at all.

It took me about two months to turn everything around and sort out some of the bigger issues and fix out the poor deployment system. After this, deployments went from guaranteed downtime to a barely noticeable event.

It wasn't possible for the client to hold off the new features on their site until the audit issues had been sorted out so the critical issues were tackled first and the other issues were fed into new feature sprints. Needless to say, the client was very happy with the result, and I worked with them for the next couple of years on the site.

5. The hot potato

This Drupal based site had been built a number of years ago and had been passed around from lots of agencies and contractors.

An audit of the site showed a litany of half completed work, poor implementations and mis-configured Drupal work. Items of note were the five different content types used to display news items, hard coded IDs of pages used in numerous places in custom code, and perhaps the most frustrating custom Drupal theme I have ever worked on. There was even a half finished ticketing system, built using Drupal components, that had never been used.

It soon became clear why the site had changed hands so much. The client was unable to make any decisions, or stick to them whatsoever. Actions agreed to in one meeting would suddenly be unacceptable a day later, meaning that developers were constantly changing focus and never completing work.

The client had big plans for the site, but simply refused to solve the groundwork and fix the problems. This meant that new features were often compromised by the poor code on the site, and therefore took a long time to finish. After much work to solve some of the major issues there were still many problems with the site. Many of these problems were caused by the terrible theme that the client refused to fix, but would often make development much longer than it should have.

We eventually parted ways and there was no love lost when that project came to and end.

Conclusion

As I mentioned at the start, rescue projects can be stressful and time consuming. It's quite hard to pick up a fumbled project and take ownership of it.

By following the above process you can help the client understand how bad their site is and how they can go about fixing it. Even if you don't go on to fix the findings in the audit, you will have created a structured document of where the client can improve their site and move forward.

Have you had experience with rescue projects yourself? Tell us your stories below. Please don't name any agency or site as these will not be published.