31 Jan 2013

6 Protips For Advanced SEO Auditors

SEO Audit GraphFor me personally SEO audits have always been a challenging and rewarding process. I can’t think of any other single task that has taught me so much about SEO, and web development. Almost, with every audit I learn something new. Sometimes it’s about SEO, but lately its been about how to improve my own audit process. So with that said, here’s a few things I have learned that have improved my audit process.

Create a Work Space

You will see in a moment that many of my tips revolve around having custom tools and specific configurations to run successful audits. When I did SEO audits on an occasional basis, I would integrate these tools and configurations into my daily environment. However, now that I do audits much more frequently I have setup a separate user account in Windows 7 called “Audits” just for this purpose. If you want to be a real hardcore auditor, like Bonnie, you might even install VirtualBox so you can have a completely dedicated operating system for analysis.

Develop a Workflow

No two sites are the same, which means that no two audits will be the same. Some would even argue that you should approach each audit differently. I think having a controlled workflow that allows for flexibility is the key. For me this means that I start with a comprehensive “checklist” of caveats to look out for. Then as I make notes, I make a separate list of things I notice that aren’t on my check list. This second list is usually the most interesting, because it usually contains things I wasn’t anticipating. You should develop your own list of things you are comfortable with, but, a really good place to start is this amazing list [GoogleDoc] by Annie Cushing.

Use a VPN

About 6 months ago I was doing an audit for a site with around 5,000 pages. To begin I fired up ScreamingFrog and started crawling URLs like crazy. After doing a bit of the audit work I decided to take a break and watch this horse over at YouTube (don’t judge, it relaxes me). However, I was disappointed to find out that YouTube was blocking my IP address entirely. I couldn’t even visit the home page. How did that happen you ask? Well apparently out of the 5,000 pages that I had crawled, around 1,500 of them linked to YouTube pages. Which means I made around 1,500 HTTP request to YouTube’s servers in under a few minutes. Any smart server admin will setup automated blocks that look out for this type of automated behavior. It’s not just YouTube either, many sites will link out to social media sites on all their pages. The best way to protect your IP’s from this unexpected ban is to use some type of proxy or VPN service.

Minimize Browser Addons

Make a list of absolutely essential browser plugins that you need to run your audits, and then get rid of the others. Many browser plugins will use external JavaScript and CSS to work correctly. And some even make API request with every page load. This can be extremely confusing when checking things like header response codes, and native CSS.

Develop Your Own Tools

What makes a pretty good SEO even better? Custom tools! Here at IMN we have a lot of great tools, but those don’t even scratch the surface of the private tools that we use internally for clients. Having tools that can do pretty much exactly what you need is an awesome thing! And the beauty is there is SO much data out there to play with! Don’t be afraid to start developing your own tools. Even if you don’t know how to program, there are a lot of folks that do, just waiting for your call!

Master Your Database

A lot of folks will use Excel to process their data sets. For me, I am more of an Open Source fanboy. So I rely on MySQL. I have an install of WAMP on my “Audits” user account with a version of PHP and MySQL installed. When I have a large data set that the typical spreadsheet app can’t handle I throw the data into a MySQL table, and go to town! Having a clear understanding of the SQL syntax can help you do things with your data that just isn’t possible with Excel. And by brushing up on your database skills you are mastering a new skill that should further your career down the road!

There are many other things to consider when doing advanced auditing, but these are just a few. If you have other tips that you would like to share, let us know in the comments! Until next time, happy auditing!


  1. Matt Mikulla January 31, 2013 at 3:45 PM

    Hardcore Joe.

    One immediate tip I’m taking from this post is the use of a VPN or proxy when crawling.

    Something I’ve never thought about but could have implications on a shared network.

  2. January 31, 2013 at 7:41 PM

    Your point about browser addons is interesting…I’ve gone through phrases where I was really hardcore obsessed with browser plugins – my browser looked like a plane cockpit….in the end I ended up only really using like three of them….lesson learned through pain and through very slow browsing experience 🙂

  3. January 31, 2013 at 9:58 PM

    great stuff Joe,

    I use my own crawler and had the ban problem. It would also get banned from the site under analysis. My solution was to slow it down (to a crawl) and run it overnight. The down side is some of my larger clients (20k+ files) can take days to crawl!

    Instead of a second account or virtual box I VPN into a second computer that does a lot of this crunching for me. It also means I can be in two places at the same time (via VPNs).

    Throwing my data into a database is on my todo list. Mainly so I can store and index lots of historical data.

  4. Mike S. February 10, 2013 at 4:57 AM

    It’s a bit unfair to compare Excel and MySQL as options to store your links. One is a generic tool, the other is a structured storage. Not many people can take advantage of MySQL, they have to have the right skills, know SQL, etc. Whereas Excel is for the layman.

Leave a Reply