Nate Klaiber

How to Make Sense of Your Web Traffic

Seth Godin has posted an excellent snippet about measuring your traffic to your website. Website statistics are some of the most skewed things that I have seen — and most of this has to do with the user behind the statistics wanting to ‘bloat’ their traffic. To me, it has always been about quality traffic versus straight hits. I want to see the users who are spending time and interacting with my websites. Ultimately, if someone comes and leaves 3 seconds later — that is not a quality hit. And, though I like the exposure — the point of building sites is to build something that is a useful tool.

I have heard and seen many web developers looking at their Webalizer stats and saying “I received {insert random number here} hits today.” This is from a glance, they don’t take several other things into consideration:

1. Hits, in and of themselves are horrible indications of web traffic and I have never trusted them.

   Example: Say I have a friend (well call him Bill) with a beautiful website. It is constructed of minimal graphics, nice layout and CSS for all controls. Their site is content rich and users spend the time browsing and reading. Bill's Webalizer stats show he has 100 hits in the last hour. Now, I also have a friend (lets call him Ted) who has a website that is based entirely of graphics, external Javascript for rollovers, flash files, audio files, etc. Ted’s Webalizer stats show that he has 3,000 hits in the last hour. Ted takes time out to boast to Bill about his massive amounts of hits.

   The problem: Hits are counted for every hit to your website (duh). The nature of HTTP is stateless, meaning each element of your page will need to be requested from the server. So — Ted’s site has to make a request for each and every element on the page. Each request is a hit. So, one actual page visit accounts for 300 hits in the Webalizer stats for Ted. Bill’s site is optimized using CSS and minimal graphics so it takes less requests to serve up one page (also conserving band-with as traffic increases). One page visit for Bill only accounts for 50 hits (requests).

   The solution: Don’t trust hits as a valid representation of your traffic (or quality).

2. Web crawlers and robots need to be taken into consideration. Yes, there is a long list of these, but do you really count this as valid (quality) traffic? It would better be served to see what people were searching for or why they were crawling your website (there are some you may not want crawling your site). Googlebots, Msnbots, yahooslurps, etc. These all count as hits to your site as they are requesting your page via HTTP requests in the background. Do you account for these as valid traffic (quality) to your website?

3. Lack of a development server. Many people don’t have a development server to work on so they are constantly working on their live website. They work on it from several locations. From the office, from home, from a friend’s house, etc. Now — though you are technically counted as traffic, are you using the site for development purposes or using it for information? Remember — all hits are counted to your site — even from the different IP’s that you are visiting from. Do you count yourself? If you are developing, then these are not quality visits to your website. Compile this with number 2, and your stats will definitely become bloated to look as though you have traffic.

Website stats are very hard to manage and keep a true understanding. Do you know your target? Do you know your demographics? Do you know who is accessing your site and from where? How do you measure quality with your website. This is never an easy task — but with an understanding of how to read your stats and monitor your traffic you can get a clear indication. Here are some tips:

1. Have a good plan in place. Just reading stats and going from there is a weak option. Define things like what you want to measure and how you will do this. This is easier for e-commerce sites as they can place monetary value on their stats. Just as Seth Godin pointed out with the Myspace/Amazon comparison. Myspace may have more traffic — but for what? Nothing is being bought (and if you have ever seen the advertisements on MySpace, chances are they don’t see much from those either). Amazon, on the other hand, has a very strong community and makes a tremendous amount of money from their system. (This is the quality vs. quantity).

2. Have a good metrics system in place. A good start would be the free Google Analytics. This allows you to get a better view (graphical).

3. Though you may not use them daily, know and understand how to read your raw server logs (access, error, agents, custom, etc). Knowing how to read these will help you bring things into perspective, as well as help you clean up your website from old pages or find content that is missing.

4. Understand some of the basic HTTP codes and how HTTP works. Seeing a lot of 404 in your logs? You may want to look to make sure you are linking everything properly or that you have the files on the server. Knowing these codes will also help you as you manage your website and the codes you send the client or bot requesting the page.