I finished reading a recent article by Terry Chay regarding Ruby on Rails and PHP. Aside from the fact that he is comparing a framework with a language, I think his rant misses a few things. He was arguing that PHP is more scalable, and that sites built on the RoR framework seem to die under heavy load. He used the example of Twitter by comparing statistics from Alexa.com. This is where I began to get skeptical (remember, I am a PHP programmer and love the language (and frameworks such as CakePHP)—but I also work with RoR). I have never seen Alexa as a valid source for statistics. After all, I am a Mac user and don’t use IE, so apparently I don’t count. A large majority of people I know that use Twitter are on Macs as well—they don’t count either. How about those users who don’t have the toolbar installed? I digress. I asked these same questions in a recent post by Chris Shiflett in regards to Twitter and its slow speed:

Does Alexa track any of the following?

  • RSS Feeds. Twitter makes heavy use of RSS feeds to keep users up to date. Now users can check their statuses from their favorite feed reader—never having to access the website directly.
  • IM services. Again, Twitter makes heavy use of IM so people don’t even have to visit the website to get their fix.
  • API services. There are many people out there making use of the Twitter API on their personal websites. Some are caching the results—others are giving the load to the Twitter servers. This is not an explicit visit to the Twitter website (so far as Alexa is concerned).
  • SMS. Twitter allows you to send and receive updates via your cellphone.

This is not in support of Twitter or Rails, it’s just questioning the validity of the attack by use of Alexa statistics. Unless Alexa tracks any of the above, then I think it is a poor comparison to say Twitter is low on traffic and slow. The speed of Twitter is a whole other discussion that has been covered many other places, this is not to re-hash those discussions. Also, this is not to take a stance against Terry Chay and his article. And finally, this is not a stance of PHP versus RoR. There—all of my disclaimers are out of the way.

So as we move to more (semi-transparent) services on the web, is Alexa going to be able to hold much water with it’s traffic statistics? Personally I think it misses way too much behind the scenes.

10 Comments Add your comment

  1. terry chay May 27th, 2007

    I’ve address the framework vs. language defense before. I don’t know why others might think that programmer of my level doesn’t know the difference between the two. How would you feel if I nitpicked this sentence: “I am a PHP programmer and love the language—but I also work with RoR.” If languages and the frameworks they inspire are so different then that sentence would read something akin to: “I drink orange juice and love oranges—but I also like to program with Apple Macintoshes.” But it doesn’t and neither does my argument. Similarly, when I piss on J2EE, I never notice the Java people invoking the “comparing framework with a language” defense. They’d rather debate me on the merits of my points.

    I used Alexa simply because the data is available and free. Other statistics will also have flawed methodology. You can quibble over that if you like, but you’re throwing out the baby with the bathwater.

    Feel free to use any statistic you like, when the differences are this large the results will be the same. We’re not quibbling over a few percent, we’re taking about several orders of magnitude here. You’ll find nobody at 37signals who’ll will put up their internal traffic data against Twitter’s. You’ll find nobody at Twitter who will put their internal traffic data against a Tagged or Facebook.

    I had a number of points which I outlined here, but the basic idea wasn’t to really piss on Ruby as a language or Rails as a framework. Rather it was to state some rather unoriginal truths which many others have, but can’t say because it is politically incorrect, and hopefully do it with a little good nature.

    Shorter terry: There are serious flaws with Alexa that are worth pointing out. But using my article as a context is probably not the smartest approach.

  2. Nate Klaiber May 27th, 2007

    RE: Terry

    <p>I&#8217;ve address the framework vs. language defense before. I don&#8217;t know why others might think that programmer of my level doesn&#8217;t know the difference between the two. How would you feel if I nitpicked this sentence: &#8220;I am a PHP programmer and love the language—but I also work with RoR.&#8221;</p>

    <p>I did not write this post to insult your intelligence or level of expertise. I have been reading your website, and I respect you as a programmer. This post wasn&#8217;t about you knowing the difference between a language and a framework (I edited my above sentence to include CakePHP&#8212;A framework for a framework). I am debating you on merits of your points. You used Alexa as your foundation of traffic to the websites, and I simply think that is a poor use of statistics.</p>
    <p>I used Alexa simply because the data is available and free. Other statistics will also have flawed methodology. You can quibble over that if you like, but you&#8217;re throwing out the baby with the bathwater.</p>

    <p>I am not quibbling over that one bit. I agree with your statement 100%. Most analytics programs that embed javascript files for tracking share the same issues as Alexa (they don&#8217;t track what is going on in the background and require technology to be turned on or available). They will have a flawed methodology, agreed. So how can they be considered a strong foundation for an argument? You were comparing sites that aren&#8217;t on a level playing ground. If you were comparing two sites that equally used tools like RSS, IM, SMS, and API&#8217;s then it might have made a little more sense&#8212;but the comparison you were using didn&#8217;t really do much to help prove your point. I understood your point, by the way, I just felt that the way you were trying to prove yourself left a bit to be desired.</p>
    <p>You&#8217;ll find nobody at Twitter who will put their internal traffic data against a Tagged or Facebook.</p>

    <p>I am not arguing this either. Some choose to keep this private&#8212;it is their data. Is there really a big argument about it? Even if they did, what would it prove? To have someone just rip it apart? Even statistics are not the end-all-be-all of a website. Different sites has different goals. Different target markets. Different demographics. Different needs. Even if you did have the stats of Twitter versus Facebook, what would you be trying to prove? Who has more traffic? They are different sites&#8212;both social networking, yes&#8212;but two different tools and purposes (updating your status is only a small part of Facebook, while it is the entirety of Twitter). Heck, I can update and check my Twitter status from Facebook now. So, Facebook is getting the traffic&#8212;but I am actually using Twitter. There are many more variables that come into play when you are comparing &#8216;traffic envy&#8217;.</p>
    <p>In short: I respect you and your opinions. This article was not an attack on you as a person. This article was not an attack on you as a developer. This article was not about comparing languages with frameworks. This article is not about comparing languages. This article is about questioning the validity of Alexa statistics.</p>

  3. John MacAdam June 8th, 2007

    Now I have a response to my brother Luke when after he sent me this link...

    http://www.alexa.com/site/ds/top_sites?cc=US&amp;ts_mode=country&amp;lang=none

    ahhhhhhhh

    thanks for coming through for me Nate

  4. Bridget Stewart June 13th, 2007

    So as we move to more (semi-transparent) services on the web, is Alexa going to be able to hold much water with it’s traffic statistics? Personally I think it misses way too much behind the scenes.

    Are their statistics generated by sources that adequately track RSS, API, IM, and SMS with regard to site statistics?

  5. Bridget Stewart June 13th, 2007

    Pardon my typo above. I wrote that without having had my morning coffee.

  6. Nate Klaiber June 13th, 2007

    @Bridget
    No problem with the typos. I do the same without coffee all the time :)

    The only true way to track these statistics is directly from your server, or a custom setup to track them. Your access logs will keep track of all connections to your website - but you still have to decipher a few things to understand the quality of the traffic.

    In the past I have used custom scripts to track calls to action and custom services (RSS being one of them). So the server logs keep track of all of the requests to the server, and the custom logging keeps track of specific actions (that could be within pages) and web services that you want to track.

    The problem with Alexa is that its like a guy peeking in a window of a house and trying to guess its inner-workings. However, he can't see into the basement or the attic to see other ways that people are getting into the house. The validity of the statistics are extremely weak.

    Javascript statistics (Google Analytics, Mint) is a step better, as you allow the person in the house to simply monitor. It's like you let them stand in the kitchen and watch people pass through. However, some people can be hidden if they choose to ignore Javascript. The validity of these statistics is much better than Alexa, but still has its holes.

    Your server logs monitor every entry point and keep track. This is the most bulletproof way to track traffic. However, it is up to you as the house owner to make sense of the data it gives you (there are some good statistics packages out there). And alongside of this is custom tracking that you can implement with your programming language of choice (and database of choice).

    Ill be sure to post up some more information on setting these things up.

    For another discussion: Making sure you have one point of entry for all of your pages will help you keep cleaner track of your statistics as a whole.

  7. Mubashar Iqbal June 13th, 2007

    Alexa has never been an accurate way to track the number of visitors to a website.

    It does have some value as a way to compare general audience levels, so if you know one sites traffic and know its Alexa ranking, looking at another sites ranking might allow you to infer its traffic. Yes even that is troubling, but hey what other methods are available.

    The type of site and your target audience can skew your stats greatly. I have a site about television which ranks very poorly in Alexa, but has more traffic than sites about web design that rank very well.

    In short Alexa isn't much use at all ;-)

  8. Nate Klaiber June 17th, 2007

    @Mubashar
    I agree - the short of it is that it really isn't that useful. It has too many loopholes to be considered mildly accurate. Maybe years ago when web services weren't as popular, but now they are invisible to so many entrances to a website.

  9. Patterns July 27th, 2008

    If alexa isnt useful, then what other alternatives are out there?

  10. Nate Klaiber July 27th, 2008

    @Patterns
    That's just it - there is no foolproof alternative out there. Alexa requires a toolbar to track. Competitor sites can't add JavaScript tracking, beacons, packet sniffers, or read directly from your server logs. Without being able to do that - there is no way to really present the full picture of traffic to a site. It just can't and won't happen, mainly for security and privacy reasons. So all Alexa is is one big guesstimate.

    Also, for many of the reasons above related to traffic that Alexa doesn't track. Add to that list the social networking connection, and Alexa gets even worse at showing a true picture of legitimate traffic.

Comments are closed.