Om Malik writes about the pitfalls of putting too much stake in Alexa. I’ve been meaning to write a post about Alexa for some time, as I’ve been collecting tidbits of data that point to the relative inaccuracies in Alexa rankings. I don’t have so much of an issue with their uptime, other than not being able to get my Alexa-fix (despite my skepticism in the accuracy of their rankings, I’m still an Alexaholic); it’s entirely possible that although the website is down, there are seperate servers to collect the toolbar data. My issue is with the lack of a correlation between rankings and site traffic and the ease of gaming your ranking.
Evidence and Data Points
A data point: Blogdigger, a site which I own and operate, currently has a three-month Alexa ranking of 16,801. If you check our graphs, you’ll see that this number fluctuates on daily basis, sometimes by up to 10K places (in both directions). I can tell you that we get around 100K page views a day. From what I have seen, there is ZERO correlation between a good day in terms of our traffic and a good day according to Alexa. Further proof of this comes from Otis Gospodnetic’s post comparing the Alexa stats to the Sitemeter stats for Lifehacker and Techcrunch, showing that there is little correlation between actual site traffic and Alexa rankings.
Otis’ post brought to my attention another astounding data point. Blogdigger gets more page views per day than TechCrunch. I’m not going to claim that we’re more popular or influential, but the fact of the matter is, in terms of raw numbers, we have more traffic, or, at least, similar levels of traffic. Yet the difference in Alexa stats is astounding: TechCrunch’s three-month average stands at 605. Now, I realize that Alexa traffic is more than just raw page views, it includes data about number of users and number of page views (although TechCrunch’s average page views per user is consistently lower than ours as well). What this points to is a severe skew in terms of the distribution of the Alexa toolbar, or possible signs of gaming (I don’t mean to suggest that the TechCrunch folks are gaming Alexa, I’m sure they have better things to do, but it is possible that there are automated nets of Alexa gaming bots that end up at TechCrunch more often than not because they are a highly linked-to site). This also shows that the relative difference between 16,000 and 600 is not that much, whereas the difference between position 600 and position 6 is logaritmically greater.
There’s many examples of this, too many to go into here, but, suffice it to say, one needs to consider other information beyond the Alexa rankings.
Alternatives
So what are our alternatives? There’s a new site called Compete.com, which takes a similar toolbar approach. While their stats seem to make sense in a lot of places, in many others they just don’t, and ultimately, if Compete.com becomes popular, it will suffer from the same pitfalls as Alexa does. There are other companies, like Hitwise and comScore, which take a Nielson-like approach (which is better than letting the userbase that you cull data from self-select itself by installing a toolbar), but, just as the Nielsen ratings need to be taken with a grain of salt (when Nielsen tells you 14 million people watched Heroes last Monday, they are extrapolating this number from, as far as I know, less than 5000 individual households, and sometimes even less than that), so do these approaches. They are also not publicly available, and cost quite a bit of money for a subscription.
Om suggests the big toolbar players, such as Yahoo and Google, begin to collect this information. Combined with other information these companies have (such as search results page click-thrus, stats-tracking applications and advertising information), they would have a better, more complete picture of web traffic (since the user base is broader). Even better, and I realize the tremendous privacy concerns here, so keep in mind that this is a theoretical proposition, would be to have the browsers themselves cull this information on an anonymous, aggregate basis. It’s unrealistic to ask the site operators to contribute this information, and would create the potential for fraud.
One last note, there’s a few hack you can use to estimate a site’s traffic when no information other than Alexa is available. Obviously you should look for a stats tracking link, such as Sitemeter; if it’s publicly accessible, it can provide good information. Another way is to visit the site, and see what kinds of third-party advertising they are using. If they use something like AdSense or AdBrite, finding the pages to signup for advertising on their sites may provide details on their traffic. This information can provide additional information for estimating a sites overall traffic.