How Accurate is Alexa?

Alexa (www.alexa.com) is a company owned by Amazon.com which offers various services to web users and developers. Alexa is widely known for their toolbar which includes a search function, website information and a popup blocker.

Alexa's main "point of difference" is their traffic monitoring. Users who have the toolbar installed transmit information about the sites they visit to Alexa, which compiles the results into publicly-viewable traffic statistics. You can visit the Alexa website and see how much traffic a site is getting, how it compares to other sites, etc.

At least that's the theory. The reality is that Alexa's method of monitoring traffic is deeply flawed. The example below illustrates this point. The first graph is taken from the Alexa.com website and shows the overall traffic trend for www.mediacollege.com from July 2003 to June 2004. The second graph is from our own statistics for the same period (see also update below).

Note: The x axis on the Alexa graph indicates a ranking in the top 100,000 sites monitored, so a lower number is a better score. The x axis on the second graph shows the number of unique visitors to MediaCollege.com. Although these graphs represent different things it doesn't matter - it's the trend that we are measuring and comparing.

As you can see, there is a significant disparity. Alexa claims that MediaCollege.com had negligible traffic during this period except during September-October 2003, when it saw a massive surge in traffic and performed quite well. This traffic then disappeared as quickly as it arrived. In their detailed statistics, Alexa showed MediaCollege.com declining further towards the end of the period, at the same time our statistics were showing a very strong increase.

How did Alexa get it so wrong? Here's how...

During September and October and 2003 we ran a test. We installed the Alexa Toolbar on one computer and used this computer to visit www.mediacollege.com once per day. This single daily visit was enough to propel the site from absolute obscurity to a very respectable position. When we removed the toolbar and stopped the test, our Alexa ranking vanished and we plummeted to a position worse than 200,000.

This clearly shows that the Alexa rankings have no statistical validity.

There are also more factors which further decrease the accuracy of these statistics, such as the fact that only certain types of Internet users are likely to install the toolbar in the first place.

Summary

The Alexa website traffic monitoring system is so inaccurate as to be worse than useless - it is very misleading. Perhaps most concerning is that it is trivially easy for website owners to manipulate their own rankings. We recommend that you do not take Alexa seriously.

Update - 2007

To be fair, the accuracy of Alexa's results has improved a little since the test above. As our traffic climbed above 10,000 visitors per day we started noticing a slight improvement. Please note that we emphasize slight — there are still terrible inaccuracies and inexplicable errors in the way Alexa reports our performance.