Lots of people criticize the reliability of the Alexa toolbar – see the comments on Matt Cutts’ blog or more recently from Om Malik. I would like to offer some qualified support for the Alexa toolbar – with some data to back it up.

At SLI we run the site search for lots of sites that have a wide range of Alexa rankings. Assuming that the number of site searches on a site is roughly proportional to it’s traffic we can use the site search data to test the accuracy of the Alexa data. This graph plots the Alexa rank against the number of site searches performed over a 30 day period for about 50 sites (randomly selected from our customers).

As you can see, there is a strong correlation between the Alexa rank and the number of searches that happen on a site, although there is a reasonable amount of variation.

We use the Alexa rank to get an order of magnitude estimation of the number of searches for a site, and it’s normally pretty good.

The assumption that number of searches performed on a site is proportional to traffic isn’t going to be always right. Some sites lend themselves to searching – particularly larger sites.

Some sites have completely unreliable Alexa rankings – particularly those that attract more (or less) people that have the Alexa tool bar installed. Case in point is the SEMPO web site. As others have pointed out, a higher proportion of search marketers have the Alexa toolbar installed. The SEMPO site has an Alexa ranking of 19521 (today) and had less than 5,000 searches over the last 30 days. This model would predict around 150,000.

I think Alexa is a useful service, providing that you use it carefully. I think there are a lot of people out there who have dismissed it as flawed and may be missing out on some valuable data.

Peter Da Vanzo spotted Google asking users if they found a link useful:

Note: It was only shown after he clicked on the result and then returned to the search results page.

This is similar to our free feedback tool that asks if the search results page was useful. The main difference is they’re asking about one search result rather than all of the results. We provide the feedback tool as one way of measuring search quality.

We normally assume that if someone returns to the search results page after a short time then they didn’t find the link useful and in our Learning Search we put less weight on those clicks. This type of survey could be a way of testing that assumption. I would expect Google would see the proportion of clicks on the Yes button rise as a function of the amount of time they spent at the link.

I came across this interesting research by Lauren Freedman of the e-tailing Group: The Customer is in Control (note: you do need to sign up to access this but it is free).

This poll of 1300 consumers contains lots of useful information for retailers. The most relevant for us is that 83% of those polled considered keyword search very to most important when buying gifts online. This is yet more evidence that if you’re a retailer you just have to have good search.

It’s been a couple of weeks since my last blog. I apologize to my regular readers, both of you.

Last week we announced another customer – Organize-It. Our team has really enjoyed working with Nick. We love having them as a customer.