Wednesday, November 26, 2008

Search Engine Optimization (SEO): Science, Art or just Guesswork?

The 1 family. friendly. food. traffic from search engines was 12 visits in July, 22 in August, 45 in September, and 303 in October. My estimate for November 2008 was 485 visits coming from search engines. In that post I wrote "In November, I expect a 60% increase in search engine traffic due to more posts (and therefore more hits), better rankings, and adding the blog to Live Search and Yahoo! Search."

With 4 days left till the end of November, we are trending to have... drum roll... an amazing 102 visits. Yes, that's not a typo - it is a third of the October value, and less than a quarter of my November estimate. On the bright side, it is still more than July, August and September combined. Should I be worried? Glancing through the first chapter of ProBlogger's book, I've seen that he had a similar issue when he got started.

I have spent a lot of time this month, tracking the results almost daily, reading about search engine optimization (SEO) and trying to figure out what is going on.

My main conclusion so far is that while SEO consultants try to paint SEO as an art more than a science (and in the process ask for a lot of money to optimize your site for search engines), it is still more guesswork than art. It is probably were medicine was 150 years ago. We know of some rough correlations between certain actions and how it might impact search engine traffic, but it is not yet possible to accurately predict the results.

I think that there is a huge business potential here (I'm giving it away for free, but will be happy to talk if someone is interested). The difficulty of accurately predicting results stems from the fact that many factors are involved:
  • The overall search traffic might be down because of various factors such as the current recession.
  • The traffic for the specific web site type or industry might change seasonally or because of other factors unrelated to my specific site.
  • The number and quality of incoming links to a site affect its ranking.
  • The changing site structure affects the search engine's ability to effectively crawl and index it.
  • The changing site content will affect how various keywords are deemed relevant by the search engines.
  • Periodical changes in search engines' algorithms will impact the ranking as well.
  • The title and description of the site's pages will impact the likeliness of users to click on the page in the search results.
  • Paid ads to the site are of course affected by the marketing campaign budget and settings.
  • And the list goes one and on...

The 3 major search engines provide very similar tools to web masters to promote their web sites. They have some web master tools allowing some basic data (e.g. is the site in the index and when it was last crawled), but very little reasons. They have analytics which track usage of the site, but again no tools to understand why that is the traffic or to model who it will change. The best tools are related to ads (which makes some sense since that is where they get all the money from) which allow some better modeling (e.g. some historic data about keywords) but it is limited to paid ads and not good enough in my opinion.

Business potential: I think that if one of the 3 bug search engines (or some third party) would create a tool allowing people to model how various changes will impact their search traffic, it would be a huge differentiator (and will allow them to sell more ads indirectly). All the data I have listed in the bullets above exists or can be gathered by the search engines. This is data that is currently underutilized. Probably in 5-10 years we'll get there. Too bad...

So what happened?

Anyway - enough with day dreaming. Let's see what actually happened to 1 family. friendly. food. First, the data (click to enlarge, the y axis is hits per week):



We started October with about 8 hits per day, it went up to a peak of about 12 per day towards the end of October. And then it fell back to 2 hits per day by mid November, with a slight increase in the past few days.

The main thing I can attribute this change to is the fact that we got a new domain name at the end of October, and redirected the traffic from the old address (wflavors.blogspot.com) to the new domain (http://www.familyfriendlyfood.com/). My assumption from trying to use the Google web master tools and trying to search for the site in Google is that it took them much less time to figure out that the old address is just a redirect and remove it, then it took them to add the new site. Hopefully the increase in traffic in the past few days will continue and get back to the old levels. One option (that I haven't seen mentioned in the SEO articles I have read) is that their search ranking algorithm takes into account the age of the site (discounting new sites), and will gradually increase the ranking. If that's the case, then it is just a matter of waiting. Note that I did see that the site was crawled starting in late October, so Google knew about it immediately after it went online.

As you can see, there is a lot of guesswork involved (more below). I have done a bunch of other things to improve the search engine traffic. Here is the list:

I have added the site to Live Search and Yahoo! Search in September. We had the first hits from Live Search and Yahoo! Search in Mid November, and 3 more hits since. Still negligible compared to Google Search, and it took Microsoft and Yahoo many months to catch up (the site has been live since July), but it is progress.

I have created several landing pages on the site (outside of the blog) to cover specific topics. The idea was that each such page will talk about a certain subject that is relevant to the site, and thus have a certain concentration of keywords increasing the likelihood to be found relevant by a search to those keywords (and also redirects to relevant pages on this topic). There is an Index page, a page about Weekly Menu Plans, a page about Cookbook Reviews, and a page about Kids and Food and so on. It is still a work in progress. So far it hasn't been successful (only 1 hit to those pages in November), but I have only seen Google pick up and index those pages a few days ago, so we'll see how it will go in November.

I have add a meta description tag to all the above pages. That should help drive traffic and cause people to click on the links if they are in the search results. The first few pages that I have created didn't have this tag, and Google indexed them with its default description. The newer pages which I have created with the description have it in the Google search results (to see all the search results, use this link). It has been almost a month since and Google has not updated the description for the older pages. I hope it will do it soon.

I have also added the blog to a couple of directories (blogged.com and bloggapedia.com) and submitted to a few others, although it is not at all clear if this influenced the search results in a any way (click this link for a list of sites linking to 1 family. friendly. food., the directories don't appear there).

Bottom line - I plan to read and learn more about this topic, and continue to do some (educated) guesses to improve search engine traffic. Hopefully, it will get to a point when it is closer to a science.

No comments: