Search Engine Optimization (SEO): Science, Art or just Guesswork?

The 1 family. friendly. food. traffic from search engines was 12 visits in July, 22 in August, 45 in September, and 303 in October. My estimate for November 2008 was 485 visits coming from search engines. In that post I wrote "In November, I expect a 60% increase in search engine traffic due to more posts (and therefore more hits), better rankings, and adding the blog to Live Search and Yahoo! Search."

With 4 days left till the end of November, we are trending to have... drum roll... an amazing 102 visits. Yes, that's not a typo - it is a third of the October value, and less than a quarter of my November estimate. On the bright side, it is still more than July, August and September combined. Should I be worried? Glancing through the first chapter of ProBlogger's book, I've seen that he had a similar issue when he got started.

I have spent a lot of time this month, tracking the results almost daily, reading about search engine optimization (SEO) and trying to figure out what is going on.

My main conclusion so far is that while SEO consultants try to paint SEO as an art more than a science (and in the process ask for a lot of money to optimize your site for search engines), it is still more guesswork than art. It is probably were medicine was 150 years ago. We know of some rough correlations between certain actions and how it might impact search engine traffic, but it is not yet possible to accurately predict the results.

I think that there is a huge business potential here (I'm giving it away for free, but will be happy to talk if someone is interested). The difficulty of accurately predicting results stems from the fact that many factors are involved:

• The overall search traffic might be down because of various factors such as the current recession.
• The traffic for the specific web site type or industry might change seasonally or because of other factors unrelated to my specific site.
• The number and quality of incoming links to a site affect its ranking.
• The changing site structure affects the search engine's ability to effectively crawl and index it.
• The changing site content will affect how various keywords are deemed relevant by the search engines.
• Periodical changes in search engines' algorithms will impact the ranking as well.
• The title and description of the site's pages will impact the likeliness of users to click on the page in the search results.
• Paid ads to the site are of course affected by the marketing campaign budget and settings.
• And the list goes one and on...

The 3 major search engines provide very similar tools to web masters to promote their web sites. They have some web master tools allowing some basic data (e.g. is the site in the index and when it was last crawled), but very little reasons. They have analytics which track usage of the site, but again no tools to understand why that is the traffic or to model who it will change. The best tools are related to ads (which makes some sense since that is where they get all the money from) which allow some better modeling (e.g. some historic data about keywords) but it is limited to paid ads and not good enough in my opinion.

Business potential: I think that if one of the 3 bug search engines (or some third party) would create a tool allowing people to model how various changes will impact their search traffic, it would be a huge differentiator (and will allow them to sell more ads indirectly). All the data I have listed in the bullets above exists or can be gathered by the search engines. This is data that is currently underutilized. Probably in 5-10 years we'll get there. Too bad...

So what happened?

Anyway - enough with day dreaming. Let's see what actually happened to 1 family. friendly. food. First, the data (click to enlarge, the y axis is hits per week):

We started October with about 8 hits per day, it went up to a peak of about 12 per day towards the end of October. And then it fell back to 2 hits per day by mid November, with a slight increase in the past few days.

The main thing I can attribute this change to is the fact that we got a new domain name at the end of October, and redirected the traffic from the old address (wflavors.blogspot.com) to the new domain (http://www.familyfriendlyfood.com/). My assumption from trying to use the Google web master tools and trying to search for the site in Google is that it took them much less time to figure out that the old address is just a redirect and remove it, then it took them to add the new site. Hopefully the increase in traffic in the past few days will continue and get back to the old levels. One option (that I haven't seen mentioned in the SEO articles I have read) is that their search ranking algorithm takes into account the age of the site (discounting new sites), and will gradually increase the ranking. If that's the case, then it is just a matter of waiting. Note that I did see that the site was crawled starting in late October, so Google knew about it immediately after it went online.

As you can see, there is a lot of guesswork involved (more below). I have done a bunch of other things to improve the search engine traffic. Here is the list:

I have added the site to Live Search and Yahoo! Search in September. We had the first hits from Live Search and Yahoo! Search in Mid November, and 3 more hits since. Still negligible compared to Google Search, and it took Microsoft and Yahoo many months to catch up (the site has been live since July), but it is progress.

I have created several landing pages on the site (outside of the blog) to cover specific topics. The idea was that each such page will talk about a certain subject that is relevant to the site, and thus have a certain concentration of keywords increasing the likelihood to be found relevant by a search to those keywords (and also redirects to relevant pages on this topic). There is an Index page, a page about Weekly Menu Plans, a page about Cookbook Reviews, and a page about Kids and Food and so on. It is still a work in progress. So far it hasn't been successful (only 1 hit to those pages in November), but I have only seen Google pick up and index those pages a few days ago, so we'll see how it will go in November.

I have add a meta description tag to all the above pages. That should help drive traffic and cause people to click on the links if they are in the search results. The first few pages that I have created didn't have this tag, and Google indexed them with its default description. The newer pages which I have created with the description have it in the Google search results (to see all the search results, use this link). It has been almost a month since and Google has not updated the description for the older pages. I hope it will do it soon.

I have also added the blog to a couple of directories (blogged.com and bloggapedia.com) and submitted to a few others, although it is not at all clear if this influenced the search results in a any way (click this link for a list of sites linking to 1 family. friendly. food., the directories don't appear there).

Bottom line - I plan to read and learn more about this topic, and continue to do some (educated) guesses to improve search engine traffic. Hopefully, it will get to a point when it is closer to a science.

October 2008 Results

October was by far the best month for 1. Family. Friendly. Food. so far, with the number of visits going up 483% month over month to 1825 visits. There were 1480 unique visitors (up 492% from 250 in September). In November, I expect the number of visits to go up 50% to about 2750 (see detailed analysis below).

Here is the week by week change in number of visits since July (click on the image for more details).



The income was still $0.00, with no clicks on ads. Currently there are no ads in the web site itself, only in searches.

Below you will find the distribution of visits by category, a quick overview of the status in each category, and the expected number of visits in November.

Search. This is traffic coming from search engines. The number of visits from search in October was 303, up 573% from 45 in September. See this post for more details about search engine traffic. In November, I expect a 60% increase in search engine traffic due to more posts (and therefore more hits), better rankings, and adding the blog to Live Search and Yahoo! Search.

Social sites. This is traffic coming as a result of publishing the posts in sites such as facebook, digg, reddit, etc. In October, there were 909 visits in this category, up from only 26 in September. This was by far the largest category in October, accounting for half the visits. 97% of all hits are from reddit.com, and I'll publish a separate post about this topic. In November I expect an increase of 75% in this category, due to a planned marketing push to several of these sites.

Other Family Friendly Food properties. This is traffic from other blogs that my wife writes (such as her new blog at Seattle P-I and Good Food Bad Food) and this blog. Traffic was up 907% to 141 visits in October. In November, I expect an increase of 50% primarily due to traffic from the Seattle P-I blog (which only came online a few days ago).

Other referring blogs. This is traffic originating from other blogs, usually food blogs. There were 93 visits in this category, up from 12 in September. Bloggers try to increase their traffic by posting comments in other blogs with a link back to their blog. Nurit is not very active in this, so I only expect a modest increase next month.

Direct traffic. This is users directly navigating to the site, probably by clicking a link in an email. There were 258 direct visits in October, up from 102 in September, but I don't expect any increase in November. I am planning to see if there are ways to improve the instrumentation for this category.

Web Mail. This is users clicking on a link in mail programs such as Gmail, Hotmail and Yahoo! Mail. There were 39 visits in October, and a similar number (35) in September. I don't expect any change in November.

Portals. This is a new category including things such as My Yahoo!, iGoogle and web news readers (such as Google Reader). It only had 15 visits in October (up from 2 in September), so it is too early to have a good plan for it. I will track it closely in November.

In terms of the quality of the visits, the number of pages per visit and average time in each visit went up slightly in October, after going down in September.

Google 4 - Microsoft 0

I've just realized that all the technology I use for the blog belongs to Google. Microsoft either doesn't have the services at all, or the Microsoft services are not up to par with Google's.

Blogging platform: I use Blogger, which was acquired by Google in 2002. While it has its own limitations (and I am considering moving to a more advanced blogging platform than Boller), Microsoft's Live Spaces is nowhere near the requirements for a professional blog. I actually like Live Spaces, but it is far behind Blogger, mainly in terms of ability to customize and extend.

Ads: I use Google's AdSense to place ads in the blog. Microsoft doesn't seem to have the ability for publishers to put ads on their sites (as far as I can tell). Microsoft has adCenter which allows advertisers to create ads (competes with Google's AdWords), but that I need the publisher's feature.

Statistics: I use Google Analytics for site usage statistics (see this post for more information). I have received a comment that Microsoft now has this service too. It is called Microsoft adCenter Analytics, and it is currently in Beta by invitation only. From reading the FAQ, the features seem comparable more or less to Google's. There is one feature that Microsoft has and Google doesn't (demographics by age, gender and occupation (not clear what is the source of the data)), and the FAQ state that "...we believe that adCenter Analytics will eventually provide the most useful Web analytic reports and customer insights on the market today,..." (not sure how eventually and today work together). Anyway - I have registered and I am in the waiting list to try it.

Search: I have now added the ability to search within the blog. I have chosen Google's search for 2 reasons: (1) Live Search doesn't currently even index the blog (even though Google found it months ago); I have now explicitly asked to add it to Live Search, we'll see how that goes. (2) Google's search supports publishing ads as part of adSense, while Live Search doesn't.

Too bad...

Search engines

I have been looking at traffic to the blog coming from search engines.

In terms of percentage of total traffic, it has grown from 14-15% in August and September, to over 21% in the first 3 weeks of October.
In terms of number of visits originating in search engines, it started with 22 in August, went up to 45 in September, and it is over 190 in the first 3 weeks of October. That is a huge growth.

Google
Google is at 97% (the remaining 3% is search.com, ask.com and aol.com (which I also think is powered by Google)). Live search and Yahoo! do not appear at all.
When investigating this, I found out that Live search and Yahoo do not even index the blog, even though it has been linked from other sites (such as wflavors.com) for more than 3 months. Sigh... I have now submitted the url of the 1. Family. Friendly. Food. site to both Live search and Yahoo!. We'll see in a few weeks if it helped.

Why is search growing?
It is hard to tell for sure why October is so much bigger than the previous months. I'll continue to track and see the trends going forward. There are two reasons I can think of:

• the blog has more and more content, and therefore more and more searches will find answers in the blog (it maps to more keywords).


• the blog gets linked from more sites as time passes, and therefore its popularity grows and it will come up more on the first page of some searches.

Is search traffic good?
When looking at visitor loyalty, search traffic tends to be lower than the site average. The average time on site was 40% lower than the average in September (but only 10% lower than the average in October) and the number of returning visitors tends to be close to 0 (as opposed to 20-30% on average).

The advantage of having a lot of returning visitors is that once you have paid the price to gain them as a customer, they come back (so are are starting to see an ongoing flow of income from them). On the other hand, given that the cost of acquiring a customer from search is pretty much 0 (we don't have any ads to bring in people), it is still good. For example, in terms of impressions from ads, they still count regardless of how much time they spend in the site.

Next steps
Here is what I plan to do next regarding search engine traffic:

• Look at sites that claim to increase traffic by submitting the site to lots of search engines, such as SubmitExpress.com


• Look at improving the keywords / meta tags to attract more traffic.


• Analyze the top keywords that caused traffic to the site


• Brainstorm changes to the site design that will help this kind of users be more loyal (stay more times / return more / subscribe to the RSS feed, etc.)

On the way to be rich

I have added ads to my blog. Not yet to 1. Family. Friendly. Food. (more about that in a separate post), but it's a start.

In the first 20 days of October I have already made $0.18. Some of you might think it's not a lot, but I can tell you that you are wrong. If I continue at this rate, it will be about $100 until I retire. As you can see, that's a lot. :-)

More info: 135 impressions, 10 clicks, 7.41 CTR, $1.31 eCPM. I only have a vague idea what these terms mean, but I am plannign to learn.

No chain letters for you!

In my previous post, I have tried to convince people to help me do some direct marketing. I wasn't expect that much (given that you guys are all a bunch of _____ [this is the part where I loose the small audience that I might have remaining]).

But still, the result was pretty much 0. Flat. No change. I won't be wearing my "glass half full" shirt tomorrow... Oh, well... Life goes on.

I'll try something more effective. Stay tuned!

Not a chain letter, but...

In one of the previous posts, I have mentioned that the main sources of traffic are direct traffic (mainly someone sending an email explaining how great a site this is), links from other sites and search engines.


The first type is not very sustainable, but before completely forgetting about it, I've decided to give it a try. And here is where you come in! I would like you to send this truly wonderful site to everyone you think might be interested, and ask them to forward it on.


Some of you might be thinking that you are not the type of person who sends chain letters! This is not the case, my friends. We are all trying to figure out how to give our kids and families good healthy food while not spending all our day in the kitchen. And we know lots of other people who are also facing the same challenges. So send them a link to 1. Family. Friendly. Food. and make them happy. And ask them to make more people happy.


"But I don't know anyone who eats or has families", some of you might say. Hmmm... That's a tough one. Really? Are you sure?




To make things more interesting, we will take some bets. Please comment on this post with either how many people you have sent an email about this, or with your guess of what spike we'll get from this exercise. My goal is 100 visitors in 1 day. Click on the image below to see the current direct traffic.



Don't procrastinate! Click send now. Let's see who will be the first to add a comment.