Search Engine Guide
Search The Internet: 

Orbidex provides two additional ways to stay current on search engine optimization techniques, trends, and other vital information:

Subscribe to have these articles delivered directly to you each week via email.

In addition to receiving the weekly articles, be sure to sign up for Orbidex's monthly newsletter.

Search Engine Optimization

Article provided by:
© 2001 Orbidex.

Back To Article Index

Tell Your Friends About This Site

How Server Logs Help Develop SEO Campaigns
By Eric Lander - December 07, 2001

While every web site has a different end goal for its users, every webmaster (and site optimizer!) wants to retain and convert more traffic. Even if you had the recipe to create the perfect traffic-generating web site, there are still some things you can review to further maximize your site's traffic and conversion rates. Those things, are called log files.

Webmasters and site owners alike, have practically fallen in love with their site's reporting and statistics. These reports are completely generated based on information contained within log files. Through careful analysis, the information in log files can give you lots of traffic-building ideas, and help you measure the success of past optimization techniques.

There are more ways to promote a web site than minutes in a day, so choosing any method of site promotion should be done with a knowledge base available. Mining the data from your raw log files will provide you with just that.

What are Log Files?
Log files are text-based files that your web server writes data into each and every time any part of your web domain is accessed. The server stores this information for security, analysis, and record-keeping purposes.

How do Log Files Work?
Each time a server sends a document off to a user, a new entry is made into the log file. For every resource that is accessed on the server, the following information is archived:

- IP Address
- Time of Request
- Formal Request to Document
- Referral of Request
- User's Operating System
- User's Browser (or "User Agent")

What Does a Log File Look Like?
An exmaple... - - [19/Jul/1999:00:00:04 -0600]
"GET /news.htm HTTP/1.1" 200 20607
"Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

How to Get the Info You Want...
Regardless of your log analysis program (WebTrends, Hit List, etc.) there are some consistent areas of reporting that you can retrieve vital information from. I'll run through these areas, as well as give some detail on what to look for.

- Most Requested Pages -
Okay, so you have a list of the pages on your site that are requested the most. While some web site owners expect to see their "home" page listed here - that is not always the case.

From an optimization point of view, there are many variables that can contribute to where these hits are coming from. Look at the top requested pages, and ask yourself:

Does my link popularity campaign target this?

Which engines & directories are directing traffic to this URL?
- - What keywords and phrases retain the most for this page?
- - How does the SERP listing for this document differ from others in my site?

Has there been a recent EMail campaign distribution to affect this?
- - If so, when, and how many were sent?

Is this URL the target for any advertising?
(Ex: Google Adwords, Overture Bids, etc.)

Is this resource linked to in the navigation of each page?

What else could influence this?

Reaching a conclusion on any data for web statistics can be difficult. As an SEO, be mindful that your attempts to optimize may not produce the type of results you had expected. Be sure not to get discouraged though if that is the case. Rather, use the information from other more prominent pages to reinforce your continued optimization trends.

- Least Requested Pages -
Well, for every good, there is a bad too! Here, you will want to look equally hard to make sure that your pages with the least traffic are pages that should be there. Your main product or service listings should NOT be here! You would hope to see some obscure pages like Contact Information, Site Map, etc.

This also helps an optimization specialist understand what pages need the most work. That is only the case if pages with importance are listed here - showing that they will soon require further optimization.

- Top Entry Pages -
This is pure gold for any SEO. If you have recently reworked the optimization of your entire site, this is where you will be able to get some critical information! What engines have these URLs listed? Who else links up to this resource? What levels of traffic do they provide? What do SERP descriptions look like? All of these factors will play a role in measuring the success of any optimization campaign.

- Top Exit Pages -
This is simply a display of where those looking at your site leave the most often. Of course, no one wants to see their mission-critical pages listed here! From an optimization point of view these pages will need some work, but should require the most attention from a usability standpoint.

Look at each one of your top exit pages and make sure that the content of that page matches the description of the links pointing to it. Often, users will click a link expecting one thing and be delivered to something completely different. All of these variables as well as other must be reviewed to make sure that you are keeping visitors interested - not frustrated.

- Single Accessed Pages
- This is where any search engine optimization specialist can find success, failure, or both all at the same time!

These listings pertain to a web site document that is served up, accessed, then the user leaves. How can you determine success from this? Simply! If you have optimized this page to no end, then you want a lot of people coming into it - and that's great. But, the failure lies in not being able to convert this visitor into someone interested.

Some percentage of site visitors will leave - because they simply anticipated something other than what they have seen. Others though may feel lost or confused based on the navigational system or worse - terminology and language. Bear in mind one thing: Optimization is great, but the site still needs to work. If your optimized pages appear on this list, start thinking less optimization and more usability.

- Document Not Found Errors (404's!) -
I covered the importance of these a short while ago, and you can read up about them in great detail at:

To reiterate some points with errors though - keep the following in mind:

Search engines follow links to more resources - so make sure you do not have broken links.

Search engines and directories may have loved your old page, and listed it. If a document that does not exist anymore is listed - get something in it's place to help retain that traffic!

Create and maintain a robots.txt - It directs search engines and if it does not yet exist, I am near positive it is in your 404 listings.

- Some More Information -
...Of course, there are many other sources of information that can help an optimization specialist to acquire more prominent links. They do however differ from one reporting tool to another. My only advice is this, look at everything and keep track of any significant changes. It takes time to draw conclusions - but the results are sure to offer more success!