Webmaster tools are becoming a hot topic in the major search engines these days. It is becoming increasingly obvious to search engine companies that they need to begin listening to webmasters and provide more tools to ensure future brand integrity. It is about time! I have been waiting for the search engines to wise up and over the past couple of years both Google and Yahoo have begun to prominently feature such tools. This article is devoted to Yahoo’s Site Explorer tool and how you can harness it to help you improve your search engine rankings and improve your odds for online success.

THEN: September 29th, 2004
Yahoo! officially launched Yahoo Site Explorer (see release). This new tool was designed to promote feedback between webmasters and Yahoo search staff. Ever since Yahoo! launched and became the powerhouse it is today it has had a rather bleak reputation for responding to customer search inquiries and ranking or directory complaints; this is another great step they took to change public opinion.

When it launched in 2005, Yahoo! Site Explorer provided a central interface for the following:

• List all subpages of a site domain (domain.com) or path (domain.com/~yourname)

• Backlinks (or inlinks as Yahoo calls them) for both site domains and paths.

• Free submission of sites not indexed yet by Yahoo. Submissions can be done either one at a time or by bulk. The bulk method requires that you identify the location (URL) of an online text file with the list of URLs to be submitted.

• Download up to 1000 search results in TSV format (TSV can be read by Excel)
  

NOW: August 8th, 2006 – New and Improved!
Yahoo has announced a fairly dramatic update to this service which provides more tools and a cleaner operating environment.

Yahoo Site Explorer has added the following new capabilities:

• Additional information about each page within your associated website(s):
  - “Last Crawled Date”: This is the last time your page was indexed by Yahoo! Slurp (Yahoo’s robot).
  - “Language”: the language of the page (a waste of space in my opinion)
  - Show pages from subdomains: this will list any subdomains that Yahoo has identified and will allow you to choose between seeing indexing results on “all subdomains” or “only this domain”. This is merely a filtering feature.

• RSS Feed Submission: submit your RSS or Atom feeds and manage them from your Site Explorer control panel.

• Automated page update notification service (info link): designed for the Yahoo Developer Network this is useful only to those that have automated applications that can interface with Yahoo!’s API. This service allows you to tell Yahoo when your pages have been updated so that it knows when to recrawl.

IMPORTANT: this new information is only available if you have authenticated your website. How to authenticate your site on Yahoo Site Explorer is covered here.

Getting Down to Business: How Yahoo Site Explorer Can Help You!

There are two main areas that I want to focus on in regards to how Yahoo Site Explorer can help you.

1. The “MySite” Area – Your Own Websites
This following points require that you have authenticated your website(s). Click on Figure 1.0 to see what this screen will look like if your site(s) are authenticated.

 

	The first and most obvious advantage of seeing your sites in Yahoo Site Explorer is verifying which pages have been indexed. If you find that many pages of your site are missing this may be a great indication that your site is not search engine friendly. Is your site search engine friendly?
	The ‘inlinks’ (aka. backlinks) option is a fantastic way to view the links pointing to your website. The best part is that unlike the linkdomain: search that Yahoo offered before, you can actually download up to 1000 results in a small file that you can then filter with Excel or your spreadsheet application of choice. Why is this helpful? First of all this is an effective means of determining which sites are providing the most links to your website. These websites are obviously very interested in your product/services so perhaps contacting them and strengthening that relationship would be a good idea.
	Again the inlinks has another advantage; determining which sites you should appear on *more* often. Let me explain by describing this review step by step because this is a little bit advanced but well worth the minimal learning curve. (If you find the following information just too much to handle keep in mind that StepForth offers professional website analytics and competitor analysis services to take the work out of your hands.)
	1) Look at the referrer information in your website statistics and write down the sites that appear to be driving a lot of traffic to your website. 2) Now, download the inlink data from Yahoo Site Explorer for your website (up to 1000 results can be downloaded) in TSV format. 3) Open the TSV file in your local spreadsheet program and sort the results by website. 4) Now that the website results are sorted alphabetically try to find any sites from your list of top referrers that are standing out as having particularly few links to your site. 5) If you have found a site that has a high ratio of traffic to backlinks then you might have just stumbled on a gold mine. Say what? Just consider what you have found for a moment. Here is the situation; a particular website has few links pointing to your site but it is still delivering a high level of traffic. Armed with this information your next step should be to figure out how to get more links from that site. Perhaps the webmaster will be interested in providing more exposure to your site freely or for a low advertising cost. You simply never know unless you research the site and/or contact the webmaster.
	And keep in mind that even if you don’t find any goldmines you are still learning a lot more about your target market such as which sites they frequent and what information appears to be of interest to them; which is always helpful.
	You can submit your RSS feed if you have a blog. This is a great way to be certain that your feed is being spidered by Yahoo.
	Are you curious why a particular page in your website is being used as an access point? Try typing in that page’s address into the search bar in Site Explorer and if that page has been indexed by Yahoo it will appear. Now click on the inlinks report for that page and you will have a complete listing of the websites that are linking to that page. Look at each inlink page and you will get a better idea of why your site page is becoming an entry point. This research method is nothing new but this quicker approach is slick.

2. The “MySite” Area – Your Competitor’s Website
The following points focus on the analysis of your competitors so site authorization is not required. Figure 2.0 provides an example of the screen when you are viewing a site you have not been authorized for.

Are you curious what your competitors are up to? Recently I wrote an article on how to conduct your own competitor analysis and I wish I had included Site Explorer’s features in it. Here is how I use Yahoo Site Explorer to check in on the competition. After I have shown you how to get the information I will explain what information you can glean from it.

1. Type in the main domain of the competitor in Site Explorer.

2. From the competitor’s result screen click on ‘inlinks’ so that you have the full inlinks (backlinks) report on that competitor.

3. Now just above the first set of results you will see the following options that allow you to filter the inlink results:

The “From All Pages” means that you are seeing inlinks from any page; including links from the competitor’s site to its own site (internal pages linking to each other). So in order to remove these links that don’t provide you with any useful information you want to change this to “Except from this domain” so that you are not getting internal links showing up. This is how it will appear:

4. The resulting list will now show only external sites linking to your competitor; a much more valuable report since you have now removed backlink information that is useless to your research.

5. Next you have two options; one is to download as many results as you can using the “Export Result to TSV” command (usually only 50 or so will download) or you can begin reviewing the sites visually, page by page. I would start with the first step and then move on to manually reviewing the sites within Site Explorer after you have exhausted the first 50 that are handily provided in spreadsheet format.

Okay you have this report… now what? Here 3 key pieces of information you may find valuable:

1. Benefit from their Work: You can now create a list of sites that your competitor(s) have managed to get links from. Next try to match these links by contacting each site’s webmaster and asking if they would consider linking to your website. This is a great method for in-house link building.

2. Advertising Revealed: Have you noticed that a large number of backlinks seem to be coming from a particular domain? Look into this domain closely; it could be that your competitor is paying for a link which is directly converting to beneficial backlinks. It may be worth contacting these site(s) and requesting pricing for advertising of your own. If advertising is not possible then at the very least you now have a part of the equation for your competitor’s ranking successes (if they have any).

3. Forums and Blogs: Keep an eye out for backlinks from forums or blogs. These backlinks may indicate a particularly useful forum that could drive business to your website. It also cannot hurt to benefit from the forum interaction by placing a backlink in your post signature (same as an email signature). Just don’t expect much of an advantage from signature backlinks because rampant spam from unimaginative spammers has rendered such links less important.

Yahoo Site Explorer is Now Unavoidable
For those that like to do research by hand you may be asking “why would I use Site Explorer when I can perform my own research by hand?” Yahoo Site Explorer has made itself somewhat unavoidable since it began redirecting most searches for “link”, “site” or “linkdomain” to Site Explorer which in-turn requires that you have a free Yahoo account.

Summary
The tools provided within Yahoo’s Site Explorer tool may not be ground breaking but they sure make certain forms of research much easier. By making queries that used to be entirely manual (link:, linkdomain:, site: etc.) available automatically they have certainly made the world of competitor analysis that much easier to experience for the average site owner.

Kudos Yahoo! I think you guys are definitely on the right track; you have certainly earned thumbs up from me for this innovation.

Discuss this article in the Small Business Ideas forum.