I've often been asked why particular pages are not indexed. Honestly, you can never be sure until you fix the problem. If you think that you've isolated the problem, you never know if you've only corrected one of multiple problems. So, it's best to take it step-by-step.
The first step is be sure that your page really is missing from the search index. (Throughout, I'll be talking about the search "index" but each engine has its own index, so you must check every blessed one individually.)
Most search engines allow the use of a special operator to reveal if a page is in the search index. As an example, if I wanted to know if a page from my Web site was indexed, I could search for "site:www.mikemoran.com/aboutmike/index.htm"--the search engine would show the page in its results if it is in the index.
Assuming that your page is not found using this method, the next step is to try to figure out why. One possibility is that most of your site is missing, which you can determine by a similar search, such as "site:www.mikemoran.com"--you can see how many pages are indexed. If it's very few, your problem is bigger than that single page.
Major site problems include:
But what if it really is just this one pesky page that isn't being indexed? Some problems are likely to be confined to a single page:
Once you've identified what's wrong, you can correct the problem and wait for the spiders to come back. Good luck getting all your pages indexed. Remember, if your page isn't in the index, it can never be found.
Mike is an expert in search marketing, search technology, social media, publishing, Web personalization, and Web metrics, who regularly makes speaking appearances.
Mike's previous appearances include ClickZ Live, RKG Summit, Ticket Summit, Webdagene, the CiTE conference, and the Forrester Marketing Conference.
Mike also founded and writes for the Biznology newsletter and blog, is the co-author of the best-selling Search Engine Marketing, Inc., and sole author of Do It Wrong Quickly, named by the Miami Herald as one of the 11 best business books of 2007.
Copyright © 1998 - 2014 K. Clough, Inc. All Rights Reserved. Privacy