More pages in the search index means more chances to be found. So maybe you never really have enough pages--since every extra page is a lottery ticket in the search sweepstakes. You've gotta be in it to win it. But this does not really offer us any answers. There are obviously some amount of pages that seem OK and other amounts that are probably Bad, like, zero would be bad. How do you determine how many pages you have in the search index, and if your number is enough?

First off, you need to understand that there is no single search index-each search engine has its own search index. Google has its own, Bing has its own, and so do many other search engines. So, you need to know which search engines are worth worrying about-in the U.S., it's Google and Bing.

So how do you find out how many pages are in Google's index and how many are in Bing's?

Both Google and Bing have a tool called the "site:" command. You can just enter into each one the word "site:" along with your domain name (Such as "site:biznology.com").  For some sites, this handy command works just fine and you can see how many pages are stored in each index. If your results look right, great. But sometimes the results just look nuts. For example, "site:ibm.com" yields 2.8 million pages on Bing but a crazy 12.2 million pages on Google.

To avoid such inaccuracies, use each search engine's Webmaster Tools sites. Both Google and Bing will tell your Webmaster exactly how many pages are in the index and will even let you know which pages they are having trouble grabbing. It's possible that the IBM Webmaster is aware that there actually is a big discrepancy between Google and Bing, which might be just fine or might be something they are working on.

I've spoken to a few experts and they have varying theories. One told me that Bing stops crawling when more than 1% of the pages get errors-the Bing Webmaster site will clue you in on this. Another speculated that Bing is only returning counts of pages that get search visits, not every page in their index. No one I spoke with knew for sure why this is happening, but it shows you the importance of checking your numbers.

Likewise, big swings in indexed pages (1,000 pages indexed in Google today vs. 5,000 yesterday) mean that you should look into it. And, in general, an inclusion ratio (pages indexed divided by actual pages) below 70% is something that should give you pause, although with these Bing errors who knows what a good inclusion ration is for Bing right now.

Regardless. knowing how many pages are indexed is the first step to seeing if you have a problem.

Originally posted on Biznology


July 28, 2016





Mike is an expert in search marketing, search technology, social media, publishing, text analytics, and web metrics, who regularly makes speaking appearances.

Mike's previous appearances include Text Analytics World, Rutgers Business School, SEMRush webinar, ClickZ Live.

Mike also founded and writes for Biznology, is the co-author of Outside-In Marketing (with James Mathewson) and the best-selling Search Engine Marketing, Inc. (now in its 3rd edition, and sole author of Do It Wrong Quickly, named by the Miami Herald as one of the 11 best business books of 2007.






Comments(6)

I found that using the site command brings up massive difference in indexed pages to those found in search console, especially if you have a Wordpress site, as site command brings up categories, thereby creating incorrect page date. Search console is in my opinion much more accurate.

This is a great example of something people check closely when they first start a site and then forget (and just assume it's running as it should). I actually went and checked some of my sites after reading this! Thanks. :)

I have also experienced crap results when I put site command to check my indexations results no doubt search console is more reliable tool to check website indexation. Moreover, you've beautifully addressed the whole scenario in your article. Thank you

Hallo there,

Good post.
What do you think about the amount of indexed pages and ranking? Do you think the amount of indexed pages can affect the Google ranking?

Lets say that your website only have few pages, thus you only have few pages that are in the index, the pages have good quality, but they are few. Do you think that to add more new pages on the site generally will affect the rankings of the pages you have, or the site as a whole?

With Google it seems that everything you do about SEO needs to be balanced.

We do SEO do both our clients and to ourselves in Norway. Some of our pages are international, and we have a feeling that on the lager global scale the websites generally need more pages to rank vs when we operate in the small country of Norway.

Do you have any idea about this?

Having more pages indexed is very helpful, first because more high quality content always helps you match more search keywords, but also because Google favors pages that come from sites about the same subject. For example, if you have a very high quality page that Google thinks exactly matches a search keyword, but it is the only page on that site about that subject, Google might very well rank it lower than a lesser page from a site that has dozens of pages on that subject.

Google would rather send a searcher to a good page from a relevant site than to a great page at an irrelevant site, because the assumption is that the searcher could click around on the relevant site and find the answer even if that particular page is exactly the right one.

So, having more indexed pages on the same subject can definitely aid ranking.

Thank you for your answer Mike.
Best Regards,
Peter

Comments closed after 30 days to combat spam.


Search Engine Guide > Mike Moran > Do you have enough pages in the search index?