Today, I have a great interview for you from Barry Lloyd of Microchannel Technologies Ltd. (Designers of MakeMeTop). Barry has tons of experience submitting his clients' sites to Inktomi through their paid inclusion program. Since I've not used this program much, I thought we could all benefit from Barry's wisdom. Please note that Barry has no connection with Inktomi and is simply giving us his perspective as a long time SEO consultant. To me, this information is even more valuable than the info (propaganda?) I might have received had I done an interview with someone from Inktomi. I'll let you be the judge. Feel free to send in your own experiences with Inktomi or any other search engine, for that matter. I'm looking for all the inside scoops I can get!

So without further ado...on to the good stuff!

- Jill

Jill: Welcome, Barry. Thanks for taking the time to answer my questions for the Advisor readers. Let's get right to it! Do you recommend that your clients use Inktomi's paid inclusion program - "Inktomi® Search Submit"?

Barry: Yes. In fact, it is included in the prices I quote to a client. I no longer view it as an optional extra - certainly as far as the main pages of the client's site are concerned. Apart from using pay-per-click (PPC) engines like Overture, it is currently the fastest way for a client to get indexed in a broad spectrum of search portals.

Jill: Inktomi's program ensures that sites will be spidered every 48 hours. In your opinion, what advantage does this frequent spidering provide to your clients' sites?

Barry: The first advantage is fairly obvious. You have the ability to adjust your page content to enhance your rankings knowing that you can see the results of your efforts in a few days. With other search engines taking weeks or even months to show results, this feature is of great help in fine-tuning your web pages to improve your rankings.

Jill: Are there any other advantages to using the paid-inclusion program?

Barry: An advantage, which I have not seen exploited frequently, is to include (or remove) special offers or short-term product offerings. As an example, I was asked to promote the sale of an Irish hotel and Pub. Within 72 hours of the brief, the property appeared on all the major Inktomi partners in the top five results for "irish pubs for sale," "hotels for sale ireland," etc. The client received several serious inquiries every day from around the World and stopped taking offers one month later when the pages were removed from Ink. As a result, the pub sold for more than the asking price - yet the cost on Inktomi was less than $100 (plus my modest fee)! Apart from using PPC, which can prove expensive, I know of no other method that can have such an immediate effect and response. Waiting for the pages to be spidered and included on their own would not have been an option in these circumstances.

Jill: Are there times or situations when it is not necessary to use Inktomi's paid-inclusion, or where it might even be detrimental to do so?

Barry: If your page(s) are already well listed, paid inclusion will not do anything for you! Indeed, it could possibly be harmful, since adjustments can damage your current position. Sites that have been included naturally into Inktomi are ranked by on-page and off-page methods, including inward links and quality of linking sites. Paid inclusion works better for pages which have not already been included and should not really be viewed as a way to fine tune pages that already rank well through Inktomi's main algorithm. I've learned this the hard way; losing a #1 position I had held for over a year and taking a long time to climb back up to the top five - and I'm still not back at #1!

Jill: Which of Inktomi's Search Submit partners have you used and which would you recommend to others?

Barry: All partners I have used have resulted in pages being included in the Inktomi index within a reasonable time. Once included, all partners are very much in the hands of both Inktomi's spider and the update schedules of the various Inktomi search partners who may not (for various reasons) update from the Inktomi database as quickly as hoped! I have used Network Solutions (Verisign), Ineedhits, and PositionTech. Of the three, I have found PositionTech to be the easiest for me to work with. Their customer service has been superb and their key staff both approachable and helpful - even though some things that occur may be out of their control. This is not to criticize the service provided by the others, at all. I have never had to contact Ineedhits, and only needed some simple questions answered by Network Solutions.

[Moderator's note: Outrider, and Trellian are Inktomi's other Search Submit partners.]

Jill: I've always been nervous that if I pay to submit a URL, Inktomi will remove all the URLs from my site that I didn't pay to be included. In your experience, have you ever seen this happen?

Barry: No, never! There seems to be a lot of confusion about this. Inktomi has a number of spiders - only one of which is their paid inclusion spider. The paid inclusion spider crawls paid inclusion pages only. However, this does not prevent the standard Inktomi spiders from finding your site independently and crawling it completely. On my own site, I only pay for two pages to be spidered which have regular new content - but over 100 pages of my site are included in the Inktomi index. But just as Inktomi can spider a site, it sometimes decides to drop a site from its index. This could be misunderstood as being a consequence of entering the paid inclusion program, when it is not. Usually I find that sites that have been added to Inktomi naturally, have good content and are well linked to. These are remarkably "sticky" and can retain their positions for long periods of time.

Jill: What happens when you remove a URL from the paid program? Does the URL get permanently removed?

Barry: If you remove a URL from the Paid Inclusion Program it will be dropped from the paid inclusion database. That URL will disappear from the listings, BUT - if you are in the standard Inktomi database, the URL will reappear in the next update of the full Inktomi database range, which seems to occur about every four weeks. Of course, most people don't know if they're in the standard database - so it could be a worrying few weeks to see if your URL reappears! Few will risk it. You can only be sure that your URLs will STAY in Inktomi by paying for them. The standard Inktomi databases are comparatively small and are constantly refreshed. If you have the confidence to feel that your site is going to be well linked to and has good content, then you can wait for Inktomi to include it. It will take several months, though, and you may not get included at all - and in my experience, most clients don't want to wait. However, if you have the patience, are getting good links to you and have a well prepared, spider-friendly site - it may be well worth the wait!

Jill: What do you know about Inktomi's different databases?

Barry: Well, let me prefix this by saying that Inktomi are the only people with the full answers regarding their databases! However, I can give some observations that I have made over my years of dealing with Inktomi. Inktomi has its own Map of the Web that includes some 1.7 billion documents. Through this web map, Inktomi selects data, which it includes in several databases. These are Best of Web (BOW), which is some 110 million documents, GEN3™ which is around 500 million documents, their Eurocluster, which is 110 million documents and the Asia Pacific cluster (APAC) of around 55 million documents.

I will be concentrating here on BOW and GEN3. Aside from paid inclusion, Inktomi can find sites that are submitted through add URL pages via one of their search partners like HotBot. They can also find sites by crawling the web and following links. When a site is initially found (by whichever method), it usually winds up in the GEN3 database. If it has been submitted via a search partner as a free submission, the URL may be subject to a penalty due to many free submissions being spam. However, if the penalized URL is found later by Inktomi spiders crawling the web via another link (or if you opt for paid inclusion), the penalty is lifted. The GEN3 index is generally refreshed monthly. Sites may be dropped out if other sites have been submitted or found which appear to be of higher "quality." Sites are likely to remain if the number of "quality" sites linking to your site increases.

If you do fall out, it is possible to get back into GEN3 as Inktomi continues crawling and finds more quality sites which link to you. The more sites of high quality that link to you, the "stickier" your site becomes. Hopefully, after a period of time, your site is considered by Inktomi to be of sufficiently high "quality" and have enough "stature" to be considered a "Best of Web" site.

The best way to become a BOW site is to have other BOW sites link to you. Unfortunately this is easier said than done. Although links from directories such as Yahoo, ODP and LookSmart are important and help - they will not guarantee inclusion. You need sites that are already in BOW to link to you, which are on a similar theme and of high stature. The advantages of getting in BOW are enormous! If you are in a situation where you have (in paid inclusion) done everything to optimize your site and still can't get into the top ten - the odds are that the sites in front of you are in BOW - where a lot more than just on-the-page criteria contribute to the rankings. Sites in BOW are very "sticky," although (like GEN3) the BOW index is refreshed regularly. If links to a site begin to fall (or lose quality), the site will fall from the index. Paid inclusion neither helps (nor hinders) inclusion into BOW. Selection is purely done on the "merit" of the site as viewed by Inktomi through its map of the Web. In many ways, this is similar to the way Google looks at sites through its PageRank system - but Inktomi's BOW is a slightly more "exclusive" club. If you are in paid inclusion and suddenly notice your site has leaped up in the rankings without doing many changes - then you may have hit the jackpot and managed to get into BOW. If you are in BOW and are also in paid inclusion - you do have a chance to adjust your pages and compete on slightly advantageous terms with others in BOW who are not in paid inclusion. You can also have time for a slight smile as you watch other sites try to beat your rankings and fail. My advice would be to leave well enough alone and look for more good links. If you are in BOW, odds are you will find it relatively easy to get them.

Jill: How do links from the major directories such as LookSmart, Yahoo and DMOZ affect your Inktomi listings?

Barry: As I said above, all good links are important - but Inktomi has a particularly close relationship with LookSmart. Like all crawler search engines, Inktomi views sites that have been reviewed by a real human being as a vote of confidence in your site. Having a good description in LookSmart can do wonders to your rankings. Often you will find sites in Inktomi use the LookSmart Title and Description and I have seen (what I call) the "Double Title and Description" effect! If a site is in LookSmart, it seems that not only is your on-page Title and Meta description used towards your rankings (as well as your main content), but the LookSmart content as well.

Remember that Inktomi can be a big provider of traffic on AOL and MSN (provided that you focus on phrases that are not going to be buried under MSN LookSmart listings). Good research on phrases can be more than worth their weight in gold and worth the work (and wait) to get them high on Inktomi.

Jill: Thank you very much, Barry, for sharing your knowledge and your time! I hope you will all check out Barry's site for more information about his company.