Martin Schaedel, my search engine optimization colleague from Sweden, recently conducted a telephone interview with Stephen Baker, Director, Internet Business Unit of Fast Search & Transfer, and managed to come away with some very interesting and surprising tidbits for you. Take it away, Martin!

The Index

It's been reported that FAST doesn't always update their index every nine to eleven days as they state they do on their site, so I asked Stephen to tell us a bit about FAST's indexing procedure. He told me that they currently update 30% of the index every seven days. The rest of the index gets updated approximately every 28 days, and they're working to make that even faster.

They're also hoping to have 2 billion URLs indexed in the near future. They currently have over 900 million URLs in the index, and have validated over 600 million. Right now they're working on crawling more and more pages in order to validate them and find more data. This new verified data will get put into the index soon.

Pay-for-Inclusion

FAST recently introduced their new InSite Select program in partnership with Lycos. I asked Stephen how the new pay-for-inclusion program would affect their index and site owners. He told me that one of the things it will have an impact on is the indexing of dynamic content. Spiders generally have to be careful when crawling dynamic pages, but with paid inclusion these are easily added because they can be reviewed first, as necessary. For Webmasters, they're planning to release some new tools such as clickthrough and keyword reporting.

What about Graphics and Flash?

FAST has plans to index many new file types. They're already indexing .pdf files and some multimedia files at Scirus.com, and are working on integrating these into their "normal search" results. FAST also has the ability to "read" the content in .gif files, and are already utilizing this to a certain extent. They've been in contact with Macromedia about "reading" Flash files through some type of converter; however, it doesn't look like something that will be happening anytime soon.

The World

FAST is unique in that they have partners on every continent in the world. They started in Europe when they didn't see any "major players" there. Now that FAST has a huge presence in Europe, they'll be focusing on North America, South America and Asia. In fact, they just announced their new partnership with Telus, one of Canada's leading telecommunications companies. There's a list of many of FAST's partner sites here:
http://www.webmasterworld.com/forum4/66.htm.

Spam

Something often discussed by those in the search engine optimization business is whether cloaking is considered spam. I asked Stephen how FAST felt about cloaked pages. He surprised me by saying that cloaked pages may be okay depending on their intent. With their free submission option, they can't tell the intent of cloaked pages, and therefore they're likely to remove or ban them. With their paid inclusion option they have more flexibility, because they can determine the intent of the cloaked pages.

If you see pages that you believe to be spam in the FAST search results, you can report them to spam@fastsearch.com. You can view their spam policy here: .

Martin Schaedel, StarGlobe
[martin@starglobe.co.uk]

[Thanks to Stephen Baker and Peter Gorman from FAST (and of course to Martin) for providing us with this great information! - Jill]