Source: Webmaster World - "We do support what we call a crawl delay. Basically it allows you to specify via robots.txt an amount of time (in seconds) that MSNBot should wait before retrieving another page from that host. The syntax in your robots.txt file would look something like..." Link to Article





About the Author

Search Engine Marketing Columnist

Search Engine Marketing Columnist