The Open Directory Project pulled off one of the funniest April Fool's Day pranks by redoing their web site as if Microsoft had bought the directory and renamed it "Microsoft Directory Project" (MDP), also being called Gates Open Directory (GOD). While this is one of the gutsiest moved pulled off by any search engine or directory, if you want to read more about the prank you should visit the ODP Forum at WebMasterWorld.com as we will not be discussing it here. The April Fool's Day joke by AOL and ODP targeting MSN inspired me to write about the DMOZ.org directory, their results, their shortening reach within the search engines, and the directory as a whole.

In parts 1 through 3 of this series of articles we have dealt with the large players in the search engine, directory, and portal industry, AOL.com, Yahoo.com, and MSN.com. If you have read these articles you will notice that each of them has something in common with the rest. To rank on Yahoo.com, MSN.com, or AOL.com you will have probably have to pay for spidering or submission. This is not the case for the Open Directory Project (ODP), which still reigns as the largest human edited directory that has not yet changed to paid submissions to the directory. ODP is hosted and administered by Netscape Communication Corporation, and is owned by AOL/Time Warner.

As of April 2, 2002, the ODP has 3,261,127 sites within the directory, with 47,263 editors managing 46,932 categories. This is quite a bit of work, especially for directory editors that are not being paid and are doing all of this in their free time. This last sentence encapsulates part of the problem with DMOZ.org and the problems that the directory is facing.

Currently Netscape Search, AOL Search, Google, Lycos, HotBot, and Fast/AllTheWeb use data and results from the Open Directory Project. DirectHit was among this list, but as of April 1st, 2002, DirectHit has shut down the web site and is redirecting visitors to Teoma. There are hundreds of smaller sites and search engines that use data from ODP, but the major search engines, directories, and portals that need to be examined are mentioned above.

In the past, AOL.com was using the results from ODP for their search results in conjunction with results from the Inktomi database. As we discussed in the AOL.com section, in the beginning of 2002 AOL has switched to using pure Inktomi results and only uses ODP data for their directory. HotBot also follows the same suit that AOL currently does as they are now using ODP for their directory and Inktomi results for their search engine results. Still following this suit is Lycos.com, who uses ODP for their directory of web sites, but uses FAST/Lycos search algorithms within their search engine results pages.

This brings us to Google, whose relationship with ODP is similar in some ways to the search engine relationships described above, but different in a number of ways. As was the case with the search engines named above, Google uses ODP data for its web directory, at directory.google.com. While the relationships between DMOZ.org and the other search engines stops here, Google's relationship goes a bit further. The best way to be included in the Google is to acquire links pointing to your web site from other web sites that are already within the Google database as Googlebot will follow these links and index your site. Another way to be included within the Google database is to get your site listed within the ODP directory as Google spiders these links and also uses this data for its directory as we mentioned earlier. A good listing within the Open Directory Project will not only assist in getting your web site included within the Google database, it will also help your ranking.

Another search engine that uses data from the Open Directory Project is Fast/AllTheWeb. Ulrich Klammt, the moderator of the Fast forum at WebMasterWord.com has given us some insight into the use of ODP data within Fast/AllTheWeb. Ulrich explains, "Fast uses ODP taxonomy for their grouping of sites. My conclusion is it helps immensely being listed in the first possible taxonomical group. Ulrich also goes on to explain that, " Fast uses ODP data as a basic structure for building their taxonomy." This means that obtaining a good listing within the Open Directory Project, within the right directory category, will assist in ranking your site within the correct keyword markets and help in ranking.

So, from what we have discussed thus far it would seem that ODP is still doing fairly well on the Internet, maintaining its reach within the search engines and directories, right? I, for one, am not so sure.

The Open Directory Project remains to have a level of allure within the directory and search engine industry as it remains as one of the last standing free submission sites, but at what price does this free submission come?

It has been noticed by many that the ODP directory has been slow on many fronts. The directory is notorious for having inclusion times of up to 5 months after submission. If you refer to earlier in the article, we mentioned that ODP has 47,263 editors that manage 3,261,127 sites within the directory. This calculates to an average of roughly 70 web sites to manage per editor, with new submissions to the directory coming daily. The amount of web sites and responsibilities that each editor is responsible for is a lot for someone to bear if they are doing the job in their free time and not being paid for their efforts.

ODP has also been noticed to take long amounts of time to update their directory, which ties into the point above. This also ties into dead listing factor. This is when a web site or web page is no longer online, which results in a 404-error page being displayed when someone clicks on the link within ODP.

The ODP's results cannot compete with other search engines and directories that are updating more frequently and regularly and adding new sites. The directory's results are quite often stale, which contributes to a smaller user base than Google or Yahoo.

What is the current status of the Open Directory Project?

The current status of ODP is lower than it has been in the past, and will take some working to restore the directory to the higher status that it once enjoyed. ODP has lost some of its footholds within other search engines, directories, and portals, and we can see evidence of this by looking at AOL.com and their decision to switch to pure Inktomi results.

It is not feasible for the directory to be able to compete with a search engine like Google or a directory like Yahoo. Google is on top because it stays on the edge, keeping a leg up on the competition by introducing new technology, having clean and fresh results that are relevant to the search, and keeping good relationships with the Internet community. While Yahoo!'s customer service is not widely known as the best in the industry, they employ a swift inclusion process, regular updates, fresh results, and a partnership with Google that allows them access to great secondary results.

So, how does ODP begin to climb the ladder that leads back to its success?

Each search engine and directory has a rough schedule that they adhere to when it comes to updates, inclusion times, and database or directory refreshes and cleaning. If ODP were to implement a timing policy or structure, it would help the web masters and Internet professionals to work with the Open Directory Project and to keep the directory fresh, full of new sites, and clean up the dead links.

Another thing that ODP could do to improve their standing is to increase their reach within the Internet world. Establishing partnerships with other search engines and directories to display their results would increase the need to focus and pay more attention to ODP.

It is not a secret that the Open Directory Project needs to make some changes to return to the pedestal that it once was on as the largest human-edited, free directory. Let's see it!


April 5, 2002





Andrew is the Director of Search Engine Optimization Operations for a market leading company. Since the Fall of 2001, Andrew has served as the moderator of the Search Engine Promotion forum within Webmasterworld.com - a leading online community of web professionals. Andrew's articles appear regularly on informative sites including Search Engine Guide, SEO Today, SitePoint and others.






Comments closed after 30 days to combat spam.


Search Engine Guide > Andrew Gerhart > Open Directory Project Search Results and ODP Status