Search Engine Guide
Search The Internet: 

Traffick Logo Article provided with permission by
Traffick - The Guide to Portals.
© 2000

Beyond ODP : The Hardin Meta Directory
GUEST COLUMN by Eric Rumsey - July 5, 2000

At the Hardin Meta Directory of Internet Health Sources, we take the Open Directory "volunteer editor" approach one step further -- Instead of inviting people to make directory pages on our site, as Open Directory does, we seek out ongoing directory sites done by specialists in the field, and make links to them from Hardin Meta Directory pages.

We got the idea to start the Hardin Meta Directory in 1996, when we were looking for the best lists of links in several areas of clinical medicine. Like many people, then and now, we assumed that Yahoo was THE PLACE TO GO to find links. But when we did a bit of digging, we found that was far from the case -- There were actually several directory sites that had much more comprehensive link-lists than Yahoo. These were generally done by people who specialize in health/medicine, and have much more time to devote to their directory pages than the staff of an encyclopedic directory site like Yahoo. An example: The Yahoo page for cardiology lists about 120 links; some of the specialist-produced lists in the Hardin Meta Directory have 500-1000 links.

From the time it was launched, the Hardin Meta Directory has been highly successful, with citations in the popular press (Money, USA Today, Consumer Reports), as well as in professional journals (British Medical Journal, Journal of Family Medicine). It's also widely-cited on important pages around the web, as shown by its high ranking in Google searches.

A meta directory is a directory of directories

We call our site a "meta directory" because it's a "directory of directories." This has always seemed to us to be a natural approach to organizing the web. Experienced web searchers soon learn that one of the best ways to jump-start their research efforts is to find a good list of links in the subject being researched. Bruce Maxwell, in his book, How to Find Health Information on the Internet, sums this up in his endorsement of the Hardin Meta Directory: "Directory sites are a godsend, but tracking them down can be difficult -- Hardin solves this problem in the health field."

Why is it so hard to find directories? One reason we've discovered is that people who maintain directory sites do a poor job of providing links to other directory sites. If they provide links to other directories at all, they're often poorly maintained, commonly with a high proportion of dead links. This is really not so surprising. After all, if you run a directory site, there is naturally the assumption (probably unconscious in most cases) that of course your directory is the best one, so why would anyone want to go to a competing directory anyway? It's just the opposite for us -- Instead of being in a competitive relationship with directory sites, we have a collaborative relationship with them. Our success is built upon the existence of directory sites, so we communicate frequently with people who run them. One of our greatest satisfactions is ferreting out little-known directory sites, and making links to them from Hardin Meta Directory pages, often serving as their main foothold on the web, and often encouraging them along in their fledgling directory-building.

A many-sided human view of the Web

The advantage of directories, as opposed to search engines, of course, is that directories are done by humans, and thus are products of human judgement. The great advantage of the meta directory approach is that it gives not one, but many, human views of the Net. With the turbulent, ever-changing nature of information on the Net, it's obviously an advantage to have a variety of viewpoints.

The original purpose of the Hardin Meta Directory -- to lead the user to the most comprehensive directories -- has developed a bit further as the site has evolved. In looking at directories, in addition to their comprehensiveness, we now evaluate their quality of upkeep and maintenance. A key element of this evaluation process is that we've developed methods to quickly measure the proportion of good links at each directory site. Sites that meet our standards are given our "Clean Bill of Health" award. It has been especially gratifying to see that, not only is our site ranked highly in Google searches, but sites that we rank highly on our pages are also ranked highly by Google. Since Google rankings are based on the frequency and prominence of links, this indicates that other people are choosing to make links to the same sites that we rank highly in our evaluations.

Why aren't there more meta directories?

Since the Hardin Meta Directory has been so successful, we can't figure out why there aren't meta directories in other subject areas. We keep looking, but so far haven't found anyone else using the same approach. The ODP has shown the benefits of using volunteers to build a directory site, but it also has shown the disadvantages -- ODP volunteers tend to be either novices, with little Web experience, on the one hand, or people who have their own ax to grind, and are using their positions in ODP to promote their own sites, on the other hand. The meta directory approach avoids both of these problems. In our experience, a meta directory can be built on the unbiased contributions of people who have already gone through the rudiments of learning their way around the Web.

New search engines may favor meta directories

With the success of Google -- which is based largely on its analysis of the link structure of the Web -- directories, and especially meta directories, are becoming increasingly prominent. As mentioned earlier, Google ranks directories highly, and it ranks our meta directory site especially highly. Future search engines may rank meta directories even more highly -- The Clever project, under development at IBM's Almaden Research Center, is seen by many as the wave of the future in search engines. Like Google, it relies on link structure in ranking search results. But, going one step further than Google, it looks not only at back-links TO the target page, but also forward-links FROM the target page. Sites like the Hardin Meta Directory that are frequently linked from other important pages, and also themselves have a high proportion of links to other important pages will likely rise to the top in Clever searches.

Prabhakar Raghavan, one of the developers of Clever, gives -- from our point of view anyway -- a glowing endorsement for the concept of meta directories, when he says that the creators of directories are "the unsung heroes of the web." We take this to be an eloquent statement of what we're doing, and what other meta directories could do -- Finding those unsung hero directory sites in the dark crevices of the web, and bringing them to the light of day!

So, how about making a meta directory?

We hope we've convinced our readers that the meta directory idea is a good one, and we'd strongly encourage people to think about starting meta directories in fields besides health and medicine. The first thing to consider, of course, is the number of directory sites in the field, which is generally correlated with the volume of Internet activity in the field. As we've learned recently, Google does an excellent job in finding directory sites, so try typing potential subject words in Google searches, and examine the first 100 hits -- If there are good directories in the field, from our experience in health and medicine, it's likely that Google will find them. We would certainly be happy to serve as a consultant for anyone considering a meta directory, so send your ideas.

For more:

The Hardin Meta Directory [Kudos and links]
How to Find Health Information on the Internet, book by Bruce Maxwell (Washington, DC: Congressional Quarterly Inc, 1998)
Google likes directory sites, Hardin MD Notes, Jan 3, 2000
Google ranks Hardin MD in the top 15, Hardin MD Notes, Feb 24, 2000
The Clever Project
How clever is Clever?, Paul Kedrosky, ForbesDigital, November, 1998

Eric Rumsey is a Librarian/Webmaster at the Hardin Library for the Health Sciences, University of Iowa, and founder and manager of the Hardin Meta Directory. He writes a regular column, Hardin MD Notes on the Net.
Please send comments to Eric Rumsey.


Can you afford not to know this stuff?
Sign up for the weekly Traffick Newsletter and stay informed!

Traffick Weekly
Are you the smartest Internet expert on the block? If not, you haven't been reading the Traffick Weekly!

The Traffick Weekly Newsletter is the fastest way to keep on top of key Internet trends. It offers a wide ranges of sources, and is edited to focus only on what you need. Enter your e-mail below to subscribe now, or read on to find out why the Traffick Weekly is a must-read!

Subscribe to the Weekly

Traffick Monthly
Stay up to date on the topics that are important to you. Subscribe to the monthly Traffick newsletter by entering your e-mail address below and clicking Join!

The monthly offers exclusive content and links to resources that you won't find anywhere else. Traffick is an Editor's Choice of Topica!

Subscribe to the Monthly

Portal Builders
Are you responsible for a portal project? Does your company provide portal building solutions? Just interested in issues related to building a portal? This discussion list will facilitate ongoing interaction amongst the community of professionals who are responsible for building portals.

Don't reinvent the wheel! Become acquainted with best practices by tapping into this valuable B2B resource. Portal solutions providers: you are welcome to introduce your company to the list. Others: please post your questions and comments to the list.

Subscribe to Portal Builders