Rich Get Richer - Why Yahoo, DMOZ, Google and PageRank are Important

By Merrick E. Lozano

Albert-Lazlo Barabasi's book, Linked, points out that "power laws mathematically formulate the fact that in most real networks the majority of nodes have only a few links and that these numerous tiny nodes coexist with a few big hubs."

The Rich Get Richer concept helps explain why Yahoo, DMOZ, and Google are at the center of the web and why Google PageRank plays a large role in determining which web sites become hubs.

Scientists over at Google understand Pareto's Law and they know that 80 percent of links on the web point to only 15 percent of web pages. This 15 percent of web pages are the hubs that guide us to the other 85 percent of the web.

Importance of Google and PageRank

The basis for PageRank is that web sites with higher PageRank confer their PageRank to other web sites through links. This makes it very important for web pages to get links from the right places, the hubs of the web.

By looking at the Nielsen NetRatings Search Engines Ratings (http://searchenginewatch.com/reports/article.php/2156451) you can see the audience reach for three Google powered search properties (Google, Yahoo, AOL) is 76.8%. These impressive stats point to the fact that Google is the nucleus of the web that helps link it all together.

As Google has become the most widely used search engine, its PageRank technology now plays a large role in defining which web sites are also hubs on the web.

For example, during a three-month period one of our company's web site went from having a PageRank of 4 to a PageRank of 6. In that same period it is no coincidence that our overall traffic doubled and 80% of the increase in visitors came from Google, Yahoo, and AOL.

Importance of the Yahoo Directory and DMOZ

With Google and PageRank being so important, getting placed in Google's search results should be a top priority for all web site owners. The faster a web site gets its content included in Google's database the faster it can become a hub, ceterus paribus.

Yahoo and DMOZ are part of the elite directories that can jumpstart a web site's PageRank and help it establish visibility in Google's search results. A Link from DMOZ or the Yahoo directory alone can give a site a PageRank of 4, which is an important number. Only web pages with a PageRank of 4 or higher are displayed in Google's backward links feature. Why? Google probably displays the 15 percent of web pages that are the hubs of the web.

The DMOZ directory generally carries the largest weight of any directory in defining how fast a web site acquires incoming links because it is also the Google Directory and its listings are syndicated to many web sites.

Implications of Preferential Attachment and Linking on the Web

With respect to linking on the web, preferential attachment holds that the probability of linking to a web page (node) has been found to be proportional to the number of links the node has. This means that the earlier a web site builds incoming links the faster it will be linked to.

Preferential attachment does not just hold true for linking on the web, it also describes an actor's ability to gain new roles in Hollywood. The Santa Fe Institute did research on preferential attachment that found that "the probability of scientists collaborating increases with the number of other collaborators they have in common, and that the probability of a particular scientist acquiring new collaborators increases with the number of his or her past collaborators".

Because Google is currently the largest hub on the Internet, establishing links on DMOZ and Yahoo are vital to establishing PageRank and becoming a hub on the web. PageRank is not just a marketing gimmick it helps illustrate which web sites are hubs and which are nodes.

Further Reading

If you are interested in reading more about Google PageRank, networks, power laws, and preferential attachment the following books and papers may be of interest to you.

Linked: The New Science of Networks
Authoritative Sources in a Hyperlinked Environment
(http://www.cs.cornell.edu/home/kleinber/auth.pdf)
This is probably the most notable paper relating to search engines as it discusses power law, preferential attachment and link analysis.

The Anatomy of a Large-Scale Hypertextual Web Search Engine
(http://www-db.stanford.edu/~backrub/google.html)
While grad students at Stanford, the founders of Google wrote about their project Google and the technology behind it including how they calculated PageRank.

Clustering and preferential attachment in growing networks
(http://www.santafe.edu/sfi/publications/Working-Papers/01-03-021.pdf)
This article from the Santa Fe Institute also discusses power laws and preferential attachment.