Search Engine Guide
Search The Internet: 

Traffick Logo Article provided with permission by
Traffick - The Guide to Portals.
© 2000

The Internet Loves a Content Aggregator
By Andrew Goodman - September 11, 2000

In any dispute which boils down to questions of control and attitudes towards information sharing -- the kinds of disputes which demarcate the divide between those who "get" this medium and those who don't -- history and the cool people are always on the side of the info aggregator, the one who can "go meta" to offer the end user information in the most useful, non-proprietary format.

Ironically, non-proprietary services -- loved by the savvy members of community who like the rush of power they feel when the floodgates of information suddenly open -- often tend to become proprietary in themselves. The cool people are willing to let the "cooler" proprietary people be proprietary as long as they're exploding someone else's grip on what is proprietary. Hence Red Hat Software and its eye-popping IPO for tools related to an open source operation system.
The best aggregators give users what they want, how they want it

Most of us are aware that a company such as Yahoo! - particularly in its personalized portal incarnation as My Yahoo! - is an info aggregator. So is AOL. It was an online information service even before there was a world wide web. Not so surprising that history was kind.

The attraction is that information is supposed to be nicely packaged and consistent in its presentation. You don't have to go hunting all over for it; someone else puts it in a handy format. Infrastructure companies like Infospace and Marketguide lurk below the surface, helping companies like AOL do their job better. Portal building companies now abound, offering software and services to help those designing information resources and corporate portals so that they don't have to reinvent the wheel of cobbling together all of this news and info. Examples include Go2Net Private Label Portal, Epicentric, VerticalNet,, and Yahoo! Corporate.

Slice it, dice it, and make it useful

The pace of innovation never slows, apparently. Information services like Reuters face a challenge from news aggregators such as Moreover, which combines editorial expertise with filtering technology to monitor breaking news from a huge range of news sources broken down by hundreds of categories. Putting together a weekly news digest about portals is made easier for Traffick by the fact that Moreover does this work in categories we cover, but we add another layer, sifting through the news and picking a selection of items and occasionally adding comments. Reader feedback indicates that there are always going to be differences of opinion about the usability of this format, but the key is, it's probably a lot better than what you could get a year ago -- and a lot more timely, too. The best part is, our friends at Moreover don't fret at the way in which we use this information. Their brand, and tracking info, is built into every news link anyway. They actually seem to want us to use the information as much as possible. How refreshing! (By the way, Reuters are no fools: they are now an investor in Moreover.)

"Is metasearch legal?," and other silly questions
At some point, aggregation alone fails to be novel anymore, and people move on to a different aggregator, and sometimes, back to a proprietary source if it's actually better or if they simply crave the "raw material." But the point is that things like metasearch engines ( Metacrawler, Ixquick); meta-auctions (AuctionRover); shopping comparison sites and product rating services (epinions, Deja, buybuddy); music download and peer-to-peer sharing tools (Napster, Infrasearch); "peanut gallery plugins" (Third Voice); etc. really do bring information to the average user in a way that they want it presented. And the flow goes in every direction. Sometimes, the power of the Internet allows users to provide feedback about products or company conduct in such a way that the whole world, or at least other users of the product, can see it ( Sometimes disgruntled former and current employees will air a firm's dirty laundry very publicly. This may not be nice, but one can't imagine how to stop it. One should probably try to stop imagining how to stop it. (Assuming, for the most part, that no laws are being broken.)
Some such phenomena, of course, cross over the line of legality or somebody's idea of morality, and most use data in ways that feel uncomfortable to the data's originators. But like any great revolution -- and the Internet is a great revolution -- you can't hold back the forces of progress once the tide turns. It's folly to hope that you can control the flow of information to ensure that it gets used only in the way that you want people to use it. The people have too many tools at their disposal. It's been going on since Martin Luther. It's just that now the tools are a lot better, and in a lot more hands.
Witness the quietly loyal following of habitual test-drivers like Consumer Reports Magazine. And you may have heard of a popular little aggregator called Reader's Digest.
We've got a couple of interesting ideas up our sleeve for aggregating other people's data about portals which we hope users will find useful. We trust the originators won't get their noses too far out of joint, but you never know.
You can't be just a little bit online: the case of Ticketmaster
In the 1980's, the cool kids danced to a song called Free Nelson Mandela without any serious expectation that such an event would come to pass. But it happened! Mandela was not only freed, but became President of South Africa.
Will they ever learn? Nowadays, companies who make their living selling things like concert tickets on the Internet -- the Internet, mind you, open to all comers, not some private corporate network -- fret that information aggregators are "deep linking" to internal pages within their directories rather than taking users to the company home page, where they're "supposed to go."
It may seem likely, on the surface, that the forces who seek to control the flow of "their" information about their own product offerings will rule the day, backed by judges, money, and power. Sometimes they will. But legal victories in the short term may obscure the fact that the tide of history is against those who don't understand the principle of markets as conversations.
Cyberspace belongs to revolutionaries
Cyberspace belongs to revolutionaries. What seems like a subculture today - something the kids are monkeying around with - very often goes mainstream. I don't know about you, but my aunt has a Hotmail account. My mom tried AOL, until her smart son told her that she could use a free ISP called AltaVista Free, powered by, with 3 local access numbers in her suburban Ontario city.
AOL had this much figured out: they knew that my Mom would try the AOL free trial because she had one of their ubiquitous disks lying around, and after all, it's "so easy to use, no wonder it's #1." What they failed to factor in was that Mom would talk to son, and son would show her that it's pretty darn easy to download and set up the free dialup service! Judging by their popularity, these free ISP's have undergone some major viral growth.
About a year ago, I asked my Dad, not a frequent Internet user, whether he'd use a free ISP instead of paying $20 a month, with the main trade-off being that he would have to use a browser with an advertising bar locked onto the top. He just looked at me without blinking, before finally stating what he felt to be the obvious: "Of course I'd use it. For the amount of time I'm online, I can look at a few ads to save $20 a month." This is not a cheap man. This is all about being given a choice, after a lifetime of suffering at the hands of utility monopolies, to pay nothing for access to information, with no appreciable downside. Free info, all you can eat. Hardly any catch.
Apparently a lot of people feel like my Dad. Canada's leading grocer, Loblaw, is setting up a free ISP under its President's Choice brand, also powered by And that Bluelight thing - essentially a free ISP offered by Yahoo! and KMart in partnership - is going gangbusters.
Consumer choice is not some weird subculture
The point: "the attitude" is now mainstream. The "Napster thing" and all the rest is not "just something the kids are monkeying around with." You give people a way to circumvent the old rules. It doesn't matter if they're 16 or 61; ask them if they'd actually want to do that, and they'll look at you like: "you're kidding, right?"
Less than ten years ago, stockbrokers used to pore over charts printed in thick books, and get all of their news and information from proprietary terminals and information services such as Bloomberg. If you had told them that someday, ordinary investors would have ten times that power in their living rooms, and that the brokers, too, would do some of their stock research on Yahoo! - and not just on weekends - they would have laughed in your face, if they bothered to listen to you at all. Who's laughing now?
The willingness of consumers to embrace companies and media outlets like Apple, Netscape, discount online brokerages, Hotmail, Google, Napster, Metacrawler, the Drudge Report, Silicon Investor, Jennicam, The Anonymizer, eBay, and many more can be boiled down to setting up a framework for letting people do what they couldn't do before, and then getting out of the way.
Sometimes, what people want is incredibly visceral and simple. I've learned from reader correspondence and server logs that one of the biggest problems people face is encapsulated in keyword phrases like "cannot get into my email," "where is my email," and "retrieving email."
Listening to recordings (or purported recordings) of people calling in when their cable service is interrupted can be an amusing way to spend time. "I CANNOT WATCH MY AUTOMAG IF I DO NOT HAVE THE SERVICE!!!!" is the printable part of what one legend of disgruntled customerdom screams into the cable company's voicemail. What people want is often simple. If you could figure out a way to help that man get back at his cable company, and then asked him if he really wanted to, he'd look at you like you were from outer space. It's not even a question. It's like, "where do I sign up?"
For better or for worse, businesses like casinos have known this for a long time. Set up the framework, and stay the hell out of the way. Hook people up, and don't make it hard. Or as marketing guru Seth Godin would put it, "make it smooth." Just look at who is hunkered over the slot machines for ten hours at a stretch. Now try to stop it!
All of this adds up to the need to seriously consider the possibility that many Internet economy leaders in their categories, from portal giants AOL, MSN, and Yahoo!, to e-commerce behemoths like Amazon, could be toast tomorrow. It all depends on how smart they are about staying out of the way, and letting the information flow. It's tough to imagine AOL changing its stripes too much on this score. In the "I can't believe they're running an information service" category is definitely this recent note from a major investment newsletter, Internet Insight by Steve Harmon: "AOL has been blocking our e-mails to you for the past few months because of a misconfiguration, but the situation has finally been corrected." Months! You get the feeling AOL users find it more difficult to accomplish basic things like, you know, receiving the information everyone else is receiving. Harmon himself has likened AOL to a digital nation. So far, this nation rules global cyberspace, but it won't be the first mighty empire that has fallen.
Final caveat: like plagiarism, aggregation is best done subtly

At some point, someone has to originate something. If I didn't write this article, for example, there would be nothing to aggregate. And there have been many hamfisted schemes like simply trapping other companies' content within a browser frame. A service called ZineZone was a clumsy attempt to allow ordinary people to act as thieves-cum-aggregators. The interface was clumsy, and no one could figure out the point, so it more or less died. It may well be that Napster or some of the Napster cousins are of this ilk. You can't just openly thieve stuff, and if you're going to aggregate it, you'll have to be creative, and maybe even figure out a way to get money flowing from users to content originators. There is such a thing as flying too close to the sun, crossing the line from aggregation or audaciousness to out-and-out lawlessness. It's generally acknowledged that an online means of pirating TV signals - Canada-based iCraveTV - was in this category. Since the iCraveTV site is not presently operational, you'll have to check out this Wired News article on the matter: They Crave Suits v. ICraveTV. While the rights of content originators should be respected, the ongoing rush of events will not be friendly to those who don't also respect the role of aggregators.

When it comes to giving stuff away -- be it information or online access -- everything can't, and won't, be literally free. It's the attitude that counts. Figure out a way to let people get around an arrogant control freak monopolist, and half the time, they'll probably pay you double, and thank you for the opportunity.


Can you afford not to know this stuff?
Sign up for the weekly Traffick Newsletter and stay informed!

Traffick Weekly
Are you the smartest Internet expert on the block? If not, you haven't been reading the Traffick Weekly!

The Traffick Weekly Newsletter is the fastest way to keep on top of key Internet trends. It offers a wide ranges of sources, and is edited to focus only on what you need. Enter your e-mail below to subscribe now, or read on to find out why the Traffick Weekly is a must-read!

Subscribe to the Weekly

Traffick Monthly
Stay up to date on the topics that are important to you. Subscribe to the monthly Traffick newsletter by entering your e-mail address below and clicking Join!

The monthly offers exclusive content and links to resources that you won't find anywhere else. Traffick is an Editor's Choice of Topica!

Subscribe to the Monthly

Portal Builders
Are you responsible for a portal project? Does your company provide portal building solutions? Just interested in issues related to building a portal? This discussion list will facilitate ongoing interaction amongst the community of professionals who are responsible for building portals.

Don't reinvent the wheel! Become acquainted with best practices by tapping into this valuable B2B resource. Portal solutions providers: you are welcome to introduce your company to the list. Others: please post your questions and comments to the list.

Subscribe to Portal Builders