The Internet Loves a Content Aggregator
By Andrew Goodman - September 11, 2000
In any dispute which boils down to questions
of control and attitudes towards information sharing --
the kinds of disputes which demarcate the divide between those
who "get" this medium and those who don't -- history and the
cool people are always on the side of the info aggregator, the one who
can "go meta" to offer the end user information in the most useful,
Ironically, non-proprietary services --
loved by the savvy members of community who like the rush of power
they feel when the floodgates of information suddenly open -- often
tend to become proprietary in themselves. The cool people are willing
to let the "cooler" proprietary people be proprietary as long as
they're exploding someone else's grip on what is proprietary. Hence
Red Hat Software and its eye-popping IPO for tools related
to an open source operation system.
The best aggregators give
users what they want, how they want it
Most of us are aware that a company such as Yahoo!
- particularly in its personalized portal incarnation as My Yahoo!
- is an info aggregator. So is AOL. It was an online information
service even before there was a world wide web. Not so surprising
that history was kind.
The attraction is that information is supposed
to be nicely packaged and consistent in its presentation. You don't
have to go hunting all over for it; someone else puts it in a handy
format. Infrastructure companies like Infospace
and Marketguide lurk below the surface, helping companies like
AOL do their job better.
Portal building companies now abound, offering software and
services to help those designing information resources and corporate
portals so that they don't have to reinvent the wheel of cobbling
together all of this news and info. Examples include Go2Net Private
Label Portal, Epicentric, VerticalNet, mydestinations.net, and Yahoo!
Slice it, dice it, and
make it useful
The pace of innovation never slows, apparently.
Information services like Reuters face a challenge from news aggregators
such as Moreover,
which combines editorial expertise with filtering technology to
monitor breaking news from a huge range of news sources broken
down by hundreds of categories. Putting together a weekly
news digest about portals is made easier for Traffick by the
fact that Moreover does this work in categories we cover, but
we add another layer, sifting through the news and picking
a selection of items and occasionally adding comments. Reader
feedback indicates that there are always going to be differences
of opinion about the usability of this format, but the key is, it's
probably a lot better than what you could get a year ago --
and a lot more timely, too. The best part is, our friends at
Moreover don't fret at the way in which we use this information.
Their brand, and tracking info, is built into every news link anyway.
They actually seem to want us to use the information as much as
possible. How refreshing! (By the way, Reuters are no fools: they
are now an investor in Moreover.)
"Is metasearch legal?,"
and other silly questions
At some point, aggregation alone fails to be
novel anymore, and people move on to a different aggregator, and
sometimes, back to a proprietary source if it's actually better
or if they simply crave the "raw material." But the point is that
things like metasearch engines (
Metacrawler, Ixquick); meta-auctions (AuctionRover);
shopping comparison sites and product rating services (epinions,
music download and peer-to-peer sharing tools (Napster,
Infrasearch); "peanut gallery plugins" (Third
Voice); etc. really do bring information to the average
user in a way that they want it presented. And the flow goes in
every direction. Sometimes, the power of the Internet allows
users to provide feedback about products or company conduct in such
a way that the whole world, or at least other users of the product,
can see it (SympaticoUsers.org).
Sometimes disgruntled former and current employees will air a firm's
dirty laundry very publicly. This may not be nice, but one can't
imagine how to stop it. One should probably try to stop imagining
how to stop it. (Assuming, for the most part, that no laws are being
Some such phenomena, of course, cross over the
line of legality or somebody's idea of morality, and most use data
in ways that feel uncomfortable to the data's originators. But like
any great revolution -- and the Internet is a great revolution --
you can't hold back the forces of progress once the tide turns.
It's folly to hope that you can control the flow of information to
ensure that it gets used only in the way that you want people
to use it. The people have too many tools at their disposal.
It's been going on since Martin Luther. It's just that now
the tools are a lot better, and in a lot more hands.
Witness the quietly loyal following of habitual
test-drivers like Consumer Reports Magazine. And you may have heard
of a popular little aggregator called Reader's Digest.
We've got a couple of interesting ideas up our
sleeve for aggregating other people's data about portals which
we hope users will find useful. We trust the originators won't get
their noses too far out of joint, but you never know.
You can't be just a little
bit online: the case of Ticketmaster
In the 1980's, the cool kids danced
to a song called Free Nelson Mandela without any serious expectation
that such an event would come to pass. But it happened! Mandela
was not only freed, but became President of South Africa.
Will they ever learn? Nowadays, companies
who make their living selling things like concert tickets on the
Internet -- the Internet, mind you, open to all comers,
not some private corporate network -- fret that information aggregators
are "deep linking" to internal pages within their directories
rather than taking users to the company home page, where they're
"supposed to go."
It may seem likely, on the surface, that the
forces who seek to control the flow of "their" information about
their own product offerings will rule the day, backed by judges,
money, and power. Sometimes they will. But legal victories in the
short term may obscure the fact that the tide of history is against
those who don't understand the principle of markets
Cyberspace belongs to revolutionaries
Cyberspace belongs to revolutionaries. What
seems like a subculture today - something the kids are monkeying
around with - very often goes mainstream. I don't know about
you, but my aunt has a Hotmail account. My mom tried AOL, until
her smart son told her that she could use a free ISP called
AltaVista Free, powered by 1stup.com,
with 3 local access numbers in her suburban Ontario city.
AOL had this much figured out: they knew that
my Mom would try the AOL free trial because she had one
of their ubiquitous disks lying around, and after all, it's "so
easy to use, no wonder it's #1." What they failed to factor in was
that Mom would talk to son, and son would show her that it's pretty
darn easy to download and set up the free dialup service! Judging
by their popularity, these free ISP's have undergone some major
About a year ago, I asked my Dad, not a frequent
Internet user, whether he'd use a free ISP instead of paying $20
a month, with the main trade-off being that he would have to use
a browser with an advertising bar locked onto the top. He just looked
at me without blinking, before finally stating what he felt to be
the obvious: "Of course I'd use it. For the amount of time
I'm online, I can look at a few ads to save $20 a month." This is
not a cheap man. This is all about being given a choice, after a
lifetime of suffering at the hands of utility monopolies, to pay
nothing for access to information, with no appreciable downside.
Free info, all you can eat. Hardly any catch.
Apparently a lot of people feel like my Dad.
Canada's leading grocer, Loblaw, is setting up a free ISP under
its President's Choice brand, also powered by 1stUp.com. And that
Bluelight thing - essentially
a free ISP offered by Yahoo! and KMart in partnership - is
Consumer choice is not
some weird subculture
The point: "the attitude" is now mainstream.
The "Napster thing" and all the rest is not "just something the
kids are monkeying around with." You give people a way to circumvent
the old rules. It doesn't matter if they're 16 or 61; ask them
if they'd actually want to do that, and they'll look at you like:
"you're kidding, right?"
Less than ten years ago, stockbrokers used to
pore over charts printed in thick books, and get all of their news
and information from proprietary terminals and information services
such as Bloomberg. If you had told them that someday, ordinary investors
would have ten times that power in their living rooms, and that
the brokers, too, would do some of their stock research on
Yahoo! - and not just on weekends - they would have laughed in your
face, if they bothered to listen to you at all. Who's laughing now?
The willingness of consumers to embrace companies
and media outlets like Apple, Netscape, discount online
brokerages, Hotmail, Google, Napster, Metacrawler, the
Drudge Report, Silicon Investor, Jennicam, The Anonymizer, eBay,
and many more can be boiled down to setting up a framework
for letting people do what they couldn't do before, and then getting
out of the way.
Sometimes, what people want is incredibly visceral
and simple. I've learned from reader correspondence and server logs
that one of the biggest problems people face is encapsulated in
keyword phrases like "cannot get into my email," "where
is my email," and "retrieving email."
Listening to recordings (or purported recordings) of
people calling in when their cable service is interrupted can be
an amusing way to spend time. "I CANNOT WATCH MY AUTOMAG IF I DO
NOT HAVE THE SERVICE!!!!" is the printable part of what one
legend of disgruntled customerdom screams into the cable company's
voicemail. What people want is often simple. If you could figure
out a way to help that man get back at his cable company, and then
asked him if he really wanted to, he'd look at you like you were
from outer space. It's not even a question. It's like, "where do
I sign up?"
For better or for worse, businesses like casinos
have known this for a long time. Set up the framework, and stay
the hell out of the way. Hook people up, and don't make it hard. Or
as marketing guru Seth Godin
would put it, "make it smooth." Just look at who is hunkered
over the slot machines for ten hours at a stretch. Now try to stop
All of this adds up to the need to seriously
consider the possibility that many Internet economy leaders
in their categories, from portal giants AOL, MSN, and Yahoo!, to
e-commerce behemoths like Amazon, could be toast tomorrow.
It all depends on how smart they are about staying out of the way,
and letting the information flow. It's tough to imagine AOL changing
its stripes too much on this score. In the "I can't believe they're
running an information service" category is definitely this recent
note from a major investment newsletter, Internet Insight by Steve
Harmon: "AOL has been blocking our e-mails to you for the past few
months because of a misconfiguration, but the situation has finally
been corrected." Months! You get the feeling AOL users find it
more difficult to accomplish basic things like, you know,
receiving the information everyone else is receiving. Harmon himself
has likened AOL to a digital nation. So far, this nation rules
global cyberspace, but it won't be the first mighty empire that
Final caveat: like plagiarism,
aggregation is best done subtly
At some point, someone has to originate something.
If I didn't write this article, for example, there would be nothing
to aggregate. And there have been many hamfisted schemes like simply
trapping other companies' content within a browser frame. A service
called ZineZone was a clumsy
attempt to allow ordinary people to act as thieves-cum-aggregators.
The interface was clumsy, and no one could figure out the point,
so it more or less died. It may well be that Napster or some of
the Napster cousins are of this ilk. You can't just openly thieve
stuff, and if you're going to aggregate it, you'll have to be creative,
and maybe even figure out a way to get money flowing from users
to content originators. There is such a thing as flying too close
to the sun, crossing the line from aggregation or audaciousness
to out-and-out lawlessness. It's generally acknowledged that an
online means of pirating TV signals - Canada-based iCraveTV - was
in this category. Since the iCraveTV site is not presently operational,
you'll have to check out this Wired News article on the matter:
Crave Suits v. ICraveTV. While the rights of content originators
should be respected, the ongoing rush of events will not be friendly
to those who don't also respect the role of aggregators.
When it comes to giving stuff away -- be it
information or online access -- everything can't, and won't, be
literally free. It's the attitude that counts. Figure
out a way to let people get around an arrogant control freak monopolist,
and half the time, they'll probably pay you double, and thank you
for the opportunity.
Can you afford
not to know this stuff?
up for the weekly Traffick Newsletter
and stay informed!