The Great Giveaway
AMAZON, EBAY, AND GOOGLE ARE OPENING UP THEIR BILLION-DOLLAR DATA TROVES. HERE'S WHY THEY'RE DOING IT--AND HOW YOU CAN TAKE ADVANTAGE.
By Erick Schonfeld

(Business 2.0) – The meeting had dragged on for more than an hour, and a restless Jeff Bezos had heard enough. Three years ago, in one of Amazon's Seattle offices, the CEO had rounded up 15 or so senior engineers and managers to tackle a question buzzing inside the company: Should Amazon bust open the doors of its most prized warehouse, containing its myriad databases, and let an eager world of programmers and entrepreneurs scavenge for parts?

For several years, scores of outsiders had been knocking on Amazon's door to gain access to the underlying data that powers the $7 billion online retailer: product descriptions, prices, sales rankings, customer reviews, inventory figures, and countless other layers of content. In all, it was a data vault that Amazon had spent more than 10 years and a billion dollars to build, organize, and safeguard.

So why on earth would Bezos suddenly hand over the keys? Because in the hands of top Web innovators, some at the meeting argued, Amazon's data could be the dynamo of new websites and businesses that would expand the company's already gigantic online footprint and ultimately drive more sales. Others worried about the risks. A free-for-all, one manager warned, would "change our business in ways we don't understand."

Bezos ended the debate with characteristic gusto. He leaped from his seat, aping a flasher opening a trench coat. "We're going to aggressively expose ourselves!" he declared.

Today, there's considerable reason to cheer Bezos's exhibitionist move. Since the company opened up its files in 2002 under the auspices of a project called Amazon Web Services, more than 65,000 programmers, businesses, and other looky-lous have tapped into the data. With it, they're building moneymaking websites, new online shopping interfaces, and innovative services for thousands of Amazon's independent sellers. Many have become Bezos's most ambitious business partners overnight. "Two years ago this was an experiment," says Amazon's engineering chief, Al Vermeulen. "Now it's a core part of our strategy."

And that's just at Amazon. A year after Bezos declared digital glasnost at Amazon, eBay chief executive Meg Whitman answered a similar cry from eBay's developer community, opening the $3 billion company's database of 33 million weekly auction items to the technorati. Some 15,000 developers have since registered to use those listings and other software features. Already, 41 percent of eBay's listings are uploaded to the site using software that takes advantage of these newly accessible resources. At Google, too, the concept is finding its legs: The company parcels out some of its search-results data and recently unlocked access to its desktop and paid-search products. Now dozens of Google-driven services are cropping up, from custom Web browsers to graphical search engines.

What's behind the open-door policies? True to their pioneering roots, Bezos, Whitman, and the Google boys are pushing their companies into what they believe is the Web's great new beyond: an era in which online businesses operate as open-ended software platforms that can accommodate thousands of other businesses selling symbiotic products and services. Says longtime tech-book publisher Tim O'Reilly, "We can finally rip, mix, and burn each other's websites."

Most people think of Amazon as the world's largest online retailer, or "earth's biggest bookstore," as Bezos called it in the startup days. Inside the company, those perceptions are decidedly old school. "We are at heart a technology company," Vermeulen says. He and Bezos have begun to view Amazon as simply a big piece of software available over the Web. "Amazon.com is just another application on the platform," Vermeulen says.

That may sound like Burger King calling the Whopper just another burger. But thanks to the emergence in recent years of new software standards, known collectively as Web services, it's no longer simply some new-economy conceit. Using these standards (such as extensible markup language, or XML), websites today can swap functions, features, and data with one another like never before, all in highly automated fashion. Increasingly, Amazon, eBay, and Google are behaving like competing software platforms that can be programmed as easily as the operating system on your PC. "All the rules are changing," O'Reilly says. "The players who stick to the old rules are going to be screwed."

Eric von Hippel, a business professor at MIT's Sloan School of Management, explains those old rules: "We come from a culture where if you invested in it, you kept it. That was your competitive advantage." The rise of open-source software certainly challenged that notion. The rise of open data and Web services goes even further, holding out the promise of automating the links between online businesses. In geek-speak, such links are known as application programming interfaces, or APIs, through which more and more companies are revealing their vital data. As Vermeulen says, "Those that succeed have to think about removing walls instead of putting them up."

For Amazon, there's some evidence to support that logic. Of the 65,000 people and companies that have signed up to use Amazon's free goodies, about a third have been tinkering with software tools that help Amazon's 800,000 or so active sellers. One of the most clever is ScoutPal, a service that turns cell phones into mobile bar-code scanners. "It's like a Geiger counter for books," founder Dave Anderson says. He came up with the idea a couple of years ago when his wife, Barbara, who sells books on Amazon, would lug home 50 pounds of titles from garage sales, only to discover that she'd paid too much for many of them to make any money. Anderson wrote an application that works in tandem with an attachable bar-code scanner. Barbara either scans in books' bar codes or punches in their 10-digit ID numbers. Then she can pull down the latest Amazon prices for the books and calculate her likely profit margin--before she pays for the inventory. Anderson says his wife's sales have since tripled to about $100,000 a year, and her profit margins have jumped from 50 percent to 85 percent. And he's now bringing in six figures too: ScoutPal has more than 1,000 subscribers, each paying $10 a month.

Other tools are also gaining traction. Software programs like SellerEngine help merchants on the main site to upload their inventory, check prices, and automate interactions such as adding new listings. Meanwhile, software from AssociatesShop.com lets thousands of other website operators--there are more than 900,000 of these so-called Amazon associates--create customized storefronts that link back to Amazon, generating new sales for Bezos and commission revenue for the associates.

Even the right-brain enthusiasts in the developer camp have landed on small but marketable products. For instance, there's Liveplasma, a visual search engine for music and movies. Frédéric Vavrille, a 31-year-old programmer in Paris, created the site two years ago. Type in the name of a band or artist and a map of colored spheres representing similar entertainers emerges onscreen; the entry "Johnny Cash," for example, brings up orbs of varying sizes containing Bob Dylan, Merle Haggard, Willie Nelson, and others. Developers experimented with similar ideas years ago by using software bots that "scraped" various sites for HTML links, but each time the sites changed their webpages, the links would break and the bots would need to be reprogrammed. Now, with direct data feeds from Amazon, Vavrille can automate the process. Liveplasma constantly mutates according to what people are buying.

There's more at work here than visual sleight of hand: Click on an album listed in an artist's discography and you're sent to Amazon to browse or buy; Vavrille collects an associates commission on the transaction if you make a purchase. While he says he hasn't made much money yet from the site, it was promising enough for him to quit his job as a software consultant last summer. After executives at MP3.com checked it out in 2004, they licensed Liveplasma's software for MP3.com's Musicvine section, which is powered by its own database of musical genres.

For the near term, the biggest benefit to Amazon of letting folks like Anderson and Vavrille tinker with its platform is that it gets experimental R&D for free. "We can try to build all the applications for sellers ourselves," Vermeulen says, "or we can build a platform and let others build them." Adds Bezos, "Right now we just want to get people to use the guts of Amazon in ways that surprise us."

The experimentation at eBay has been just as ambitious. The company says that more than 1,000 new software applications have emerged from its 15,000 or so registered developers. As with Amazon, the most popular are apps that help sellers automate the process of listing items on eBay or displaying them on other sites. Many of these outfits, such as ChannelAdvisor (itself a multimillion-dollar business), Marketworks, and Vendio, offer auction-listing software or services to eBay sellers. (See "Corporate America's New Outlet Mall," April 2004.) Jeff McManus, eBay's chief of platform evangelism, marvels at the benefits. "Sellers who use our APIs become at least 50 percent more productive than those who use the website itself."

The data links also let companies create storefronts filled with their inventory, while making transactions over eBay's network. One example is Las Vegas-based SuperPawn, which runs a chain of 46 pawnshops in Arizona, California, Nevada, Texas, and Washington. The company (recently acquired by the larger pawnshop operator Cash America International) uses eBay's APIs to automatically upload the latest pawned items from its physical stores to eBay. The system already generates more than 5 percent of SuperPawn's $40 million in annual sales and thousands more transactions for eBay.

While Amazon and eBay have unlocked enough databases to allow people to use them to build their own Web storefronts, Google is taking baby steps. What's available? Developers may grab 1,000 search results a day for free, but anything more than that requires special permission. Even so, several new Google Web browsers that organize search results into handy visual maps have popped up, including ones built by TouchGraph and the Hive Group. (Both have also created similar Amazon browsers and use their applications to demonstrate their core offering, so-called visualization software.)

"In Google's case it is really an experiment," says Jason Bloomberg, an industry analyst with research firm ZapThink. After all, giving away an unlimited number of search results for free would potentially allow someone to set up a rival search engine without the Google ads. For that reason, the company keeps a tight lid on the search data it gives away. Access to its ad data is a different story. In January, Google opened up APIs to its AdWords paid-search service, allowing marketers to automate their Google ad campaigns. Says Nelson Minar, the software engineer who heads up Google's Web services, "I want creative hacker-programmer types to do cool stuff with Google."

Not to be left behind, Yahoo, too, has locked arms with the freeloaders. Since 2001 its Overture subsidiary has provided data similar to the AdWords data offered by Google; in March the company announced that it will also provide access to its search, news, photo, and video databases--but only for noncommercial purposes.

By opening up parts of their platforms early and voluntarily, Amazon, eBay, and others increase the chances that developers and businesses will organize around their systems. Yet all companies with platform aspirations walk a fine line between fostering innovation and shooting themselves in the foot. Amazon, for example, is carefully managing what it reveals and what it doesn't. Some information, such as individual customer data and Amazon's minute-by-minute revenue and profit figures, is strictly off-limits.

For many other successful online companies, the giveaway strategy makes little or no sense. Match.com COO Joe Cohen, for example, has no intention of giving up the company's 15 million individual profiles, even on a limited basis. For one thing, Match.com's database is filled with very personal information, which raises all sorts of privacy issues. Besides, as Cohen says, "our competitive advantage is we think we have the highest-quality database of singles online. We're not going to give it up to the market."

For some, then, giving away the store carries more risk than potential reward. But companies like Amazon, eBay, and Google that want to become mass-market technology platforms can't do everything themselves. Of all the people and companies that use the free data, says von Hippel, "some fraction will increase sales, but you can't figure out who will and who won't." So a platform contender needs to give its data to all comers. University of California at Berkeley economist Hal Varian points out another advantage: "I can look at all of these innovative things my customers are doing and build the more interesting ones into my own service."

That's not to say any of these platforms is likely to become the new Microsoft. A killer app may indeed emerge from the era of open data, but when it does, say O'Reilly and others, it most likely won't be based solely on Amazon or eBay or Google but will suck in data from all three, and probably others.

What might this look like? Consider Pluck, a startup that's created a superbrowser that can search Amazon, eBay, Google, and the blogosphere--all within the same interface--then remember those searches and keep them constantly updated. Co-founder Andrew Busey hopes to make money from Amazon and eBay referral fees, as well as Google paid-search ads that appear on Pluck. Then there's A9, Amazon's own search engine subsidiary. It's built on APIs from Amazon, Google, Guru.net, and other sites. Applications like these suggest that there probably won't be a single platform winner, but rather a cluster of hublike Web platforms that collectively feed thousands of smaller niche businesses.

The "rip, mix, burn" philosophy could turn out to be just as liberating for websites as it has been for music. But the real killer app won't arrive until somebody figures out how to turn all this free data into serious money--for platforms and piggybackers alike.