There's no success like failure

Letting crowds organize themselves may be the most efficient form of research - even if most proposed organizations fail to materialize.

More from Fortune

Will Mmmhops be a hit? NBA confirms L.A. Clippers sale to ex-Microsoft CEO Steve Ballmer FBI and SEC probe into Carl Icahn and golfer Phil Mickelson

FORTUNE 500 Current Issue Subscribe to Fortune

(Fortune) -- One of the most popular groups on Meetup.com - the Web site that allows communities of any kind to self-organize with the goal of ultimately meeting in person - is The Stay at Home Moms. Some 65,000 people in ten countries - including Singapore and Japan - have signed up and organized nearly 100,000 meetings. The groups' page even has sponsorship from companies that make diapers and other relevant products.

The Moms have pressed the generic capabilities of Meetup into service to create a sense of local community that is otherwise hard to arrange in a physically dispersed culture. It's obvious why they would find Meetup valuable.

Less obvious but at least as remarkable is how that particular group came to be in the first place. Every Meetup group navigates the tension between specificity and size. A Meetup group perfectly fit to an individual (bald fathers of two in Brooklyn who teach at NYU and like bagpipe music) would have exactly one member, while a Meetup that included huge numbers of potential members (parents, or TV watchers, or residents of Atlanta) would provide little in the way of commonality or conversational fodder: "So, you watch TV too, huh?"

The ideal group exists in some equipoise between too specific and too generic. The Stay at Home Moms group fits that description well enough that it is more popular than all the other parenting groups and one of the most popular Meetup categories overall. Even accepting that Stay At Home Moms exists at some optimum point between size and specificity, there's still a mystery about its formation: How did Meetup know that the group would be as appealing as it was? Most of the people who work at Meetup are overeducated, undermarried urbanites who face a completely different set of problems than do the North Charlotte Stay at Home Moms. How could they have known that SAHM groups would be such a hit?

They didn't. To have predicted such a thing, the employees of Meetup would have needed research about the changing face of American communities, current trends in self-definition of mothers, interactions among suburbanites, and so on; demographics, psychology, sociology. Even if someone had told them that Stay at Home Moms was a good idea for a Meetup group, the staff might have been loath to propose such a thing. Coming from a bunch of single urbanites, it might have seemed patronizing, to say nothing of polarizing. The company might have become a target for political protest by people upset about the exclusivity implied by the name. The Meetup staff could not have gathered enough information to understand which parenting groups to suggest in the first place, could not have picked a winner even if they'd had all that information, and could not have launched the winner even if they'd been able to pick one, because of the potential negative reaction.

Though it seems funny for a service business, Meetup actually does best not by trying to do things on behalf of its users, but by providing a platform for them to do things for one another. There are hundreds of thousands of Meetup users, and each is presented with many possible Meetups that they could attend. In a midsize city the potential combinations among people interested in Meetup groups are overwhelming.

The only sensible way to solve this problem is to turn it over to the users. The most basic service that Meetup provides is to let its users propose groups and to let other users vote with their feet, like the apocryphal university that lets the students wear useful paths through the grass before it lays any walkways. Most proposed Meetup groups fail because they are too generic, or too specific, or too boring. Most of the rest have only moderate success, leaving only a relative handful of very popular groups, like Stay at Home Moms.

This distribution - lots of failure, some modest success, and few extremely popular - is the same pattern that we have seen elsewhere. The advantage of having a system where failure is normal and significant success rare is that, by its very existence, Meetup continually readjusts to its current context. The standing question that Meetup poses to its members is "What kind of group is a good idea right now?" Not in the twenty- first century generally, but right now, this month, today.

The rise of new groups and the retiring of old ones is not a business decision, it's a by-product of user behavior. Meetup didn't have to establish or even predict the popularity of the Wiccan or LiveJournal groups; nor did it have to predict the time when those groups would be displaced as the most popular. Users are free to propose and pass judgment on groups, and this freedom gives Meetup a paradoxical aspect.

First, it is host to thousands of successful groups, groups of between half a dozen and a couple dozen people who are willing to pay Meetup to help them meet regularly, usually monthly, with other people in their community. Second, most of the proposed Meetup groups never take off, or they meet once and never again.

These two facts are not incompatible. Meetup is succeeding not in spite of the failed groups, but because of the failed groups. This sounds strange to our ears. Particularly in the world of business, with its Pollyanna-ish attitude toward all public pronouncements, we rarely hear about failure. Meetup's core offer - an invitation for a group of people to get together at a particular place and time - fails with remarkable frequency, as user-proposed groups often don't materialize.

Yet Meetup, the company, is doing fine, because the successful groups meet regularly, gain more members, and often spawn new groups in new locations. Meetup is a giant information-processing tool, a kind of market where the groups are products and where the market expresses its judgment not in cash but in expenditure of energy. Failure is free, high-quality research, offering direct evidence of what works and what doesn't. Groups that people want to join are sorted from groups that people don't want to join, every day. By dispensing with the right to direct what its users try to create, Meetup sheds the costs and distorting effects of managing each individual effort. Trial and error, in a system like Meetup, has both a lower cost and a higher value than in traditional institutions, where failure often comes with someone's name attached.

From a conventional business perspective, Meetup has no quality control, but from another perspective Meetup is all quality control. All that's required to take advantage of this sort of market are passionate users and an appetite for repeated public failure. Meetup shows that with low enough barriers to participation, people are not just willing but eager to join together to try things, even if most of those things end up not working.

Meetup is not unusual in this respect. Most pictures posted to Flickr get very few viewers. Most Weblogs are abandoned within a year. Most Weblog posts get very few readers. On YahooGroups, an enormous collection of mailing lists on topics from macramé to classic TV shows to geopolitics, about half the proposed mailing lists fail to get enough members to be viable. And so on. The "power law distribution" of many failures and a few remarkable successes is general.

From "Here Comes Everybody: The Power of Organizing Without Organizations" by Clay Shirky. Copyright (c) Clay Shirky, 2008. Published by arrangement with The Penguin Press, a member of Penguin Group (USA), Inc.