How 'Waiting for 'Superman" misses

FORTUNE -- It would be churlish not to recognize that the new documentary "Waiting for 'Superman'" is a well-intentioned cry for reform in American public education. Director Davis Guggenheim, who won a 2007 Oscar for "An Inconvenient Truth," even performs political magic: "Superman" is a lefty's screed that right-wingers have lauded. Almost every reviewer loved it. I did, too, but on reflection I've got a beef.

The problem is the movie's conclusion that the charter schools it champions are a panacea. The movie is wise enough to recognize that parents, teachers, unions, politicians, philanthropists, and social scientists all have axes to grind on charter schools. And yet the film fails to question its own assumptions. It ought to.

"Superman" follows five young students from poor neighborhoods across the country. All five want to attend charter schools, but there aren't enough spaces. The law requires a heartbreaking lottery to fill the slots; there are no tests or interviews. Whether the students get in (and thus out of their traditional public schools), we're told, will largely determine their futures. Their odds aren't good. Bianca wants to attend Harlem Success Academy in New York City; there are 35 spaces for 767 applicants. Daisy has applied to KIPP in Los Angeles (10 spaces, 135 applicants). And so forth. Fair enough, yet the movie fails to ask: If these charters are that good -- and they're free -- shouldn't every child in the eligible area be applying? Why isn't the number of applicants far higher?

One reasonable answer is that the applicants represent a self-selected sample. By definition, they and their parents were highly motivated to find alternatives to the default neighborhood schools to which their zip code sentenced them. So this group may already be headed for better test scores and other gauges of achievement -- in the same way that students in Scarsdale, N.Y., and Palo Alto succeed less because of what happens inside their schools and more because of who they are and how they were raised.

Some academics have tried to account for the self-selection flaw. Last year Stanford economist Caroline Hoxby published a controversial study examining the academic performance of students who "lost" charter school lotteries in New York and wound up attending traditional public schools. Hoxby found that those students did less well than their charter-school peers over a seven-year period. But other scholars have challenged her methodology. There's also the reality that many charter schools with lotteries don't produce great results like those exalted in "Superman." Therefore, motivated families alone don't dictate educational outcomes. The quality of a charter and its teachers surely matters.

The ideal way to measure a charter school's utility would be to pick students randomly from a pool of all eligible candidates, thus removing the distorting influence of self-selection. That's unlikely. But in the absence of truly random selection, the movie's certainty that schools make all the difference seems misplaced. So, too, may be the dollars of businesses and foundations clamoring to fund elite charters.