China isn't the bogeyman hiding in America's closet. Congress might be, though.

At least that's what three former U.S. Treasury secretaries believe.

Yes, the U.S. is facing a serious economic challenge from China and other faster-growing economies. And yes, China and the U.S. will continue to butt heads on the global stage.

But the real threat to American dominance might be found much closer to home.

"I think it's important most Americans understand we are still the master of our fate. The biggest risk to our economic interests in the U.S. are our politics in Washington," Timothy Geithner said Monday at the Milken Global Conference in Los Angeles.

Related: Bubble trouble: China's stock market looks too on fire

Complex problems: Geithner, along with Henry Paulson and Robert Rubin, said they fear America has been hurt by Congress's inability to tackle critical issues like immigration reform, the tax code and income inequality.

"The world is changing quicker than our policies," said Paulson, who served during the 2008 financial crisis. "We need fundamentally different policies."

The comments come at a time of deep geopolitical uncertainty. Tensions between Russia and the West are elevated, much of the Middle East is in chaos and the eurozone is grappling with existential questions.

"I think the world needs America to be America again. Because without the United States, there really is nobody to play the role the United States has played for many decades now," said Rubin, who led the Treasury under former President Bill Clinton.

Not too late: The good news is that despite Washington's fumbles, it's not too late for America to get back in the game.

"I don't see anyone out there that is going to overtake us as the predominant force in the world if we fix our problems and we lead by example," said Paulson.

Even Geithner, who was knocked by moderator and Facebook (FB) exec Sheryl Sandberg for being too pessimistic, found a silver lining.

"Our politics are terrible, but I think you'd rather have our challenges than any other country's," he said.