Former Federal Reserve chairman Paul Volcker warned Monday about potential dangers from what he calls "unorthodox" and aggressive moves by central banks around the world.

Central banks, including the Fed, could eventually inflict more harm by "what they're doing with their portfolio to save the world economy," said Volcker speaking alongside Sir John Vickers, the chair of the U.K.'s independent commission on banking, at NYU Stern School of Business early Monday.

"Central banks are no longer central banks," said Volcker. "I think it gets dangerous when they lose sight of the basic function of the central bank."

Volcker said that rather than trying to stay out of the market as much as possible and simply tinker from the sidelines, central banks have been aggressively trying to influence economic growth and even inflation.

"The central bank is not an all-powerful tool," he warned.

Related: Bank of Japan shocks markets

Last week, Japan's central bank pledged to double its balance sheet over the next two years. That sent the Nikkei rallying and the yen to four-year lows against the U.S. dollar.

"Once you have the idea that a little bit of inflation is a good thing, it is very hard to get rid of it," said Volcker.

Moreover, he says the more central banks become key players in financial markets, the more dangerous their actions can become.

"The Federal Reserve is the world's largest financial intermediary," he said.

When asked about a Financial Times article about central banks pushing into riskier assets to get yield in a low interest rate world, Volcker said he wasn't aware of such moves, but said that he's sure central banks will ultimately face losses on their portfolios when interest rates rise.

Related: Volcker: Big Banks are too big, but that's OK

He stopped short of offering details about how he saw these scenarios playing out.

But he did see one area as nominally safer since the financial crisis: banks. Volcker said that the his rule -- the "Volcker Rule," which is designed to limit commercial banks' abilities to make big trades with bank deposits -- is working so far.

Proprietary trading has moved out of commercial banks and into other parts of the financial markets, which he said is a good thing.

Still, he acknowledged that JPMorgan's London Whale losses show the limits of the Volcker Rule and some of the perils of regulation.

Volcker didn't comment about whether JPMorgan's London Whale trades were proprietary or not, but he did note that even the bank regulators embedded in JPMorgan failed to see these trades taking place.

With five agencies involved in implementing and enforcing the Volcker Rule, it's tough to ensure they're all on the same page.