WASHINGTON (CNNMoney.com) -- Fed Chairman Ben Bernanke made his strongest case yet to Congress on Wednesday for the Federal Reserve keeping its regulatory oversight powers over banks large and small.
Bernanke told the House Financial Services panel he's "quite concerned" about proposals to limit the Federal Reserve's regulatory power to watching out for only the biggest banks. He called the proposal a "bad idea."
"It makes us the 'too big to fail' regulator, and we don't want that responsibility," Bernanke said. "We need a connection to Main Street, as well as Wall Street."
Bernanke said the Fed, alone, has the infrastructure and expertise to keep a close eye on big banks as well as regional and community banks. He suggested that losing the ability to monitor smaller banks would impact the Fed's role in stablizing the economy.
"The Federal Reserve's participation in the oversight of banks of all sizes significantly improves its ability to carry out its central banking functions, including making monetary policy, lending through the discount window, and fostering financial stability," Bernanke said.
Earlier this week, Senate Democrats released a draft of financial overhaul that would strip the Fed of overseeing all but the largest three dozen bank holding companies -- those with more than $50 billion in assets.
By contrast, under the House version, the Federal Reserve would keep most of its bank supervisory duties.
However, House Financial Services Chairman Barney Frank, D-Mass., said he held the hearing because some on his panel are now more willing to consider stripping more regulatory powers from the Fed.
"There's been some movement and the differences (about the role of the Fed) are less than they were," Frank said at the beginning of the hearing. "The desire and the need for a need for a bill will be greater than any individual differences. And an important part of this will be how the head of the Fed should be structured."
Bernanke has consistently fought to retain the Fed's oversight power over banks, although he has been open to the possibility of losing some powers to protect consumers and submitting to some sort of Congressional review that doesn't impact monetary policy.
In his testimony, the Fed chair pointed to the financial crisis as evidence of the importance of keeping those in charge of monetary policy also in charge of banking supervision.
"The financial crisis has made clear that all financial institutions that are so large and interconnected that their failure could threaten the stability of the financial system and the economy must be subject to strong consolidated supervision," Bernanke said.
Former Fed Chairman Paul Volcker also wholeheartedly defended the Fed's ability to continue as the supervisor of smaller banks.