Bernard Madoff paid $326 million to the IRS to keep his Ponzi scheme going, according to the court-appointed trustee, who reached a settlement to get the money back.
NEW YORK (CNNMoney) -- The court-appointed trustee in the Bernard Madoff Ponzi case has hatched a $326 million settlement with the Internal Revenue Service.
The deal would recover taxes Madoff filed to keep his pyramid-style scam going. Trustee Irving Picard claims that Madoff paid taxes for no other reason than to appear authentic in the eyes of the IRS.
Madoff masqueraded as a Wall Street wizard, heading the now-defunct Bernard L. Madoff Investment Securities LLC as a front for his long-running Ponzi scheme.
"I believe the payments made to the IRS falsely identified the funds as income tax withholding in order to give the investment advisory arm of BLMIS an air of legitimacy and to avoid injuries," wrote the trustee, in court documents filed to U.S. bankruptcy court in the southern district of New York Tuesday.
The trustee said Madoff paid the taxes "purportedly on behalf of certain foreign account holders," but this was just part of his scam.
"My investigation has revealed that there is no record of BLMIS having purchased or sold any securities for the benefit of foreign account holders," said the trustee in court documents.
He said that Madoff actually paid $330 million in taxes, but the IRS mistakenly refunded him about $4 million, resulting in the $326 million total payment.
The IRS was not immediately available for comment.
The settlement still has to be approved by a judge, which won't happen earlier than Dec. 21, according to the court clerk's office.
The agreement is part of Picard's ongoing effort to recover money and assets lost to the pyramid-style scheme of Madoff, who pleaded guilty in 2009 and is serving a 150-year sentence at a medium-security federal prison in North Carolina.
Madoff stole much of the money and didn't invest any of it. Some of the money was use to pay remittances to clients when they wanted their cash back, in order to keep the game going. But the house of cards fell apart in December 2008, when Madoff ran out of money to pay the remittances.
At the time of Madoff's arrest on Dec. 11, 2008, about $20 billion had been lost to his scheme. The trustee has since recovered at least half of that money, with the largest chunk coming from a $7.2 billion settlement with the widow of Jeffry Picower, believed to be Madoff's biggest beneficiary.
The money is to be returned to the 2,425 investors who the trustee has declared to be legitimate victims.
The $326 million settlement would be added to this tally, assuming it is approved by the judge. 
