Hoping to get your tax refund soon? Don't hold your breath.

Congressional dithering over a fiscal cliff fix could force the Internal Revenue Service to delay the start of the filing season. So those expecting to file their returns in mid-January, when the agency usually begins accepting them, may have to wait several weeks.

If Congress doesn't approve a patch for the alternative minimum tax in the next few days, then up to 100 million taxpayers will not be able to file their returns -- or collect refunds, if owed -- until late March, said Steven Miller, the agency's acting commissioner. (The AMT itself would hit nearly 30 million filers with higher tax bills, and returns of the others would be held up while the IRS reprograms its systems.)

Filing and refunds would also be put on hold by the uncertain fate of a dozen other provisions, including the deduction for state and local sales taxes and the $250 tax break for teachers who buy their own school supplies.

This isn't the first time the IRS has had to deal with this. Two years ago, it took until mid-December for President Obama and lawmakers to reach an agreement on many of the same issues. That delayed the opening of the tax season by four weeks to mid-February, affecting some 9 million taxpayers.

Most of them were likely owed refunds, since that's who files early, said John Lieberman, managing director of Perelson Weiner, an accounting firm in New York. They are often folks whose income is just from wages and who take the standard deduction, making their returns fairly simple. Many are low-income families who file for the earned income tax credit.

"They need that money back to pay Christmas bills, for a downpayment on a car or a security deposit on an apartment," said Lieberman.

Related: Fiscal cliff confusion on Form 1040

This year, the situation is even more dire since we are only days away from the start of 2013.

While the IRS has published the Form 1040 for 2012, several lines are listed as "reserved." The designation is a "placeholder" for provisions that have yet to be passed, an agency spokesman said. The IRS has yet to publish an instruction booklet for filling out the tax forms, leaving tax preparers in a holding pattern.

"How they are going to handle this?" said John Roth, senior federal tax analyst for CCH, a tax services company. "We are in virgin territory."

The IRS spokesman declined to answer questions on how the Congressional delay will affect taxpayers, but said more information would be available shortly.