He said he was from the IRS, then stole $30,000 from me

September 2, 2014: 7:37 AM ET
phone scammer
NEW YORK (CNNMoney)

When 32-year-old Seila So received a call from a man with a deep voice claiming to be an IRS agent, she froze in her tracks.

Then he told her she owed $30,000 in back taxes.

He said the IRS had been trying to contact her for months, and this was her last chance to pay. He explained that the local police were ready to arrest her if she didn't immediately fork over the money.

The first thing that came to mind was a notice she had received from the IRS years ago saying she owed several thousand dollars -- but she had immediately paid up. Maybe she had actually owed much more? Maybe the IRS had sent her notices that she just hadn't received?

The caller knew her full name and address. He had given her a case number for her records. And the call came from a number with a Washington, D.C. area code.

He must be legitimate, she thought.

Related: 7 scams that will make your blood boil

She asked if she could take a moment and call him back, but he said hanging up would be interpreted as a refusal to pay and that she would immediately be arrested and face jail time of up to eight years. Her house would also be taken, her driver's license suspended, her bank accounts frozen and her wages garnished, he said.

"He was so aggressive and threatening, it scared the daylights out of me," So said. "I didn't know what to do except follow instructions. How can I question the IRS?"

He directed her to drive to the bank and withdraw cash. He stayed on the phone the whole time, saying the call was confidential and legal action would be taken if he heard her tell anyone about what she was doing.

She withdrew nearly $5,000 from her own savings account. Then she went to the bank where her father had set up an emergency account and withdrew around $25,000, leaving just a couple thousand dollars in the account.

Next, the man on the phone instructed her to go to several different stores to pick up 30 GreenDot MoneyPak cards, which are used to transfer money to prepaid cards. She asked the cashiers at each of the stores to load $1,000 on each card.

With cash reload cards, fraudsters don't need to get their hands on the physical card -- if they get the code on the back, they can transfer money onto their own prepaid card and then cash it out, says Susan Grant, director of consumer protection at Consumer Federation of America.

Related: Fake IRS callers take $5 million from victims

So scratched off the strips on the back of the cards and read him the codes.

In a matter of minutes, her entire life savings -- and the money her parents had saved -- was drained.

She returned to her house that night shaking, and called a friend to tell him what happened. A few minutes later, he emailed her a link to a CNNMoney article about an IRS impersonation scam that has now been linked to $5 million in stolen funds.

So began sobbing as she read the details: scammers use caller ID spoofing to make it appear they are calling from IRS offices, they use case numbers and IRS jargon to try and add legitimacy and often ask victims to load money onto prepaid cards because they are harder to trace.

"I couldn't stop crying, because all my money was gone," she said. "I thought, 'how could I be so stupid?' I didn't know what to do -- my body was numb."

Related: 'A scammer stole my tax refund'

So quickly went to the police station to file a report, and was told that a detective would look into the case.

But it's generally rare that victims of this type of scam recover their money -- especially when the money is loaded on prepaid or cash reload cards, the CFA's Grant says. Unlike debit cards or credit cards, which are attached to a financial institution, these cards can be impossible to track and often aren't insured.

"Putting the money on a prepaid or cash reload card is like putting cash in an envelope," said Grant. "[This scam] works because people are afraid of running afoul of the IRS and being penalized for something, so they'll send the money even if they are not sure they owe it."

(Other red flags to watch out for and what to do if you're a victim can be found here.)

As the investigation into So's case continues, she is looking for a second part-time job to help her make ends meet.

"I'm starting from scratch," she said.