NEW YORK (CNN/Money) - Credit card holders are paying more in late fees to banks than ever, according to a survey by an online publisher specializing in payment cards.

Since 1996, late fee revenue generated to bank credit card issuers has soared to $7.3 billion from $1.7 billion annually, Cardweb.com found.

In an informal online poll conducted in early May, 58.3 percent of consumers said they have been hit with a late fee during the past year.

Since the first of this year, average late fees have jumped more than 5 percent, to $29.84 from $28.29, as several major issuers have boosted late fees this year to $35 from $29. Since 1996, late fees have more than doubled from an average of $13.28, CardWeb.com said.

Late fees now represent the third-largest revenue stream for banks, just behind interest revenue and merchant fee revenue, CardWeb.com said.

More consumers are getting snared by late fees as due dates are being moved up. The average grace period has been reduced to 21.2 days, down from 29.7 days in 1990. And the number of days a credit card company may allow your payment to be past due before assessing a late fee has "evaporated totally," said Robert McKinley, CEO of CardWeb.com.

There are ways to have a late fee waived if you feel you've been fined unjustly.

The first step: "Call and complain. Most customers who do that have been very successful," McKinley said.

You have your best shot at success if you've been a good customer, you normally pay on time, you don't max out your credit limit, and you have had a longstanding relationship with your credit card company, said Brad Dakake of the consumer advocacy group MassPIRG.

"Most companies will want to keep your business," Dakake said. Even if you don't carry a balance -- and hence pay interest to your credit card issuer -- the companies make money from merchant fees every time you charge something using their card, he noted.

But the best way to avoid late fees is to make sure your payment is received on time. That means allowing at least five business days for your check to make it through the mail. At the very least, McKinley suggested, send in the minimum due when you receive your credit card statement and then send a second, larger payment near the due date.

If you're cutting it close to the wire, you can pay online, but find out if your credit card company takes a day or two to process such payments and plan accordingly. Some card issuers will also allow you to call them and make electronic payments, but there may be a fee of $5-to-$10 on top of whatever your bank may charge you, McKinley said. If you want to send a payment overnight, call first and find out the best address to ship it to, since the address you usually use is often a post box, where payments may sit for days, he explained.

If saving an extra $35 isn't incentive enough to avoid a late fee or get it waived, consider this: If you get more than one late fee in a year, you could be assessed a high penalty interest rate on your balance. That means you'll be throwing even more money away to line the pockets of your creditors.