NEW YORK (CNN/Money) - Merchants are calling into question the more-than-$20 billion in transaction fees a year charged by credit card companies, alleging that the practice amounts to price-fixing, collusion and conspiracy, a newspaper report said Thursday.

In an expanding credit card fight, more stores are accusing banks of working with Visa and MasterCard to illegally fix prices, the Wall Street Journal said, adding that some stores have already won large, undisclosed settlements.

"These fees amount to at least a $20 billion annual tax on merchants and the economy, raising prices for consumers on almost any product they buy," Craig Wildfang, a lawyer in a recently filed lawsuit against "interchange fees," told the Journal.

Merchants have long complained about the high fees, but the issue flared following a recent round of fee increases by Visa and MasterCard, the paper said.

Wal-Mart has won concessions valued at more than $1 billion over the fees; and Best Buy, Toys "R" Us, Home Depot and CVS are negotiating or have already won fee cuts, according to the report.

Visa and MasterCard have defended the fee increases, levied mostly on premium cards, saying that they are an essential component of a global system that has provided benefits to consumers and merchants, while covering fraud risks and the cost of float.

"Interchange is highly beneficial, efficient and procompetitive," Noah Hanft, MasterCard's general counsel, was quoted by the Journal as saying.

"We believe the merchants in this suit are seeking to shift their normal costs of doing business onto someone else -- the consumer," Visa said in a statement.

"Visa delivers enormous value to all participants in the payments system, and we will defend against any actions that would drive up costs for our cardholders, reduce funding for security measures or open the door for merchants to charge fees to consumers for choosing their Visa card," the company said.

The growing number of lawsuits against Visa, MasterCard and banks could prove more costly than the $3 billion settlement merchants won in a legal battle last year over debit-card fees, the Journal quoted analysts as saying.

The stakes are especially high for large banks with big credit-card operations such as Chase, Bank of America and Citi, since interchange fees represent as much as 15 percent of their annual revenue, the report said.

Worried about credit card fraud? Click here.