A federal appeals court on Friday agreed with a Federal Reserve rule that placed a 21-cent cap on swipe fees that banks charge retail stores for each debit card transaction.

The U.S. Court of Appeals for the District of Columbia Circuit struck down a lower court ruling that sided with retailers saying the Fed should lower fees even more.

The ruling is a win for the Federal Reserve and the banks. For consumers, nothing is expected to change.

The fight over swipe fees stems from new laws enacted after the U.S. financial crisis. Congress ordered the Federal Reserve to ensure that fees to process debit card purchases were "reasonable." At that time, the average fee per transaction was about 40 cents.

After initially proposing to limit fees to 12 cents, the central bank issued a final rule in 2011 that capped fees at 21 cents.

Soon after, the National Association of Convenience Stores and the National Retail Federation filed a lawsuit against the Fed, saying that the fees should be lower.

It was not clear that consumers got any of the savings. One industry-sponsored survey found that 67% of retailers kept prices the same or raised them instead of passing on savings to customers, according to the Electronics Payments Coalition, a group of banks and credit unions.

A 2013 report by the retailers, called the Merchants Payments Coalition showed that swipe fee caps saved consumers $5.8 billion in 2012 through lower costs for goods and services.

But banks Bank of America (BAC) , J.P. Morgan Chase (JPM) and Wells Fargo (WFC) took big hits to revenue. The banks tried to charge new debit card fees, but public outrage caused them to reverse course.

Related: Big Wall Street bonuses are back

The ruling also impacts charge card processors Visa (V) and Mastercard (MA), which collect swipe fees for the banks and keep a portion for use of their payment networks. Shares of Visa rose 4% in trading in Friday. MasterCard was down 1%.

A Federal Reserve spokesperson said the Fed board is "pleased with the outcome of the appeal."

-- CNN's Bill Mears contributed to this report.