After touching new all-time highs earlier Wednesday, the Dow and S&P 500 ended firmly in the red as investors wrestled with mixed messages from the Federal Reserve.

The Dow Jones industrial average fell 0.5% while the S&P 500 lost 0.8%. The Nasdaq sank 1.1%.

All three indexes had rallied earlier as investors welcomed comments from Fed chairman Ben Bernanke, who told lawmakers that withdrawing monetary stimulus prematurely could derail the economic recovery.

But the momentum faded after the Fed released minutes from its latest policy meeting.

According to the minutes, some members of the monetary policy committee were willing to dial down the Fed's bond-buying program as soon as June if the economic recovery becomes sustainable.

While a few members were concerned about investors' expectations for the overall size of the program, the committee voted to continue buying $85 billion worth of long-term Treasuries and other assets every month.

"There seems to be a bit more dissent at the Fed, or at least that's how people are reading the minutes," said Joseph Saluzzi, co-head of equity trading at Themis Trading.

The Fed's stimulus policies have been a big driver of the bull market in stocks over the past few years. But investors have been rattled by what appears to be a split within the Fed's monetary policy committee over the size of the asset purchases.

The minutes suggest the Fed will begin to curb its bond-buying activities sometime in the second half of the year, according to Paul Ashworth, chief U.S. economist at Capital Economics. He said the Fed will probably wait until September to make the change since that will give Congress time to sort out the nation's budget and raise the debt ceiling.

Related: Fear & Greed Index still at extreme greed

Housing still strong. The National Association of Realtors said existing home sales edged up 0.6% in April, matching economists' expectations.

Lowe's (LOW) reported an increase in quarterly sales and profit, but the home improvement retailer missed estimates. Still, the stock was up on the news.

The results came one day after rival home improvement retailer Home Depot (HD) reported stronger-than-expected quarterly results.

Meanwhile, Toll Brothers (TOL) reported strong results thanks to accelerating demand for houses and higher home prices.

Click for data on commodities, currencies and bond prices

Other retailers miss the mark. Shares of Target (TGT)fell after the retail chain reported a worse-than-expected decline in earnings.

Staples (SPLS) reported a decline in quarterly sales because of store closings. The office supplier also reported a drop in profit. But shares were higher.

Shares of Saks (SKS) rallied 13% following a strong sales report and a New York Post story saying the luxury retailer may be looking for a buyer.

Carnival (CCL) dropped one day after the cruise ship operator took a beating following its dour sales outlook for the year.

Shares of GE (GE) edged slightly higher after CEO Jeffrey Immelt said that the company would consider spinning off certain non-core assets within its financing arm GE Capital.

After the market closed, Hewlett-Packard (HPQ) said earnings and sales fell in the company's fiscal second quarter. Despite the declines, HP's profits were better than expected. Shares surged after hours on the news.

Related: Tornado victims may face long haul with insurers

European markets closed higher, while Asian markets ended mixed. Tokyo's Nikkei rallied by 1.6% after Japan's central bank pledged to maintain its ambitious quantitative easing program, saying that economic conditions in the country are improving.

Sony (SNE) shares spiked to their highest level in two years on reports that the company would consider a proposal to spin off its movie and music division.

CNNMoney's Maureen Farrell contributed to this story.