Hewlett Packard(HPQ) was by far the biggest gainer of the day. Shares of the PC maker rallied 17%, a day after the company's earnings beat estimates and CEO Meg Whitman said she was "encouraged" by the turnaround plan.
A big sell-off in Japan dragged down Europe and put pressure on U.S. markets in early trading, but they clawed back throughout the day.
China manufacturing and Fed fallout: Some investors blamed the weakness in Asian stocks on a report that showed manufacturing activity in China slowed in May for the first time in seven months, raising concerns about growth in the world's second biggest economy.
But others argued that the selling was a continuation of losses sustained late Wednesday after minutes from the Fed's latest policy meeting showed some officials were willing to start slowing the pace of the central bank's bond buying program as soon as June.
"The red today is all fallout from the Fed yesterday," said Phil Orlando, chief equity market strategist with Federated Investors.
Bernanke told lawmakers Wednesday that withdrawing the Fed's stimulus measures prematurely could derail the economic recovery, though he hinted that the central bank could slow the pace of its bond buying later this year if the economy improves.
"Everyone's concerned the Fed starts to taper in June," said Orlando. "We think there's zero chance that happens."
The Fed's stimulus policies have been a big driver of the bull market over the past few years. But with the major indexes up more than 15% so far this year, some investors have called for a pullback.
"I think the market is looking for excuse to take some profits," said Orlando. "The bears who missed out on the way up will take this opportunity to put money back to work."
Bond yields back up. Meanwhile, the yield on the 10-year Treasury note held above 2%, a level not seen in more than two months.
The uptick in yields was another sign that investors expect the Fed to move toward a tighter stance on monetary policy, but traders say a major sell-off in bonds still seems unlikely.
"We would not expect much selling and for that cash to go under the mattress while the storm is weathered," said Suki Mann, credit strategist at Societe Generale. "Investors will hold firm and ride it out."
Separately, new home sales rose 2.3% in April compared with the month prior, according to government data.
Eye on retail: After the market closed, Gap(GPS) reported earnings that topped expectations and affirmed its outlook for the year. The retailer said sales rose 2% in the quarter, down from 4% in the same quarter last year. Shares were down after hours.
Shares of Sears(SHLD) plunged 10% after hours, after the company reported a much bigger loss than expected.
Shares of Ralph Lauren(RL) slumped after the retailer failed to meet lowered revenue forecasts, even as earnings jumped 35%.
Meanwhile, discount retailer Dollar Tree(DLTR) reported better than expected earnings, sending shares up nearly 4%.