NEW YORK (CNNMoney.com) - Few would argue that retailers got a lucky break over the past holidays. Despite a very challenging year for consumers, shoppers were able to put money worries aside and dug deep in the spirit of year-end gift-giving.

But that was yesterday.

A new year has sprouted a fresh round of speculation from industry observers, and more tellingly, from retail executives themselves about the health of the consumer.

The National Retail Industry Federation (NRF), the industry's own trade group, set the tone when it delivered a much softer sales outlook for the year. The group expects total retail sales will rise 4.7 percent in 2006, down from last year's more robust 6.1 percent gain.

The group said higher energy costs and a slowing housing market will pressure consumers to retrench spending in 2006, causing a slowdown in overall retail sales growth.

Bob Negron, group president for Liz Claiborne (Research)'s special sales and cosmetics licensing division, said that given the broad spectrum of economic headwinds swirling around, it's questionable whether consumer spending will hold up through the entire year.

"I think consumer spending will soften somewhat," Negron said in an interview with CNNMoney.com. Negron was in New York this week attending the NRF's annual convention.

"Consumer confidence levels are not robust. Interest rates are rising, credit bills are high and savings are running negative. Corporate investments will go up but the [benefits] of that type of spending usually takes a while to trickle down," he said.

Economists say the nation's housing market has made consumers feel wealthier, acting as a bulwark against rising energy prices. When interest rates were falling and home prices were rising, Americans quickly refinanced their mortgages at lower rates, effectively turning their homes into piggy banks, and raiding them for cash.

Now, with interest rates rising and home prices softening, refi activity is slowing and consumers could end up feeling strapped for cash.

At the same time, Negron talked up the outlook for specialty retailing. Said Negron, "The specialty environment will consist of winners and losers. Those with superior products will win. "

Susan Davidson, group president of the Liz Claiborne's women's denim division, chimed in.

"Product is a big differentiator," said Davidson. "Consumers are extremely savvy and they're quick to shop across channels. We have to make sure that the price/value is sharper."

The other reason why he thinks specialty players will grow faster than other sectors is because of the ongoing consolidation among department stores. "Department stores are losing market share because they're closing more stores," Negron added.

Others list "flexibility" as a key attribute for retailers in today's economic environment.

"We're certainly operating in a different world," said Edwin Mosher, CEO of San Jose-based privately-held men's apparel retailer Mosher's Ltd. "But the American consumer is very resilient and has been able to withstand devastating hurricanes and pain at the pump."

"From our perspective as retailers, we have to better manage our inventory and operating expenses," he said.

Margery Myers, a spokeswoman for women's clothing and accessories retailer Talbots (Research), agreed.

"You look at what your history is and where your strengths are," she said And even though retailers plan their seasonal inventory months in advance, they've got to become more nimble. "If some products are working with customers, give yourself enough wiggle room to make the necessary changes, " she said.

At least one industry watcher, however, says this year's sales growth doesn't hinge on gas prices, interest rates or the housing market.

"It's all about jobs," said George Whalin, an independent retail consultant. "As long as people are confident about their jobs and being able to find one, they will spend. The job market so far is holding up well."

He also thinks it's a mistake for retailers to be obsessed about selling at the lowest price.

"Look at last year. The retailers that did the best gave customers the most value for the money and not necessarily the lowest prices," he said, referring to luxury sellers like Nordstrom (Research) and specialty stores like Chico's (Research) and Abercrombie & Fitch (Research). "Value means more than price. It's about service, store design and product quality. That's how you win customers."

-----

The National Retail Federation expects a challenging year for retailers and consumers. Click here for more.