Luxury consumers demand the best. Apparently the same is true for luxury investors, especially those who own Michael Kors.

The fashion retailer revealed another quarter of strong growth on Monday that many rivals would kill for -- surging sales, swelling profits and rapid overseas growth.

But Wall Street was not impressed. Michael Kors (KORS) shares crumbled as much as 7% and end the day down almost 6%.

"The brand is clearly losing popularity," said Rick Snyder, an analyst who covers Kors at Maxim Group.

Related: Can this British designer save Coach?

Growth is slowing: The latest numbers from Kors have failed to quiet concerns that the company's brand is overexposed, especially in North America. Sales, while impressive, are cooling, and there is more pressure to markdown merchandise.

While North American same-store sales soared almost 19% last quarter, that represents a slowdown from about 25% the year before.

"The headline number is impressive, but we never like to see those decelerating trends," said Snyder, who has a "hold" rating on Kors.

The other big worry is that the Kors retail apparatus is not as profitable as it once was. Kors executives warned margins will continue to fall.

Related: RadioShack's days are numbered

While Kors said markdowns were flat from a year ago, Snyder said he believes "markdown cadence is picking up" since customers are no longer as willing to pay full price.

But Kors' profitability has also been hit by efforts to expand by investing in existing stores and building new ones, especially overseas.

"It's healthy to see the company want to invest in future growth as opposed to capitalize on the short-term popularity," said Simeon Siegel, an analyst who covers Kors at Nomura Securities.

During a conference call on Monday, Kors CEO John Idol defended the long-term focus and noted the company always expected margins to recede.

"We are not here for a quarter-by-quarter situation. We are here to build something that is very sustainable, and to do that you have to invest," Idol told analysts.

Related: Walmart is 'going to a very dark place'

Is this sell-off overdone? Siegel, who has a "buy" rating on Kors, believes concerns about margins and slowing sales have been overblown.

"This stock more than any other I look at right now is just frothing with negativity," he said. "I don't get how you can argue this company is fading when they're putting up the results they are."

Even Kors skeptics won't argue the company is fading overseas.

Total sales in Europe surged over 125% thanks to impressive same-store sales growth and the opening of 21 locations. Revenue in Japan spiked about 90%.

"I think we're going to continue to hear more and more about acceleration in Europe and Asia and 'nice' growth in North America," said Siegel.

But will that be enough to satisfy the company's demanding investors?