(FORTUNE Magazine) – WALKING THROUGH THE mall a couple of weeks ago, Hayden Cobb, a 32-year-old systems engineer at Lockheed Martin, couldn't resist a few impulse buys. But after realizing that all his crisp new shirts didn't fit right, he headed back to the Express store near Atlanta, receipt in hand. The clerk asked for his driver's license, swiped it, and then handed him a small slip of paper that read "Return Declined." "I was dumbfounded," he says.

Cobb is just one of the many customers who are finding that returning merchandise isn't as easy as it used to be. Retailers including Express, the Limited, Victoria's Secret, and the Sports Authority have begun tracking consumer return and exchange habits to help curb the $16 billion that stores lose in "return fraud" each year. All the companies mentioned have enlisted California-based Return Exchange, a five-year-old for-profit company that stores customer ID and payment information and tracks shopping behavior, looking for patterns of fraudulent or excessive returns. The system also aims to prevent "wardrobing," in which people (women in particular) buy clothes, wear them to a party, and return them the next day. "We're not accusing you of being a thief," says King Rogers, a consultant who advises Express on security matters. "We're suggesting that you're not a profitable customer." While stores have long reserved the right to refuse returns, shopper tracking has privacy watchdogs like Jordana Beebe of the Privacy Rights Clearinghouse alarmed (she's particularly worried that data across stores may eventually be aggregated).

For his part, Cobb insists that he just changed his mind about the shirts. But after requesting his "return-activity report" from the Return Exchange (which is available free by calling 800-652-2331), he saw that dozens of his returns over the past year had been tracked. Worse still? He's stuck with those shirts for good. -- Barney Gimbel