NEW YORK (CNN/Money) - Howard Stern picked a puzzling time to make good on his longstanding threat to defect to satellite radio -- he's still got more than a year left on his contract with Infinity Broadcasting, distributor of his morning talk show.

CNNfn's Chris Huntington reports on Stern's move to satellite radio. Play video (Real or Windows Media)

For Infinity, the timing was anything but propitious.

The radio industry is facing an unexpected slowdown in advertising, prompting some Wall Street analysts to downgrade stocks of both Clear Channel Communications and Viacom, which owns Infinity. Clear Channel (CCU: Research, Estimates) and Viacom (VIAB: Research, Estimates) are the No. 1 and 2 radio operators in the country.

Just a few months ago, analysts and company executives were predicting radio would benefit from strong consumer spending and rising ad revenue.

But while other media outlets, most notably the Internet, are enjoying a healthy advertising rebound, the radio industry as a whole has been left out in the cold.

"Radio stocks are in growth purgatory," wrote Morgan Stanley analyst Michael Russell in a report last week. He cut growth forecasts for the radio advertising industry from 3.3 percent to 2.6 percent this year and from 3.8 percent to 3.5 percent in 2005.

Infinity has been struggling in recent years, and there have been rumors that Viacom, which acquired the company in 2000, might shed all or parts of the unit. That Viacom shares are down 25 percent this year, despite solid quarterly earnings, has further fueled the speculation.

Stern, who started at Infinity in the mid-1980s, has proven to be a solid revenue generator for Infinity.

The saucy shock jock's exit "is certainly not a positive for Viacom's radio business," Richard Greenfield, an analyst with Fulcrum Global Partners, told clients in a report Wednesday.

"I believe he's just trying to plan his career," April Horace, analyst with Janco Partners, said regarding Stern.

Viacom owns 185 radio stations, 27 of which air the "Howard Stern Show."

More on Howard • CBS fined $550K • What is Howard worth? • Clear Channel to pay $1.75M • Clear Channel cuts Stern • Report: $1.5M Stern fine • First Mel, now Howard?

Fulcrum's Greenfield estimated the show draws from 8 to 10 million listeners a week. But Stern's likely impact is greater than that. Greenfield noted that Viacom has been able to leverage Stern to sell advertising throughout the day.

"The radio business is already under pressure" due to poor marketing and too many ad minutes per hour, Greenfield said.

With Stern moving to Sirius Satellite Radio (SIRI: Research, Estimates) and Opie & Anthony, two more radio stars formerly with Infinity, moving to XM Satellite Radio (XMSR: Research, Estimates) Monday, the "long-term growth prospects (for radio) are only weakened further."

It remains to be seen how Viacom responds to Stern's exit and the problems plaguing its radio unit. Chairman Sumner Redstone has publicly defended his radio properties, noting they generate lots of cash that helps fund the company's other businesses.

Redstone has also said he plans to unload weaker stations but will not sell Infinity.

There's another factor worth considering.

YOUR E-MAIL ALERTS Viacom Stocks Advertising Shareholder Relations or Create your own Manage alerts | What is this?

If, as it appears, Stern stays at Infinity through the end of his contract in 2005, and, assuming he doesn't spend the next 15 months shamelessly promoting satellite radio while on the air, the impact of his departure won't be felt for more than a year.

Even then, analysts said Stern's exit from Viacom should be relatively mild.

Radio represents only 15 percent of the company's earnings before interest, taxes and depreciation, far less than the 75 percent that cable and broadcast television contribute.

If Viacom divests its weakest stations, radio should be an even smaller piece of the Viacom profit pie come 2006.