The Federal Reserve's No. 2 just raised a warning flag for the U.S. economy.

Fed Vice Chair Stanley Fischer says America's economy could be sluggish for a very long time. Low interest rates are partly to blame, he believes.

"To the extent that low long-term interest rates tell us that the outlook for economic growth is poor, all of us should be very concerned," Fischer said Monday at the Economic Club of New York.

U.S. economic growth has averaged a sluggish 2% since 2009. Between 1990 and 2005, growth averaged a healthier 3% pace.

And Fischer knows what you're thinking: "Well, if you and your Fed colleagues dislike low interest rates, why not just go ahead and raise them?" he asked rhetorically.

Fischer argues that factors outside the Fed's control -- such as low productivity and aging Baby Boomers -- are holding down interest rates of all kinds. Remember, the Fed's own policy has direct impact on short-term rates, but long-term rates react to more than just the Fed.

The Fed has raised rates once, in December 2015, since putting them at zero in 2008. The Fed is widely expected to hike again this December.

Related: U.S. dollar rallies on Fed rate hike hopes

Why Fischer is 'very concerned' about low rates

Low rates spell bad news in Fischer's view for three reasons:

1. Low rates foreshadow sluggish growth.

2. They limit the Fed's options to handle the next recession.

3. They encourage investors to take risky bets. Not good.

Typically during a recession, the Fed cuts interest rates to make it easier to borrow money and spend, which should help stimulate growth. But if rates are already near zero, the Fed can't do much, except put rates into negative territory -- an option it really doesn't want to use.

Being close to zero "limits the room for central banks to combat recessions," by cutting rates, Fischer said. He went on to say the Fed would be in "deep trouble" if a recession hit soon.

Low rates also encourage investors to go into risky investments. The return on a 10-year U.S. Treasury bond -- a super safe asset -- is really low at 1.7%. Knowing they won't get much return on safe assets, investors turn to stocks or high-return bonds in emerging nations.

Related: Sign up for CNNMoney's morning newsletter

Why rates are so low

Yes, the Fed cut rates to zero in 2008 but Fischer argues that they're not low because of the central bank.

In his view, rates are low for four key reasons:

1. Global growth is slowing down.

2. Baby Boomers are retiring, making the labor force smaller. Fewer workers = slower growth.

3. Businesses aren't spending much on new projects.

4. There hasn't been a game-changing innovation (read: internet) that boosts workers' productivity.

All those factors encourage Americans to save instead of invest. While saving is good, a lack of investment cuts down economic growth.

What would make rate go up

Interest rates would rise if there was an increase in business and government spending on better education, worker training programs, roads and bridges and innovation, Fischer argues.

One of those could be around the corner: Both Hillary Clinton and Donald Trump have promised to spend big on infrastructure to fix or improve America's rails, roads and runways.

But business spending has been low for years and doesn't appear to be turning in the right direction soon.

Fischer warns there's a possibility that "low long-term interest rates are a signal that the economy's long-run growth prospects are dim."