NEW YORK (CNN/Money) - Record oil prices aren't the only big threat to the stock market's decidedly sunny mood of late. There's also the double whammy of August and September -- typically a tough time for investors.

Since 1950, September has been the worst month, on average, for the Dow and S&P 500 index, according to the Stock Trader's Almanac. August has been the third worst month.

September has also been the worst month, on average, for the Nasdaq composite since it began in 1971, according to the Almanac, and August has been its third worst month.

If you look shorter term, over the last 15 years, August has been the worst month for the S&P 500 and second-worst month for the Dow and Nasdaq.

The Almanac has also found that the first month of a quarter is typically its strongest, in this case July, and that the subsequent two months are weaker. That is especially the case in the summer.

Add in the fact that the S&P 500 and Nasdaq recently hit four-year highs, which could spark profit taking, and investors are in for some rough sledding over the next two months.

"August through October, you historically have weakness," said Peter Brodie, director of investments at Bryn Mawr Trust Wealth Management. "Even with a number of supportive factors in place right now, it's likely that these seasonal factors will kick in."

Brodie said that he is encouraged by the recent economic numbers, the improvement in payrolls and a significant drawdown in inventories of late. He says these are some of the factors that have been supporting the market, and should contribute to a strong rebound in the fourth quarter, even if August and September are weak.

Stocks have also been helped and will continue to be helped by the Federal Reserve making it clear that rates will likely keep rising at a measured pace through the rest of the year.

However, in the shorter term, "a natural retrenchment is probably due," Brodie said.

Low volume in August can make for volatile trade as many investors spend more time on the beach than in the boardroom.

Wall Street is closer to full attendance in September, but rather than usher in a wave of new money, the increased attendance tends to bring a greater number of people wanting to prune their portfolios into better shape by the end of the third quarter.

"There tends to be a lot of housecleaning from institutions in September," said Jeffrey Hirsch, editor-in-chief of the Stock Trader's Almanac.

Hirsch said that an unusually strong stock market in July was also likely to hurt stocks in September and October. Since 1950, when the Dow gained at least 3.5 percent in July, it's followed by a second-half of the year low sometime between mid-September and mid-December, Hirsch said. The Dow gained 3.6 percent this past July.

The good news is that a strong July typically brings a good buying opportunity in the fall as the market makes a low, Hirsch said, as it then leads into the November through January period, which constitutes the best three months of the year.

A correction over the next few months would be a positive for the market, said Paul Rabbit, president of Rabbit Capital Management. "I think the year will ultimately be positive, but we need some consolidation in the short term.

Rabbit said that due to the 'best of all worlds' scenario the stock market has had for the last two months, sentiment has gotten extremely bullish. That's demonstrated by the fact that what's been driving the market lately are the high beta, or riskier and more volatile stocks. Should the market correct, those stocks will get hit hard.

"That said, the economy is strong, earnings have been good and we've withstood higher energy prices," Rabbit added. "You can't complain about the fundamentals."