After a 42-year career on Wall Street, Fred Dickson chuckles when people ask him why the job is so stressful.

"It's part of the nature of the beast," he says.

When Dickson started on Wall Street in 1973, Americans were waiting in long lines to buy gas, and an oil embargo sent the world economy into a recession.

As if that wasn't bad enough, stocks plunged again the following year when President Richard Nixon resigned because of Watergate.

"A number of Wall Street firms collapsed in that very significant downturn," he says. He was at Goldman Sachs (GS) at the time. "While I didn't expect it to collapse, it was ugly."

Despite the tough start, Dickson managed to survive -- and thrive -- in stints at Goldman and Lehman Brothers and later at a brokerage firm in Portland. He carved out his niche as an investment strategist, helping clients figure out how to navigate the markets. At times, he managed billions of dollars of money.

He is retiring this year.

"Wall Street can be a narcotic," he says, describing New York as the "financial emotional epicenter" of America -- and the world. He uses the term "emotional" since people on Wall Street feel the need to react in nanoseconds, often without much thought.

Wall Street also breeds a sense of self importance in the people who work there.

The only time Dickson was fired in his career was for what he dubs a "personal attitude adjustment."

"I was suffering from minor points of delusion of grandeur and that caught up with me. And it always does," he says.

He was 41 at the time. Like most mid-career financial types, he thought he was doing a terrific job and riding high on performance, but he let arrogance creep in and overwhelm his day-to-day decision making.

"People don't talk about getting fired, but I learned from that. I learned that you have to keep growing every day," he admits, with the humility that comes from hindsight.

Related: Sallie Krawcheck's 10 tips for succeeding on Wall Street

Dickson bounced back quickly from getting the ax. He landed a prestigious post at Lehman Brothers. Then October 19, 1987 hit.

"That was the day the Dow dropped 22% in about five trading hours and nobody had any idea what was going on," he says.

The Federal Reserve, with Alan Greenspan at the helm, came to the rescue.

But Dickson's most stressful day in his career was September 15, 2008, the day Lehman Brothers filed for bankruptcy.

"We really didn't understand that corporations like General Electric (GE) were financing themselves with commercial paper daily and they had to have a marketplace and Lehman was that marketplace," he says.

By then, Dickson was out in Oregon, but the repercussions were felt everywhere.

Dickson left New York City in 2001 to join D.A. Davidson, a brokerage in Portland, Oregon. Headhunters called him often with offers to return to Manhattan and make four to five times what he was earning. He turned them all down. He had been there, done that.

Seeing the financial world from outside New York has given him more insight into what many Americans think of Wall Street. When he started in the industry in the 1970s, investors had a "reasonable" level of trust in Wall Street. That is no longer the case.

Dickson thinks that's the key reason investors are holding record amounts of cash at the moment.

Related: Investors are getting out of U.S. stocks

One of the biggest changes he's seen over the years is the rise of technology and information about companies and stocks.

"Seventy and 80-year-old grandmothers are wired and looking at how their portfolios are doing -- a little bit too often," he jokes. "We've come a long way from the days when companies would only do print interviews and people would call their brokers to find out what was happening in the markets or with company earnings information."

Lately, he's been asked a lot about his views on high-frequency trading. Michael Lewis slammed the practice in his recent book "Flashboys," going as far as to claim the markets are rigged.

Related: Michael Lewis says markets are rigged

Dickson said that he's told most individual investors that high-frequency trading has had "very little impact" on them.

He remembers how when he first got into the business, orders were sent to so-called "specialists" who supposedly negotiated prices for you. These people basically did what high speed computers do now. And they took a lot bigger fees for their work.

In those days, the specialists would make 12 cents or even a quarter a share. They justified this because they were supposedly stabilizing the market, Dickson explains. Today, high-frequency traders get something akin to 0.00125 cents a share, if that.

Overall, Dickson thinks the stock market has become much more efficient, and that has helped everyone.

He's pleased that people are taking greater interest in their investments, even if it means he's gotten a lot more calls in the past decade from people who read things on blogs that don't always offer sound advice.

"People are more responsible for their own financial future than they ever were in the past," he says.