Papa John won't leave Papa John's alone.

The company has tried hard to distance itself from founder John Schnatter, who blamed the NFL for poor pizza sales last fall and then admitted using the N-word on a conference call this spring.

Sales slumped, and Papa John's stock declined. CEO Steve Ritchie and the current leadership blame Schnatter, who stepped down as chairman in July.

Schnatter told CNNMoney in an interview Tuesday that he's being scapegoated.

"You can't blame everything on two comments," he said. "I wish I had that kind of power, but I don't."

Instead, Schnatter pinned the company's problems on Ritchie, who became CEO in January. Schnatter described him as a poor leader who has created a culture of intimidation at Papa John's (PZZA), and let quality and customer service slip.

"We need new leadership," he said. "He struggles as a CEO."

"Steve'll make a great executive somewhere else," he added. "He's just the wrong guy for the job."

He described upper management under Ritchie as vindictive and controlling.

"People right now are scared to talk," he said.

Schnatter, who is still the largest shareholder and owns almost a third of the company, insists he doesn't want to return as CEO. But he has mounted a scorched-earth campaign to drive Ritchie out of his job.

In a letter to franchisees posted to his personal website on Monday, the founder said the company is struggling because of "rot at the top."

In a statement, Papa John's called the accusations "untrue and disparaging," characterizing them as "a self-serving attempt to distract from the damaging impact his own words and actions have had on the company and our stakeholders."

"John Schnatter also publicly supported Steve Ritchie's appointment as CEO at the end of last year," the statement said.

Schnatter flipped that argument around.

"There's a little bit of a farce going on here," he said. "Steve Ritchie promises great things, and then bad things happen, and then he blames somebody else."

Schnatter's lawyer Garland Kelley said the company allowed Schnatter's comments about the NFL and his use of the racial slur to be misrepresented in the press.

"There's a critical disconnect between what John actually says and how the company permits it to be portrayed publicly," Kelley said. "We think there's a reason this is occurring."

In July, Forbes reported that Schnatter had used the N-word while on a conference call with his marketing agency.

"Colonel Sanders called blacks n-----s," Schnatter reportedly said during a training on how to avoid gaffes like the NFL comments. Forbes said Schnatter was complaining that Sanders didn't receive backlash, even though his comments were worse than Schnatter's own. KFC did not immediately respond to a request for comment.

After the Forbes story broke, Schnatter apologized and resigned as chairman.

On Tuesday, he described the conversation differently.

"I simply said, 'Colonel Sanders said what he said, and we're not going to say that,'" he said, adding that he regrets saying the slur.

"What I said was anti-racist," he added. "I don't talk that way."

"I think the company has made the situation a lot worse," Schnatter said. "[The comment has] been misquoted, it's out of context, it's been portrayed in a way that's not truthful. But I'm still going to feel bad about that."

"I love my employees. I love my franchisees," he said. "For anything that hurts them, then I'm going to feel bad about that, and I do."

Schnatter also thinks his remarks about the NFL leadership have been misconstrued.

Last year, some NFL players knelt during the National Anthem to protest treatment of black Americans, particularly by police. The protests sparked a controversy, and the NFL put a final decision on the protest on hold.

"I felt like the situation was not a winning situation for the fans, the sponsors, the players and the owners," he continued.

A few months after Schnatter called out NFL leadership, Papa John's ended its NFL sponsorship. Pizza Hut took its place.

Papa John's is trying to put both comments, and Schnatter himself, behind it.

The company is stripping Schnatter's image from its marketing materials and has taken the unusual step of approving a provision that would prevent him from gaining more control of the company.

Ritchie went on a listening tour, mandated bias training for all employees and promised to increase diversity among staff. The company also launched a social campaign acknowledging customers' concerns.

Papa John's has also commissioned an investigation into its diversity and inclusion practices.

Asked whether the investigation would find any examples of misconduct by him, Schnatter said: "At the end of the day, I'm the principal owner of the company."

"They've got to point bad results on somebody, and that's probably going to be me."

Correction: A previous version of this article misstated the NFL's position about players kneeling during the Anthem.