Whole Foods founder and co-CEO John Mackey really envies his rival Trader Joe's.

Not because Trader Joe's has many fans. Or because its Hawaiian shirt-wearing workers seem to have a lot of fun.

What he envies is that Trader Joe's is a private company.

"Companies have less incentive and motivation to go public," Mackey said during a panel discussion at the Milken Global Conference in Los Angeles. "I have great envy for some of my private competitors. Trader Joe's is a great example of a company that nobody knows anything about them."

Whole Foods (WFM) is listed on a stock exchange and has to release quarterly earnings and disclose a litany of information to the Securities & Exchange Commission.

Mackey says publicly-traded companies are "under this microscope all the time."

Related: Price wars: Trader Joe's is beating Whole Foods

Whole Foods certainly has been under scrutiny. But the recent attention didn't have much to do with whether it was a a public or private company.

Last June, Whole Foods came under fire for overstating the weight of pre-packed goods and "systemically overcharging" customers for them. Whole Foods was investigated by New York City's Department of Consumer Affairs. It came after Whole Foods had already paid a fine in California on allegations of overcharging customers there.

Whole Foods first apologized for the New York pricing errors, saying they were unintentional. But it finally ended up paying a fine of $500,000.

Related: Whole Foods shoppers: the price isn't right

But regulation is just one hurdle for Whole Foods, which faces stiff competition from organic rivals like Fresh Market, Sprout and Trader Joe's to bigger supermarkets like Walmart (WMT)and Costco (COST).

In September, in an effort to lower prices, Whole Foods announced it would cut 1,500 jobs. That strategy has yet to bear fruit: a report in March by Deutsche Bank analyst Karen Short found that a basket of 77 products was 21% cheaper at Trader Joe's than Whole Foods.

Mackey and his co-CEO Walter Robb are determined to shake off the company's "Whole Paycheck" image.

All the bad publicity has hurt. Since the New York allegations in June, Whole Foods' stock is down 30%. In the first quarter this year, Whole Foods same store sales declined 1.8% compared to a year ago and profits declined about 6%.

Related: Weird job interview questions

Still Mackey argues that the incentive to become a public company is dwindling fast: Why would big private companies like Uber and Trader Joe's want to open their books?

"They fly under the radar largely and Whole Foods never does," Mackey said. "Everybody sees what we're doing. We can't see what our competitors are doing if they're private."

Some agree. Blake Mycoskie, founder of Toms Shoes, spoke on the panel with Mackey and expressed little, if any, interest to take Toms public for now.

"I don't have a desire to rush to go public," Mycoskie said after listening to Mackey's grievances.