That sentiment comes through clearly in many of the emails and comments CNNMoney has received in response to articles about middle class struggles.
Marty Slover, 53, gets annoyed when he sees people living in subsidized housing that's nicer than what he can afford on his middle school teacher's salary. The Illinois resident added that he doesn't mind giving a helping hand to those who truly need it, but he's frustrated by those who could work but choose not to, especially when his taxes are funding their sloth.
"It's easy to get assistance and once you do, it's a way of life," said Slover, who says he has to watch every dollar. "We're working, working, paying all of these taxes and they are getting the benefits."
Another reader wrote that "we, middle class people, see people in the grocery stores buying things that we cannot afford, having phones that we cannot afford, and getting health care coverage that we lack ... all from money that is taken away from me."
Of course, not all middle class Americans think the poor are to blame for their own economic situation. Some say the government is at fault for constantly approving new mandates and regulations and then raising taxes on the middle class to pay for them.
Others blame big business, which they accuse of doing anything to make a buck and putting the screws to the American worker in the process.
And to a lesser extent, at least among CNNMoney readers, there is a sense that the rich are thriving while the average American continues to struggle. Incomes for the rich have soared in recent decades and the pay discrepancy between CEOs and regular workers has widened substantially.
It is certainly true that the number of people on public assistance has soared over the past few years, particularly in the wake of the Great Recession. For instance, more than 46 million people received food stamps in 2012, up from 28 million in 2008. Medicaid provided health coverage to 52.6 million people in June 2011, up from 43.6 million in 2008. These programs are funded with taxpayer dollars, which fuels the anger among some in the middle class.
Advocates for the poor, however, note that many people only depend on public assistance for a short period of time and then cycle off when their financial situation improves. Also, 91% of people receiving government help -- including Social Security and Medicare -- are either elderly, disabled or in working families, according to the left-leaning Center on Budget and Policy Priorities.
"People get hit with job loss or illness, but then they bounce back," said Melissa Boteach, director of the Poverty to Prosperity program at the Center for American Progress.
Blaming the poor for draining the nation's and taxpayers' coffers has long been embedded in the American psyche.
Former President Ronald Reagan decried the "welfare queen" -- whom he said created fake identities to collect Medicaid, food stamps and other government assistance -- when he ran for the presidential nomination in 1976. (There have since been several news reports, academic studies and a popular book that has debunked the notion of welfare queens, however.)
Class warfare was also a dominant theme in last year's presidential race, with President Obama stressing that the rich should pay their fair share and Republican candidate Mitt Romney highlighting that 47% of people are dependent on the government and pay no income tax.
It's this perceived lack of effort that enrages some readers, who also suffer from some misconceptions about how public assistance works.
"A good majority of them are just too lazy and sorry to work and pay their own way. The government gives them free food stamps (to buy cigarettes and whiskey), free housing so they can stay home and have more babies so they can get more money..." writes one reader, who resents having to pay for other people's health insurance.