Mitt Romney's videotaped remarks made public this week point to at least one true fact: Nearly half of American taxpayers do indeed end up paying no federal income tax.

In footage taped with a hidden camera at a Romney fundraiser in May, the Republican presidential candidate describes the "47% of the people who will vote [for President Obama] no matter what." He says they are "dependent on government" and "believe they are victims." They "pay no income tax," he says.

In fact, the main reason so many Americans end up with no federal income tax liability is Congress' addiction to tax breaks -- tax breaks both Democrats and Republicans have supported. (Related: Obama v. Romney on taxes)

Intentionally or not, Congress set it up this way.

Over the years, lawmakers have increased the number of non-payers by enacting reams of tax breaks -- such as tax credits for work, kids and education.

The Bush tax cuts, first passed more than a decade ago, expanded many of them. Indeed, if the Bush tax cuts are allowed to expire the number of non-payers could drop significantly, according to Roberton Williams, a senior fellow at the Tax Policy Center.

Related: Billionaires with 1% tax rates

Lawmakers also approved a number of temporary tax breaks after the 2008 financial crisis to stimulate the economy.

The Tax Policy Center has estimated that in tax year 2011, 46% of households owed nothing in federal income taxes. About half are nontaxable strictly because of tax breaks. The other half didn't owe federal income tax because of exemptions allowed for subsistence level income and dependents.

The No Tax Club includes some of the very rich -- a few thousand a year usually.

But the group of nonpayers is made up disproportionately of low- and lower-middle-income households, including seniors. More than 90% of all the those who had no federal income tax liability in 2011 had income below $50,000.

And if payroll taxes are counted, the number of non-payer households drops precipitously -- to an estimated 18% in 2011.

Why? Because payroll taxes, which fund the big entitlement programs, hit low-income households harder than most since 100% of their income is subject to them.

By contrast, high-income filers pay a lower percentage of their income in payroll taxes since wages subject to the Social Security tax are capped at $110,100.

Of course, that's all just about federal taxes paid on income. Once sales taxes as well as state and local income taxes are thrown into the mix, virtually no household in America would qualify for the non-payer group.