NEW YORK (CNNMoney.com) - In the heated debate over the impact of illegal immigration on the U.S. economy, Andrew Sum is one of those focusing on the negative.

The economist - the director of labor market studies at Northeastern University in Boston - argues that the large supply of immigrants has displaced low-skilled U.S.-born workers, particularly the young and the poor, from jobs.

Protestors wave American, Mexican and Guatemalan flags as thousands march at an immigration rally in Homestead, Fla.

Immigration reform No ditches and dirty plates Small business owners are sweating over Congress' debates because they fear a future without 'guest workers.' (Full story.) More stories Debate over the jobs Americans 'just won't do' Wal-Mart pays $11M over illegal labor Notes From the Underground Economy

"About 85.5 of every 100 new workers are new immigrants in this decade," he said. "At no time in the last 60 years have we come close to this. They're really displacing young workers at a very high rate."

But even Sum would concede that the U.S. economy is larger, and growing faster, due to the supply of illegal immigrants, and that most Americans with higher job skills are better off for their presence.

"Without the immigrants, we would have a decline in labor force of 3 to 4 percent," he said. "We couldn't have grown nearly as much as we did in the '90s if we didn't have immigrants. And in the last few years our growth would have been slower. The only thing I've argued is that we've ignored that illegal immigration has put a lot of young adults into economic jeopardy."

Sum's views point out the dichotomy that many economists see when looking at the impact of immigration on the economy.

Few economists will argue with the concept that the economy is stronger for the presence of the low-cost labor force.

And while most admit they have to make guesses rather than the educated estimates they would like to make, most say that economic growth would be a half a percentage point to 2 points lower without immigrant workers..

But even most of those who think it's good for the economy do see an impact on lower-skilled U.S.-born workers.

Few economists expect the economy to take a noticeable hit Monday from the call for immigrants to stay away from work and take part in protests against legislation that will crack down on illegal immigration.

"It's only for a day; much of the work not done on Monday is just going to be made up for on the week afterwards," said Benjamin Powell, senior fellow at the Independent Institute, an Oakland-based think tank.

But Powell and many economists say that the economy would face significant problems if there was any significant cut in the amount of immigrant labor coming into the country.

"Immigration is actually critical," said Bernard Baumohl, executive director of the Economic Outlook Group, a research group in Princeton Junction, N.J. "It allows the U.S. economy to grow more rapidly without higher inflation pressures."

Some economists argue that not only do U.S. consumers benefit from lower prices as a result of the low wages most immigrants are paid, but that the availability of lower-wage labor helps create more work for higher-skilled, higher-paid workers who are generally native born.

"If I'm a builder and I can hire more wallboard guys cheaply, my (ability to use) skilled carpenters goes up," said Northeastern's Sum.

And home building is one sector that might take the biggest hit from a cut off of immigration.

The Bureau of Labor Statistics estimates that 22 percent of construction workers are foreign born, with 2.4 million immigrants working in the sector, the largest source of jobs for immigrant labor. Jerry Howard, CEO of the National Association of Home Builders, estimates that 25 to 30 percent of those working in resident construction are immigrants -- though no one knows how many are here legally.

"You take 30 percent of the labor out of any sector and you're going to have serious impact," said Howard. "The costs would go up and it would suppress demand to some extent because of the higher costs."

Howard said that in some regions of the country, such as Buffalo, N.Y., very few construction workers are foreign born, while in California, Texas and other places immigrants account for up to half of workers on construction sites. And he said that some working in skilled jobs, such as stone masonry, would be difficult to replace.

Howard says that even without foreign-born labor on home construction sites, record home sales in recent years have been aided by immigrants, even if they're not buying the six-figure homes that some of them are helping to build.

"The four cornerstones driving demand for building has been boomers buying second homes, (children of baby boomers) coming into the market, people using equity to renovate existing homes and immigration," he said. "You take away any one of those cornerstones, it's going to affect the market for all home sales."

Some economists say that if immigrant workers weren't present, rather than native-born workers getting better wages to do the same jobs, many jobs done by immigrants might not get done at all.

If immigration reform pushed wages higher for lower-skilled workers that would probably stop many average Americans from hiring household help they can now afford. The same is true for some manufacturers and service sector employers as well.

"The average wage of the low-income American would be higher. But some of those jobs wouldn't get done at all and output would be lower," said David Wyss, chief economist for Standard & Poor's if immigration reform reduces the low-wage labor pool.

More economic news ... click here

A crackdown in illegal immigration in 2004 caused a shortage of workers needed to bring in the lettuce crop in the Western United States, said Powell, which he said caused a $1 billion loss for the industry as many growers had to leave their fields unharvested.

"To hire Americans to do it, they would have had to raise wages so far, it wouldn't have been worth it for them," said Powell at the Independent Institute. "It caused less of a loss to leave the crop to rot."

As for complaints that many critics of immigration cite - demand for social and government services by immigrants - most economists believe that is outweighed by the increased economic activity, even if some specific school districts or public hospitals struggle with the costs associated serving the immigrant community.

-------------------------------

Program note: Find out how the immigrant walk out could affect you. Watch Lou Dobbs Tonight, 6 p.m. ET on CNN.

Businesses brace for immigration rally. Full story.

Immigration reform could cost you: More here.

For a look at how immigrants have lower unemployment rate than native-born workers, click here.

For full CNN.com coverage of Monday's protests, click here.