As the temperature plunged into the low teens on a January morning in Camden, N.J., seven aid workers set out on a daunting mission: to count the homeless in the nation's poorest city.

Making rounds to shelters, warming centers, soup kitchens and "tent cities," the workers and volunteers helping out with this year's count were trying to find as many homeless people as they could within 24 hours.

The federal government requires communities to conduct a homeless count at least every other year to qualify for funding for homeless services like emergency shelter, food and job training.

In Camden, the need is huge. No city in the country is poorer: The poverty rate is roughly 40% and the median household income is a mere $26,000 -- compared to $51,000 nationally. It's also the most dangerous city in the country; drugs are everywhere and unemployment stands at 13%.

Social services agencies across the country already had their federal funding cut by at least 5% last year. That led many organizations in Camden to run out of money by October, says Shantel Garner, a project specialist at Community Planning & Advocacy Council, the nonprofit in charge of organizing Camden's homeless count.

Related: Camden's homeless share their stories

Advocacy groups like Garner's conduct these Point-in-Time counts, and the results help the U.S. Department of Housing and Urban Development determine local and national trends in homelessness and assess need in individual areas.

Last year's homeless count found only 589 homeless people in the 513,000-person county of Camden, and homeless advocates say this number is significantly understated.

CNNMoney joined Garner when she started counting people around 8 p.m. on Tuesday night. Her first stop was Joseph's House, a 75-bed shelter that opened a week earlier.

"Where will you spend the night of Tuesday, Jan. 28?" Garner asks Kareim Nurdeen, a 48-year old who struggles with schizophrenia and was evicted from his home a couple months ago after falling behind on rent.

Garner rattles off a list of 18 options from HUD's survey, including emergency shelter, hotel or motel, permanent housing, hospital, or jail. "Emergency shelter," Nurdeen answers.

Next come questions about how long he has been in his "current living situation," whether he has been homeless at least four times in the last three years, and what sources of income and non-cash benefits he receives.

Related: Homeless college students seek shelter during breaks

There are only about 10 homeless people at Joseph's House before dinner and Garner surveys them all. She leaves a pile of surveys with the director to fill out for the other 65 or so people expected to show up later in the evening.

Garner then moves on to the next shelter on her list: New Life Ministries, a church that started sheltering the homeless two weeks earlier after a "Code Blue" alert was issued -- allowing police to pick up the homeless and bring them to organizations that open their doors when temperatures drop below freezing.

HUD sets the count for January so the cold weather drives more people into shelters, making it easier to find them. But at New Life, only one person had arrived by 11 p.m., causing Garner to worry that it was so bitterly cold this year that more homeless people were resorting to alternative shelters -- like friends' basements, emergency rooms and abandoned houses.

At 6 a.m. the next morning, other nonprofit workers and volunteers from Garner's agency and other local aid groups drove around in a van hunting for homeless people who don't live in shelters. Among their stops: "tent cities" scattered just off the highways where dozens of homeless people live, along with a population of stray cats.

Related: Pay $2,000 to be homeless

Aaron Howe is the unofficial "mayor" of "little tent city" ("big tent city" was disbanded last year after the city deemed it unsanitary). He is responsible for distributing supplies from aid organizations and deciding who gets to stay and who needs to leave. Howe moved into his tent two years ago, after the trucking business he says he ran for 18 years collapsed. When he set up his tent, there were only three people here. Now there are 17 -- all of which Garner says were counted this year.

But that's not always the case. A big criticism of the homeless counts is that it misses people -- those who are couch surfing or staying in a prison, hospital or motel, for example. Others are in hiding, while many more won't admit that they are homeless.

"It's set up to be kind of an impossible task," said Maria Foscarinis, founder of the National Law Center on Homelessness & Poverty. She said one way to get more accurate findings would be to extend the count over a longer period of time or to do multiple counts per year.

Several homeless people CNNMoney interviewed in Camden said they had never been counted. Michael Powell lives in a "tent suburb," on the outskirts of one of the tent cities. He wears a key to the padlock on his tent around his neck, and behind the layers of tarps he uses as insulation sits a mattress propped up on crates, a knife and his "lady friend." Despite being homeless for 14 years -- ever since he got out of jail for murder -- he has never been asked to fill out a homeless count survey and had no idea the count existed.

"How would you possibly do that? There's so many of us and nobody wants to be seen," said Powell.

Related: Healthy food chain helps kids get off the streets

Howe of tent city agrees. "There's people that live in abandoned houses, there's people that live under bridges, I've met people that sleep in their cars at night ... you'll never get an accurate count," he said. "The organizations that come out here [to count], they don't know where everyone is."

Mark Johnston, acting assistant secretary in the Office of Community Planning and Development at HUD, stresses that the count is just one of 50 to 60 factors it looks at when determining funding -- including how successful agencies are at getting people jobs and housing. He acknowledges that the count isn't perfect, but says it at least provides a rough estimate, which is crucial to have before homelessness can be reduced.

"[The count] certainly isn't scientific, and I'm sure there are some people we're missing ... but we don't want to let perfection get in the way of solving the problem," said Johnston.