NEW YORK (CNN/Money) - When people learn that Nancy Jack and her husband, Bill Walsh, have four children, they usually have a similar response.
"Invariably you get a reaction -- it's a lot of 'How many do you have?' " says Jack, who is from Stamford, Conn. "They're surprised and like the idea of four, but they're scared to do it."
America's families have been shrinking since the mid-1960's, when the average household had 2.4 kids and 23 percent of homes had at least five individuals under one roof. Today, a typical family boasts 1.8 children and only one in 10 has five or more relatives at home.
Nevertheless, some brave -- and more affluent -- souls are bucking the trend.
While their numbers are too small to make a blip on national Census charts, such families are becoming more visible in leafy communities where moms can afford to stay at home and nannies become part of the family.
As such, they're slowly challenging traditional perceptions of who has lots of kids. Today, it's not just the devoutly religious with lots of tots. It's also the power professionals.
"The largest families I have tend to be very wealthy," says Suzanne Royer, president of Annie's Nannies, a nanny agency in Seattle that serves families in well-heeled Washington state communities like Redmond, Richmond, and Kirkland.
"The moms keep wanting to have another baby and they have so much help that it's possible," Royer continues. "They can afford it, whereas a lot of other middle-class moms might want another child but they know they can't, because they have to think about expenses like college."
That's the case with Melissa Gray from Mystic, Conn. She and her husband have two sons and are in the process of adopting a little girl from China. But they won't have any more children, even though Melissa was one of five siblings and knows the fun of a huge family.
"All of us have graduate degrees and my father put all five of us through school," says Gray remembering her childhood. "I've accepted the fact that I won't be able to put three children fully paid through university."
Though parents of large families balk at the suggestion that their children are some sort of status symbol, they will acknowledge that money does make it possible to go big.
"We are in some ways part of that demographic group that drives oversized cars," said Bill Valentine, a private money manager from Bend, Ore., whose fourth child is due on Christmas Day and whose wife, Jessica, is a stay-at-home mom.
"Anything that's good in two is great in four," he said. "We make relatively good money. That gives me the comfort to bring four kids into the world."
Or listen to Rob Connor, another father of four from Cazenovia, N.Y., with a spouse who's opted to stay home with the kids.
"We've been able to manage and live just fine," Connor said. "But I think were I making less it would become a very real consideration."
These days, the cost of raising one child to age 17 – including housing, food, transportation, clothing and health care – runs somewhere between $127,500 and $254,400, depending on a family's income, according to recent Agriculture Department statistics. The richer the family, the more it spends on things like private school, housing and other expenses.
Lots of kids -- for the wealthy only?
Valentine and his wife, Jessica, are like many other parents with lots of kids. He has a successful career that enables him to provide for their family. She left a high-profile job for a national sporting goods chain to be a mom.
Valentine says that where they live, it's "not politically incorrect for women to stay at home" with the kids. They know two other couples who also are expecting fourth babies.
"In a world where we don't have terrorist threats and are always making money, it's easy to justify less time with family," he said. "But Sept. 11, combined with the bear market of the past few years, has refocused us."
Part of that focus is devoting more time to children. So the Valentines, like many other large families with some means, are now hiring "reverse nannies."
Instead of whisking the children away -- only to present them to parents when small faces have been freshly scrubbed -- these hired helpers handle the barrage of daily chores like laundry, grocery shopping, cleaning and cooking. That way, mom can roll around on the nursery floor playing with the kids.
"It's a lot easier having four kids and a nanny than three kids and no nanny," said Nancy Jack, who quit her career as an environmental lawyer to be a mom.
To be sure, the tab for a live-in nanny and four kids can give anyone sticker shock, and even parents who euphemistically call themselves "comfortable" have to make adjustments.
YOUR E-MAIL ALERTS
Follow the news that matters to you. Create your own
alert to be notified on topics you're interested in.
Or, visit Popular Alerts
Jack says she and her husband (who is a vice president of a large international insurance firm) used to be "price indifferent." But their fourth child changed that.
"When I found out I was pregnant, we refinanced everything," said Jack.
The process helped the couple find cash in the household budget to hire their nanny. But there are bigger items the family will now have to do without.
"It's out of the question to put four children through private school," said Jack. "We feel we don't have that option anymore."
Others make similar sacrifices even as they note that there was a time when it didn't seem so difficult to raise four or more kids.
In Connor's own youth, "there were a lot of big families, with five, six, seven, eight kids," he said. "I can't imagine it today. I think a big change is more and more families need two incomes to live the way they want to live."
Though Connor and his wife, Mary, manage to pay the bills on his salary, he acknowledged he "can't even begin to think about college. We're woefully unprepared for that. But we have an optimistic view of the future that things will work out."
In the meantime, the Connors say it's not just money that can fall in short supply all too quickly. Like other parents, they want more time to lavish on their children, too.
"You might get lucky and find tremendous caregivers," he said. "But there's no substitute for good parenting."