Big Data knows when you've fallen on hard times.

Whether you're deep in debt or a senior on a fixed income, data brokers can track and sell your information to payday lenders, debt consolidation firms and other marketers.

The problem is the information can be used by businesses or scammers that prey on the poor and the elderly, lawmakers say.

For example, lists of frequent payday loan borrowers could be targeted for other high cost loans -- ones that could sink them even further into debt.

And lists of elderly consumers could be used by scammers posing as telemarketers.

Related: What your zip code reveals about you

How do data brokers get this information? By tracking the websites you browse, the credit cards you apply for and the surveys you take, among other sources.

One data broker, List Connection, markets a slice of the data called: "The Help Needed -- I'm 90 Days Behind with Bills."

List Connection's website says the leads from it are "excellent prospects for payday loan, secured credit card, debt consolidation, subprime credit, or financial assistance offers."

Ken Wood, vice president of management at List Connection, said the list is for firms trying to help people with bad credit.

Senator John "Jay" Rockefeller of West Virginia, called out this list in a February letter to NextMark, a website that advertises a variety of data broker lists. Rockefeller, who heads the Senate Commerce Committee, is sponsoring legislation that would allow consumers to see and correct the information collected about them and to opt out of the lists entirely.

Another list Rockefeller cited featured leads for sweepstakes offers.

The data broker, Multimedia Lists, markets it as a list featuring "highly responsive opportunity seekers" who are ripe for promotional offers, sweepstakes, contests and other "opportunities to make easy money."

Related: Big Data is secretly scoring you

Multimedia Lists did not respond to phone and email requests for comment.

Under the Fair Credit Reporting Act, marketers aren't allowed to use your sensitive credit information, like your credit score, unless it's to make a "firm" offer of credit, say for a credit card or auto loan.

But there is a lot of other information -- like whether or not you're facing foreclosure or get denied for a credit card -- that data brokers argue is fair game.

While data broker's lists are often used for harmless purposes, like sending catalogs, the growing use of all of this personal information is raising red flags among privacy advocates.

"We shouldn't use our most private information for just any marketing purpose," said Ed Mierzwinski, consumer program director at U.S. Public Interest Research Group. "We need additional protections."

PIRG is calling for increased transparency regarding who is buying our information and how it is being used.

Some large data brokers say they already impose controls on how clients use their data. Epsilon, for example, says it has strict use guidelines and ensures that its clients comply.

Meanwhile, the Direct Marketing Association said it expects data brokers and marketers to adhere to its set of ethical guidelines and reports egregious offenders to the Federal Trade Commission.

Related: Find out what Big Data knows about you

The organization noted that lists of financially vulnerable customers can be used for positive purposes as well, such as when a nonprofit is seeking people in need of services.

Still major industry players like Acxiom acknowledge that more could be done to protect consumers.

"Data could help people in financial stress or be used to harm them," said Acxiom's chief privacy officer Jennifer Barrett Glasgow at the conference. "We're getting into a very complex world where the ultimate use of the data... is where the controls need to be."