The trickle of jobs leaving Brexit Britain has yet to turn into a flood.

Several big financial services firms have announced they will move about 9,000 jobs out of the country in order to safeguard their business once the U.K. leaves the European Union.

That is not yet showing up in employment data. The jobless rate was just 4.6% in the three months to March 31, according to data published Wednesday.

But experts say there could be a much larger exodus of jobs to come, and the scale will depend on the future ties between the U.K. and the EU, its biggest trade partner. Weaker economic growth will hurt employment too.

"We retain the view that jobs growth will slow over the coming months and unemployment will start creeping up later on in 2017," said Howard Archer, chief economist at IHS Markit.

Check out CNNMoney's Brexit jobs tracker

Prime Minister Theresa May wants a clean break from the EU. If she is reelected in June, she plans to take the U.K. out of Europe's unified market, which allows the free flow of goods and workers across borders.

If May doesn't manage to negotiate a deal giving British firms full access to European customers, companies might have to shift operations to an EU member state. Tariffs and other barriers to trade may prompt others to shed jobs.

Banks go first

Banks are a good example of the problem. Under EU rules, they can operate freely across the bloc as long as they are based in one of the member states. Once the U.K. leaves the EU, British banks will likely lose these rights.

Consulting firm EY monitors 222 U.K. financial services firms. More than a quarter of them have announced that they are moving some staff or part of their operations out of the country, EY said.

Germany's Deutsche Bank (DB) is preparing to move up to 4,000 jobs from the U.K. Swiss bank UBS (UBS) will probably have to move about 1,000 positions abroad.

HSBC (HSBC), Britain's biggest bank, is planning to move 1,000 workers to its Paris office.

Hundreds of jobs at JPMorgan (JPM) and Goldman Sachs (GS) are also likely to go, and Credit Suisse (CSGKF) is currently considering alternative locations for some of its staff.

A few banks, such as Barclays (BCS), say they should be able to cope without moving staff.

Others may follow

But the risk to jobs posed by Brexit goes way beyond banking, insurance and related services.

If U.K. crashes out of the EU without a trade deal, its exporters will be hit by tariffs. They may have to relocate some of their production to Europe.

Carmakers that support hundreds of thousands of jobs in Britain are particularly vulnerable. They rely heavily on parts imported from the EU to manufacture cars in the U.K., only to ship them back to the EU.

In October, Japan's Nissan (NSANF) said it had been convinced to invest in its massive production facility in Sunderland after getting reassurances from the government about Brexit. That was before May announced her plans in detail.

Since then, Nissan has refused to confirm that the factory -- and its 7,000 jobs -- are safe, saying it may have to adjust its business depending on how Brexit pans out.

There are also many British companies at the other end of the supply chain, producing parts for companies within Europe. They are likely to be hit too.

A survey this week by the Chartered Institute of Procurement and Supply showed nearly half of European businesses expect to buy less from U.K. suppliers.

Airbus (EADSF) faces just such a supply chain challenge. The company spends around $5 billion a year with suppliers in the U.K., supporting tens of thousands of skilled jobs.

The company is lobbying the U.K. and governments in Europe to limit the introduction of new trade barriers.

Related: The harsh reality of Brexit is finally setting in

Several hundred British jobs will also go when two EU agencies currently based in London -- the European Banking Authority and the European Medicines Agency -- relocate.

European officials have made it clear the institutions must be based within an EU country -- and several member states are already lobbying to host them.

Falling wages, slower growth

While unemployment is currently very low, the country has been struggling with low wage growth since the global financial crisis. Wednesday's data showed wages fell in the three months to March, after taking inflation into account. They also fell in the three months to February.

"The worry today is less about jobs and more about pay, with spending power shrinking once again as inflation has surged," said Ian Stewart, chief economist at Deloitte.

Slower economic growth won't help the labor market, either.

The Bank of England expects the U.K. economy to grow by 1.7% this year. Before the referendum last year, the bank expected 2.3% growth in 2017.

Governor Mark Carney has warned Brits to expect a "squeeze" on living standards this year, because wage growth won't be sufficient to compensate for rising prices.

Inflation spiked to a four-year high of 2.7% in April, according to data published Tuesday. Much of that is due to a 14% fall in the pound since the Brexit referendum last June, which has made imports more expensive.

Jobs already moving out of the U.K.

Deutsche Bank -- up to 4,000 jobs
UBS -- 1,000 jobs
HSBC -- 1,000 jobs
JPMorgan Chase -- hundreds of jobs
Goldman Sachs -- hundreds of jobs
Credit Suisse