The departures come just a few weeks after its Dutch parent separately announced 1,000 job losses at Barings itself, as part of a decision to pull back from global investment banking. ING sold the Barings US business to its rival ABN Amro for $275m (&euro303m) in January.
Charterhouse lost three sales traders last week - Steve Adams, Alan Monk and Kurt Mayer. Adams was a 27-year veteran at Charterhouse, while Monk had been there for 12 years. The securities firm also lost equity sales staff, notably Rachel Kemp, who joined Charterhouse when it took over the Sutherlands broking business.
There was also speculation that the gilts team, largely made up of former Sutherlands staff, might move, but it is thought Charterhouse has persuaded them to stay by offering generous golden handcuffs.
Bob Benton, chief executive of Charterhouse, could not be reached to comment on the exact number of people leaving or on the reasons for their sudden departure.
Some of the departures came last Thursday, a day after bonus payments and golden handcuffs awarded when ING Barings took over Charterhouse were paid out.
'We have lost a few long-serving people who we're very sorry to lose, but some result from the sort of natural turnover you expect at this time of the year,' said Mike Osborn, deputy chairman of Charterhouse.
Monk, contacted at home, seemed to indicate there had been a row. 'I don't think we all really wanted to leave, but I am not prepared to talk about why we all left.'
Meanwhile, ING's sale of Williams de Broë, the second London broking business in the group, appears to have stalled. Belgian financial services group Artesia pulled out of a deal earlier this year. It is not clear whether ING is still actively trying to sell the business or will keep it. John Millar and senior staff at de Broë are in a strong position to have a say in who buys the firm as they retain a 22% interest.