Big data is, of course, a hot topic right now and financial services firms in particular are grappling with the challenges faced by size and scale of information available and how to capture, store, share and utilise it effectively. But while banks have yet to kick-start big recruitment plans around this, the industry is facing a talent shortage in this area.
The amount of data in floating around cyberspace is estimated to be an astounding 1.2 trillion gigabytes currently and is tipped to grow by 500% over the next five years. Financial services firms are, predictably, trying to harness this data – whether its web content, videos, social media streams, PDFs or Excel documents – in order to make better trades decisions and generate more revenue.
Already there are signs that banks are recruiting for this – there’s a raft of new IT roles around business intelligence, whether that’s consultants, programme managers, developers or data warehousing – and, as we’ve pointed to previously, the role of the chief data officer is going to become increasingly prevalent.
However, specialist banking IT recruiters tell us that few firms are currently citing ‘big data’ initiatives as the reason behind the hires.
Research by McKinsey released last year suggested that financial services and insurance firms were likely to gain substantially by the use of big data. More interestingly, it said that by 2018, in the US alone (admittedly across all sectors) there could be a shortage of 140,000-190,000 people with the necessary deep analytical skills as well as 1.5m managers and analysts with the know-how to make the right decisions on big data.
In the financial sector, however, most firms are taking the buy option on technology, which would obviously limit the number people recruited in-house. Apache’s Hadoop, an open source software project, is gaining some traction as a way to handle large datasets, but even then some people are needed within the banks and they're not exactly in plentiful supply.
As Neil Palmer, partner, advanced technology at Sungard, told Wall Street & Tech: “It's not very business user-friendly. You need technical people to plug in Hadoop. It's not like a Microsoft reporting service where you can generate reports and play with drag and drop. You need developers to build algos, to process the data. It's not close to a package solution that you can put on a trader's desk.”
Some argue that the skills necessary to harness big data don’t exist, and that the financial services firms embracing the technology are training their employees as and when new methodologies become apparent. The talent could also still be learning the skills at university and – as we’ve pointed to before – finance isn’t the first choice for technically-minded graduates.