‘Do We Really Need to Mind the Big Data Skills Gap?’

– Or do we just need data signal boxes and navigators?

As we enter a new year there have already been numerous articles about big data and technology trends for 2014 – and many of them have highlighted the big data skills shortage.

Computer Weekly’s article on ‘Dark data could halt big data’s path to success’ states that having sufficient skilled users is rated as the next biggest challenge after dark data. 34 per cent of early adopters either outsource projects or bring in outside expertise (13 per cent). A further 33 per cent say they would need to identify and train internal specialists, with the remaining third relying on existing in-house expertise.

Furthermore, the 2013 Harvey Nash CIO Survey reported that one in four CIOs have difficulty finding qualified staff for big data projects.

“Skills have been an issue from the beginning, and this will remain an issue for the foreseeable future,” says Gartner’s Buytendijk. “How do you find people who have a background in econometrics, statistics and mathematics, and who know how to programme in modern environments and have business sense, because big data analytics is all about interpreting context, why something is happening in a certain context. This skillset is really, really hard to find.”

We’ve found a skills gap ourselves when working with our customers in the telecoms industry. However, the skills shortage is often more perceived than real; there is indeed a shortage of data scientists (or people who have renamed themselves data scientists) – but they aren’t fundamental to big data.

The over-riding challenge is to be able to ask the right questions and have the application technology to deliver the knowledge; this is nothing new. All our customers know what questions to ask when they know what their applications are able to deliver. Once you have a direction in mind and a functioning signal box then any destination on the data underground is within reach.

Too often we’re worried about a shortage of data scientists and what their alchemy can magic from big data. Stay with your industry knowledge, trust your team and their history of delivery – but get the application to turn the data into knowledge.

Read more in our interview with Big Data Republic – ‘Visualizing an Exit for Your Big Data Naysayers‘. Or get in contact for more information on how we could help with your 2014 big data projects.