The use of Hadoop within the enterprise is showing strong growth as more businesses lookRead More
What skills will your organisation need to make big data work?
Successfully deploying a big data analytics programme can bring many benefits to a business, from better productivity and a clearer insight into customers, to higher revenue and reduced time to market for new products. But putting in place effective hardware and software solutions to achieve this will only be half the task.
The real benefits of big data can only be realised if companies have the appropriately skilled staff on board to manage such operations. These personnel will not only be essential in initiating and building an advanced big data solution, but ensuring that the right questions are being asked of it and the answers are being fed back to the people best placed to act on them.
A fight for talent
As the importance of skilled personnel is being increasingly widely recognised, competition for the best talent is fierce, particularly as individuals with the right combination of expertise and experience are proving hard to come by.
A 2014 report by Nesta and the Royal Statistical Society found 80 per cent of UK businesses said it is difficult to find the skills they need to meet growing demand. Typically, those with the right range of talents will be expensive to hire, while upskilling existing staff can be a challenging and time-consuming process.
This has come about as a result of the new generation of data analytics solutions, which are leading to organisations moving beyond traditional business intelligence activities. Today’s environment calls for solutions that are able to accurately predict future demand and customer behaviour, rather than just reacting to previous events. This will require professionals who are able to think outside the box and understand how to set up and engage with big data solutions to gain valuable and relevant answers.
The skills needed for success
The pinnacle of this will be the appointment of data scientists – experts with the right mix of understanding data, coding skills, statistical knowledge and a thorough understanding of the wider business environment. However, getting to this high value point can be a lengthy process and one that requires significant investment of resources.
Whether big data talent is being brought in or nurtured from within a company, there are a new skills that will be essential to success. According to a 2014 survey from recruitment company Dice, experience with NoSQL (Not-only-SQL) is the area in highest demand, with interest rising by more than 50 per cent on the previous year.
An understanding of key Hadoop components such as MapReduce, HDFS, YARN and Spark will be necessary for businesses to succeed in their big data implementation. Hadoop can be tricky to use due to a steep learning curve, but as it will have a central role to play in any big data applications, a strong grounding in the technology will be hugely advantageous.
As well as experience in programming for general-purposes languages such as Java, Python or C++, personnel with expertise in statistical and quantitative analysis tools like R, SAS and Stata will have a key role to play in successful initiatives.
Beyond the technology
Even if companies are able to find staff with all the coding and technical skills needed to operate a big data solution, it will not be much use if these professionals are isolated away in an IT department, where they have little idea or exposure of how their work will affect other parts of the organisations.
That’s why it’s important that businesses do not overlook so-called ‘soft’ skills when considering what will be required, such as a strong ability to ask questions, communicate and understand the business value and implications of any decision.
One of the best things organisations can do to support this and ensure they have the skills they need on hand is to create a dedicated big data team. The development of a big data competency centre – which may be on a departmental or enterprise-wide level – ensures that talent is placed in a centralised location, making it an integral part of a company’s decision-making, rather than an afterthought. Bringing skills together fosters innovation and peer review for improved creativity and faster delivery.
This also helps strengthen corporate attitudes to big data. By keeping the technology and capability visible across all parts of an organisation, it becomes an intrinsic part of a company’s culture. In the long term, this is likely to prove very beneficial, as integrating data analytics into every aspect of operations will help give all staff a better understanding of what the business is capable of and what they need to do to empower beneficial change.