Walmart turns to crowdsourcing to tackle big data skills shortage

One of the biggest challenges for many companies when they are embarking on a big data analytics project is ensuring they have people with the right skills on board to make the initiative a success.

This is something that is proving very difficult at the moment as there is a shortage of key skills. For instance, research by Gartner suggests more than half of business leaders feel a lack of talent is hindering their ability to carry out complex data analytics operations.

Therefore, enterprises need to be innovative in order to find the right personnel. It was reported by Smart Data Collective that one company taking a different approach is US retail giant Walmart, which employed crowdsourcing to assist with this.

Last year, the company turned to crowdsourced analytics competition platform Kaggle to help find top talent. This allows professional and 'armchair' data scientists to turn their skills to analytical problems submitted by companies, with the designer of the best solution being rewarded – in this case, with a job.

"The Kaggle competition created a buzz about Walmart and our analytics organisation," explained Mandar Thakur, senior recruiter for the retailer's technology division. "People always knew that Walmart generates and has a lot of data, but the best part was that this let people see how we are using it strategically."

Participants in the contest were provided with a range of historical sales data and associated events, such as clearances, and asked to develop models illustrating how these events would affect sales across a number of departments. As a result of the entries submitted, Walmart was able to hire several people into its analytics team – including some who would otherwise not have come to the company's attention.

For example, Mr Thakur says one successful entry would not have normally been considered for interview based on his CV alone, as while he had a strong background in physics, he had no formal analytics experience.

"He has a different skillset – and if we hadn't gone down the Kaggle route, we wouldn't have acquired him," Mr Thakur said.

As well as skills such as a working knowledge of Python or R, it is also vital that professionals working in Walmart's big data team have a strong understanding of the business. Therefore, the company works hard to ensure its staff continue learning once they have been hired.

For instance, it runs an Analytics Rotation Programme, which requires every new staff member with analytics responsibilities to spend a period of time working within every department, to gain knowledge of how data is used across the company.

"This allows them to combine their analytical knowledge – whether they have gained it in education or in work experience – and helps to assimilate that knowledge with what Walmart is doing in different pockets of their business," Mr Thakur continued. "Once they are on the ground running, they can run in the right direction."