Big data used to maximise performance at eBay

One sector that is set to benefit from increased adoption of big data tools in the coming years is retail, with ecommerce firms particularly well-placed to make the most of the growing amounts of customer information they gather.

Recent research from Frost and Sullivan revealed that in Europe alone, retailers spent €59 billion ($81.24 billion) on IT services in 2012. This is set to rise to €75 billion by 2017 – and big data solutions will be one of the key factors in this.

One firm that is looking to use this technology to improve customer satisfaction and optimise selling prices is eBay. The online auction house's principal architect Tom Fastner told an audience at a recent conference about how it has been meeting the challenges of dealing with the huge amounts of information it has.

"We have data about everything," V3 reported him as saying. "We know what [customers] look at, we know what they like, we know what they buy."

Around 12TB of data a day is created through the monitoring of eBay's 100 million users – which registers everything from what buttons they click to what products they buy. This is then continually added to a four petabyte table containing four trillion rows of information.

However, gathering this is only half the challenge, as eBay needs to swiftly analyze this in order to get real insight – which is where it relies on tools such as Hadoop. One of the strengths of this platform is it can handle any type of data, including images.

"We were wondering if the quality of the picture in a listing had an impact on selling price," Mr Fastner said. "To do that we moved a couple of petabytes of pictures from our picture servers to Hadoop, analyzed the pictures and got some more structured information such as how much they were sold for, how many people viewed."

This proved that there is a relationship between picture quality and selling price, which helps inform the company and its sellers of what they need to do to get the best results.