How HelloFresh embraced Hadoop

28

Nov
2016
Posted By : admin Comments are off
how HelloFresh embraced Hadoop
Categories :#AnalyticsNews

As businesses grow, it becomes more critical for them to have a solution that will effectively handle the increasing amounts of data they generate. However, one problem that many organisations find when they are expanding is that tools that were adequate when they were developed are not able to scale along with the company.

This was the problem facing Berlin-based home meal delivery firm HelloFresh. The five-year-old firm has expanded rapidly and now delivers more than 7.5 million meals a month to 800,000 subscribers in multiple countries. Therefore, it found itself quickly outgrowing the custom-made business intelligence system it had long relied on, and needed a new solution.

In a recent interview with InformationWeek, chief technology officer at the company Nuno Simaria explained how the company had been using a home-built business intelligence system based around PHP, using a mix of a relational database and key value storage for pre-calculated data. However, as the business grew, the limitations of this became clear.

One problem was it did not offer the flexibility or detail analysts needed. While it could track essential KPIs to provide details of what was happening within the business, it was unable to offer insight into the reasons behind any changes.

"It was definitely not a good idea, but at the time it was the technology we were most comfortable with," Mr Simaria said.

The system was also approaching the limits of its capacity, so it became obvious a change was required. The company looked at several options that would offer improved big data analytics performance, including MemSQL and SAP HANA, but ultimately, it was Apache Hadoop that won out.

Part of the reason for this was its low cost compared with competitors. Because the tools can offer high performance even on inexpensive commodity hardware, there was no need for HelloFresh to upgrade these areas. This made Hadoop a highly attractive option, even though the company's team did not have much familiarity with the technology.

This led to its own challenges. Mr Simaria explained that finding skilled engineers in the market was very difficult. Therefore, the firm's approach was to give two of its existing staff the time and resources they needed to learn about the tools.

"We'll give you the budget, and we'll give you the time," he said. "This is something we've done with other technologies as well. If it is not easy for us to access talent in the market in the short term, we will empower our developers and our engineers who are interested in problem solving, and we will let them discover the complexities of that technology."

At the end of this process, the engineers had to answer three questions: is Hadoop the right technology; how can the firm migrate existing resources to it; and what distribution should be used moving forward?

The result of the Hadoop deployment is that HelloFresh now has much faster insight into goings-on within the businesses, and is also able to delve much deeper into its data in order to uncover insight.

Mr Simaria said: "This technology has allowed us to spread data-driven decision-making to anyone in the organisation, from local teams to global finance to whoever needs to use data insights to make decisions."

Joe Biden: Big data could assist cancer research

29

Feb
2016
Posted By : admin Comments are off
Joe Biden: Big data could assist cancer research
Categories :#AnalyticsNews

Big data is making a huge difference to a number of companies and industries, allowing professionals to handle data more effectively than ever before and gather fascinating new insights. 

As the number of devices capable of connecting online increases, it is vital that companies find ways of interpreting data that can help to improve their products and services. Sectors such as marketing, retail and insurance have already reaped the benefits of big data. 

However, one area that is perhaps understated is how the technology could help to improve research into diseases, particularly cancer. Vice-president of the US Joe Biden recently met up with healthcare specialists in Utah and explained how a better approach for sharing information will be necessary in order for new treatments to be realised. 

How has big data helped to treat cancer?

Speaking to The Spectrum, Mr Biden emphasised how big data is helping to trace genetic and environmental factors that influence the disease, giving practitioners new knowledge that can be used to enhance their understanding. 

Mary Beckerle, Huntsman Institute CEO, told the news provider: “Half of those folks who succumb to cancer succumb to a cancer that could have been prevented. I think there's a really important emphasis to treat cancers that could have been prevented.”

Mr Biden has been visiting hundreds of doctors in a bit to improve federal engagement on curing cancer. He visited Duke University earlier this month and is set to make an appearance at the University of California San Francisco today (February 29th). 

The work is part of the White House’s cancer “moonshot” initiative, which aims to improve the level of progress towards curing cancer. As well as providing $1 billion (£721,000) towards research, the government is hoping to generate new ideas for cancer treatment specialists across the country. 

President Barack Obama is asking Congress for $755 million for cancer research in the coming budget, which would be on top of the $195 million already given the green light by officials last year. 

What developments could occur in the coming years?

Big data is still growing in the market and many industries should be able to improve how they manage information and increase the quality of their products and services. 

In recent years, devices such as laptops, tablets and mobiles have all become key technologies for businesses as employees adopt more flexible ways of working and, with big data now introduced by many companies, there is more information for managers to decipher than ever before. 

As analytics technologies improve, companies can easily identify new ways of making money and producing better products and services. Big data allows organisations to easily gain insights from complex statistics and ensures that information is being used as effectively as possible. 

For causes such as cancer research, where thousands of documents have been created for organisations to look at, big data makes it easier than ever before to collate and understand information. Without it, sorting through large numbers of files and trying to extract insights from them can be a painstaking process. 

Facebook

Twitter

LinkedId