IoT and cloud ‘the future of Hadoop’


Posted By : admin Comments are off
Iot, cloud storage, hadoop, big data
Categories :#AnalyticsNews

The creator of Hadoop, Doug Cutting, has said that cloud computing and Internet of Things (IoT) applications will be the basis for the next phase of growth for the platform.

So far, most deployments of the big data analytics tool have been in large organisations in sectors such as finance, telecommunications and internet sectors, but this is changing as more use cases emerge for the technology.

Much of this is down to the growing use of digitally-connected sensors in almost all industries, which are generating huge amounts of data that businesses will need to quickly interpret if they are to make the most of the information available to them.

Mr Cutting highlighted several major companies that have already adopted HAdoop to help them handle this huge influx of sensor data.

"Caterpillar collects data from all of its machines," he said. "Tesla is able to gather more information than anyone else in the self-driving business, they're collecting information on actual road conditions, because they have cars sending all the data back. And Airbus is loading all their sensor data from planes into Hadoop, to understand and optimise their processes."

One sector that is on the verge of a revolution in how it manages information is the automotive industry, as a growing number of cars are being equipped with IoT sensors and networking capabilities.

Mr Cutting noted that almost every new car now sold has a cellular modem installed, while almost half of new cellular devices are not phones, but other connected items.

Until now, Hadoop has often been deployed as a key component of a 'data lake', where businesses pool all their incoming data into a single, centralised resource they can dip into in order to perform analytics. However, use cases for IoT typically have a need for data to be exchanged rapidly between end-devices and the central repository.

Therefore, there has been a focus recently on the development of new tools to facilitate this faster exchange of information, such as Flume and Kafka.

Mr Cutting particularly highlighted Apache Kudu as having a key role to play in this. He said: "What Kudu lets you do is update things in real-time. It's possible to do these things using HDFS but it's much more convenient to use Kudu if you're trying to model the current state of the world."

He also noted that while the majority of Hadoop applications are currently on-premises, cloud deployments are growing twice as fast, so it will be vital that providers can deliver ways to embrace this technology in their offerings.

"We are spending a lot of time on making our offerings work well in the cloud," Mr Cutting continued. "We're trying to provide really powerful high-level tools to make the lives of those delivering this tech a lot easier."

How big data helps the hospitality sector


Posted By : admin Comments are off
big data hospitality sector
Categories :#AnalyticsNews

The hospitality sector is a highly competitive part of the economy, with hotels in particular always under pressure to deliver the highest-quality experiences at the lowest cost possible. 

A key challenge for this industry is that in the age of constant connectivity, customers have higher expectations than ever before and will demand a personalised experience. If they do not get this, they will often not have to go far to find a competitor who will meet their needs.

Fortunately, there are steps hotels can take to deliver this service. Anil Kaul, chief executive of Absolutdata Analytics, wrote in a recent piece for Dataquest that big data analytics is a natural partner for the travel and hotel sector, due to the large amount of information that travellers generate.

"Hotel companies can use this data to personalise every experience they offer their guests, from suggesting local restaurants to finding an irresistible price point. They can also use this flood of data to fine-tune their own operations," he stated.

In the past, the hotel sector has not taken full advantage of this vast data source, as many companies did not know how to make the most of it. But as new developments such as mobile technology, powerful analytics solutions and more user-friendly dashboards become available, companies will be able to hugely expand their capabilities.

For example, Mr Kaul stated that on a person-to-person level, smartphone-enabled staff members can pull up instant information about their guests to alert them to needs or requests and help them respond accordingly.

On a wider level, big data can help hotels save money by cutting back on utilities when the location is not at full capacity. Local factors such as the weather or expected events can also be factored in, so room rates can be dynamically adjusted if a major conference is nearby, for example.

It can also help hotels determine which customers will offer the best lifetime value. For instance, Mr Kaul noted that while a guest on a special, once-in-a-lifetime holiday may spend a large amount in their visit, they are unlikely to offer repeat business. On the other hand, a frugal business traveller may seem like a less valuable customer, but if the hotel can make them happy, they could return on a regular basis for years to come.

By using big data analytics to study trends and identify what customers expect, hotels can better understand what they have to do to deliver a personal service and turn a one-time visitor into a repeat customer.

As well as improving the hotel's performance, a successful big data implementation will result in happier customers and an enhanced reputation for the hotel.

"Big data might still be in the adoption phase for the hotel industry, but it has a lot of benefits to offer," Mr Kaul said. "The data is there; it just needs to be put to work. Hotels that fully leverage it will gain a significant competitive edge."

Salaries on the rise for big data professionals


Posted By : admin Comments are off
Big data skills are in demand  Image: iStockphoto/cifotart
Categories :#AnalyticsNews

IT professionals specialising in big data are benefiting from growth in pay as employers show more demand for their skills, research has revealed.

In its latest Tech Cities Job Watch report, IT resourcing firm Experis revealed that average salaries for people with big data expertise have risen by almost eight per cent in a year.

That's nearly three per cent higher than the Bank of England's projected three per cent pay increase for the whole of Britain.

Experis' research is based on over 60,500 jobs advertised across five key tech disciplines: big data, cloud, IT security, mobile and web development.

The latest figures showed 5,148 big data jobs available in the first quarter of 2016, 87 per cent of which were based in London.

One of the key factors in the recent growth in this sector is the rising importance of personal data for businesses that want to improve their customer understanding and predict forthcoming trends.

Many companies are also bringing big data and compliance skills in-house to ensure they stay in line with new EU data protection regulations.

Geoff Smith, managing director at Experis, said big data will continue to be a "major driver" of UK economic growth as the digital revolution gathers pace.

"Yet, many companies have been slow to react and there's a limited talent pool to choose from," he added.

"Employers are willing to pay highly competitive salaries to attract these experts, so they can help with compliance, uncover valuable customer insights that can transform their business and innovate for the future."

Big data and the Internet of Things are set to add £322 billion to the UK economy within the next four years, according to a recent report from the Centre for Economics and Business Research and software provider SAS.

Manufacturers ‘failing to invest in IoT’


Posted By : admin Comments are off
The Internet of Things is set to grow in significance for businesses  Image: iStock/Hin255
Categories :#AnalyticsNews

The Internet of Things (IoT) is one field where many UK manufacturers could be investing more, research has suggested.

Business software provider SAP conducted a survey of 100 senior executives in the manufacturing sector, around a fifth (19 per cent) of whom expected no investment in IoT technologies this year.

As a proportion of overall IT spend, average anticipated investment in these innovations was only eight per cent.

This low level of financial backing for IoT tech and platforms is not the result of a lack of confidence in this burgeoning field, with just over half (51 per cent) of the respondents to the SAP survey saying it could help them cut costs.

Nearly four out of ten manufacturing executives (38 per cent) thought IoT could help their business with product development.

Despite the relatively low level of projected investment, Nayaki Nayyar, general manager and head of IoT Go to Market for SAP, said it was encouraging that businesses recognise how technology can deliver improved insights, proactive measures and better outcomes.

She said IoT is set to have a "major impact" on the manufacturing sector in particular.

"Industry 4.0 and the Internet of Things can enable end-to-end transformation for manufacturing companies and connect the shop floor to the top floor, optimising supply chains and manufacturing operations and ultimately helping them stay competitive," added Ms Nayyar.

Nearly a third (30 per cent) of the manufacturing firms surveyed by SAP identified procurement and supply chain management as areas that could benefit from growth in IoT.

This point was also made in the UK Logistics Confidence Index for the first half of 2016 from Barclays, Moore Stephens and Analytiqa, which was based on the opinions and insights of over 100 chief executives, managing directors and finance directors from the logistics sector.

Nearly six out of ten respondents (58 per cent) said they would be implementing innovative supply chain solutions over the next 12 months. Within this group, 18 per cent of decision makers were set to focus their efforts on IoT.

However, at the top of the list of technologies set to improve supply chain management over the coming year was big data and analytics, cited by 27 per cent of respondents, followed by automation (21 per cent) and cloud computing (21 per cent).

Publishing their findings, the report authors noted that many companies still have a long way to go before they can properly grasp and realise the potential of the latest technologies.

"Some industry observers suggest companies may still be at the beginning of a journey to fully understand the potential solutions that can be developed from the real-time data that is starting to be generated by the IoT," they said.

Last November, Gartner released a report predicting that the number of connected 'things' in use across the globe will reach 6.4 billion this year, up 30 per cent from 2015.

Executive involvement ‘boosts big data profitability’


Posted By : admin Comments are off
boosts big data profitablity
Categories :#AnalyticsNews

Companies that ensure business units play a key role in the development of big data analytics solutions are more than twice as likely to be profitable as those managed solely by the IT department.

This is according to new research by Capgemini and Informatica, which found that currently, less than a third of big data initiatives (27 per cent) are profitable. A further 45 per cent are breaking even, while 12 per cent are said to be losing money.

The study noted the majority of organisations therefore still have significant work to do in order to see a return on investment, those that have strong support from the C-suite are in a much better position.

Almost half of organisations (49 per cent) that had high levels of executive buy-in reported that their initiatives were profitable, compared with just six per cent of companies that had no executive support.

John Brahim, head of Capgemini's Insights and Data Global Practice team, commented: "The study provides insights into those organisations that are realising positive business impact from their big data investments. The companies that are reaping benefits are embracing business ownership of big data which drives a step-change in performance."

The study also found a significant split between the US and Europe when it comes to taking ownership of big data analytics projects, with almost two-thirds of European firms (64 per cent) having their projects controlled by the CIO, compared with just 39 per cent in the US.

Capgemini noted that projects that are led by the chief operating officer are the most likely to be progressing effectively, while organisations that are turning a profit from their big data also tend to be those that are most effective at managing data governance and quality.

Three-quarters of profitable respondents stated they had made excellent or very good progress in improving data quality and data governance, compared to 50 per cent overall.

"The survey findings show a direct correlation between the use of data quality and governance practices and profitable outcomes from big data projects," stated Amit Walia, executive vice-president and chief product officer at Informatica. 

He added: "Achieving business value repeatedly and sustainably requires focusing investments around the three key pillars of data management: big data integration, big data quality and governance, and big data security."

Capgemini offered several recommendations for businesses that are looking to make the most of their big data initiatives. 

For instance, it stated it will be vital to get buy-in from the very top in order for projects to be successful. Anything below boardroom level will not be enough to effect lasting change.

It also advised businesses to modernise their data warehousing systems and create a "robust, collaborative data governance framework" that enables organisations to react quickly, while also ensuring data security and data quality.

Security issues hindering big data adoption, survey warns


Posted By : admin Comments are off
security issues big data analytic adoption
Categories :#AnalyticsNews

The lack of a clear strategy for securing businesses' most sensitive information could be one of the major barriers that prevent companies from fully exploiting big data analytics, a new survey has warned.

Research conducted by Dataguise revealed that almost three-quarters of enterprises (73 per cent) report that their big data initiatives have been delayed or even abandoned altogether as a result of security concerns.

Even when companies have put in place multiple layers of defences, there is still a lack of confidence that data will be safe. Less than half of respondents (47 per cent) had faith in their security solutions.

One common issue is that too many people will have access to data. While four out of five companies (80 per cent) indicated their IT teams were able to access the business' most sensitive information, 40 per cent also said test and development teams also had access. More worryingly, nearly a third of firms (29 per cent) indicated that end-users throughout the enterprise maintained the ability to view sensitive information.

When it comes to testing the security measures of their big data initiatives, although 62 per cent stated their solutions had passed audits, 11 per cent reported a failure, while 20 per cent did not know whether or not their systems had passed.

JT Sison, vice-president of marketing and business development for Dataguise, said: "As we have experienced, many companies are throwing everything they have at IT security challenges. The problem is that even multiple point solutions still leave gaps that put these organisations at risk."

This is likely to become an even more widespread problem as companies continue their transition to big data frameworks. The study found that 28 per cent of respondents reported more than a year's experience with these platforms, while a further 38 per cent are in various stages of adoption.

One key step that will be essential of security challenges are to be overcome is to make it clear who has responsibility for this. The research found that 88 per cent of companies stated their IT security team – including the CISO and CIO – would face scrutiny if they encountered a breach, while 47 per cent added that the CEO and board of directors would also shoulder some responsibility.

Dataguise noted that this illustrates how IT teams are at greatest risk if there is a security incident, and so must take the lead in strengthening big data infrastructure to reduce the risk of unauthorised access.

However, it also makes clear how the C-suite also needs to be focusing on this area. Big data analytics initiatives are unlikely to be successful without support from the board level, and this needs to extend to ensuring the security of frameworks.

Big data to help drive mobile app market past $100 billion


Posted By : admin Comments are off
big data helping mobile app market
Categories :#AnalyticsNews

The global market for mobile apps is set to reach the $100 billion mark by 2020 as more developers seek ways of monetising their products – but this will only be possible with the help of effective big data analytics.

Prabhjot Singh, co-founder and president of business intelligence service Pyze, told TechCrunch that even though there are millions of entrepreneurs around the world building apps and offering them to the public via Apple's App Store and Android's Google Play, very few of these are financially successful.

Although the world's leading apps can rake in upwards of a million dollars a day, the vast majority have difficulty generating any revenue, with less than 50 per cent of developers making more than $500 a month.

However, this could be changed through effective use of big data analytics. One major reason for the failure of many apps to monetise at the moment is a lack of data and business intelligence that can help developers improve their decision-making and identify better prospects.

But as the technology for evaluating data and turning it into useful information becomes cheaper and more widely available, this presents a great opportunity for smaller app developers to improve their results.

"Companies that can build big data and analytics pipelines to learn about how their apps are being used and who uses them are in the best position to build a community of 'sticky users' who will continually use their apps," said Mr Singh.

He added these players are better able to link into media outlets and social services such as Facebook and LinkedIn to retarget their users and deliver the most relevant, interesting experience. This is something that could not be achieved by the majority of app developers, who may be made up of just one or two people.

Until now, the problem with mobile analytics has been that the solutions that were available did not scale particularly well to provide developers with the type of answers they needed. The tools struggled to effectively analyse millions of data points and then draw out the key information that could give an insight into user behaviour and desires.

Dickey Singh, co-founder and CEO of Pyze, explained that this problem could be solved by using machine learning tools that can automatically cluster groups of users into segments, making it easier to offer more personalised services.

This can enable developers to better monitor usage habits to identify different patterns of app use. 

"From here, an app can begin to develop personalized messages for given users based upon how they use the app," TechCrunch noted. "Nonprofit and for-profit uses of the app can be identified; this enables app developers to more clearly see who is using their apps in a premium, pay-for mode, and where they should be investing their efforts to further monetise their products."

In a study of 12 companies using such tools conducted by Pyze, customer engagement has increased by an average of 35 per cent, while revenue has gone up by 20 per cent.

Prabhjot Singh noted this can help level playing fields between independent designers and large app companies, while also improving rates of engagement for large enterprises that are not primarily in the app business, but that want to use such tools to improve relationships with customers.

Big data spending ‘to reach $187bn’ by 2019


Posted By : admin Comments are off
business big data spending
Categories :#AnalyticsNews

Global spending in big data analytics technologies, including hardware, software and services, is set to grow by more than 50 per cent between 2015 and 2019, with investments increasing from $122 million to $187 million over the forecast period.

This is according to research from International Data Corporation (IDC), which noted that the key driver of this will be services-related opportunities, with IT services generating more than three times the annual revenues of business services. 

Meanwhile, software will be the second largest category, generating more than $55 billion in revenues in 2019. Hardware spending is expected to grow to nearly $28 billion in 2019.

Jessica Goepfert, Program Director, Customer Insights and Analysis, at IDC, said: "There is little question that big data and analytics can have a considerable impact on just about every industry. Its promise speaks to the pressure to improve margins and performance while simultaneously enhancing responsiveness and delighting customers and prospects.

The most forward-thinking organisations will utilise the technology at every level of their company in order to make better, data-driven decisions, she continued.

The sectors that will show the greatest interest in big data analytics in the coming years include discrete manufacturing, which is expected to be responsible for $22.8 billion worth of investment in 2019, banking ($22.1 billion) and process manufacturing ($16.4 billion).

However, telecommunications, retail, professional services and government services will also each generate revenues of more than $10 billion from big data by 2019, while the fastest-growing sectors include utilities and healthcare, illustrating how the technology can impact every part of the economy.

IDC also noted that it is not just large enterprises that are set to increase their big data investments. Although the largest firms will naturally generate the higher revenues for the industry, with some $140 billion of the total being created by firms with more than 500 employees, small and medium businesses will remain a significant contributor. Nearly a quarter of the worldwide revenues will come from companies with fewer than 500 employees. 

More than half of big data revenue will come from the US, following by Western Europe and Asia-Pacific. However, it will be in Latin America and the Middle East and Africa where the fastest growth is seen, as companies from around the world get on board with the technology.

Dan Vesset, group vice-president of Analytics and Information Management at IDC, said: "Organisations able to take advantage of the new generation of business analytics solutions can leverage digital transformation to adapt to disruptive changes and to create competitive differentiation in their markets." 

He added that these enterprises should not just be using the tools to automate existing processes, they need to treat data as they would any other valuable asset, by using "a focused approach to extracting and developing the value and utility of information". 

Many firms still lacking big data strategy


Posted By : admin Comments are off
big data strategy
Categories :#AnalyticsNews

Big data continues to be high on the agenda for many businesses, but despite the growing recognition of its importance, a large number of firms still do not have an effective plan in place for making the most of the technology.

This is among the key findings of a new survey conducted by DNV GL – Business Assurance and GFK Eurisko, which polled nearly 1,200 professionals from across Europe, Asia and the Americas. It revealed that many expect big data to play a significant role in future operations, but here is still a long way to go for many enterprises before this can be achieved.

More than three-quarters of respondents (76 per cent) predicted that investments in big data technology will be maintained or increased in the coming years, while two-thirds (65 per cent) are planning for an environment where big data is a key part of their operations.

But even though 52 per cent of professionals agreed that big data presents a clear business opportunity, only 23 per cent have a clear strategy in place for embracing the technology.

DNV GL noted that in order to make big data analytics a success, companies should treat it as a new journey, and make preparations and changes to their existing processes accordingly.

For example, 28 per cent of respondents say they have improved their information management procedures in order to make the adoption of advanced analytics tools as smooth as possible, while 25 per cent have implemented new technologies and methods for handling data.

However, fewer companies have worked on changing their day-to-day activities. Just 16 per cent have made efforts to change the culture or organisation to reflect a more data-driven approach, while 15 per cent have changed their business model.

"Big data is changing the game in a number of industries, representing new opportunities and challenges," said Luca Crisciotti, chief executive of DNV GL – Business Assurance. "I believe that companies that recognise and implement strategies and plans to leverage the information in their data pools have increased opportunities to become more efficient and meet their market and stakeholders better."

The survey found that all companies that have already adopted big data analytics report clear benefits from their efforts. For example, 23 per cent stated they have seen increased efficiency, 16 per cent reported better business decision making and 11 per cent witnessed financial savings. 

Meanwhile, 16 per cent stated their customer experience and engagement has improved as a result of big data, while nine per cent reported better relations with other stakeholders.

However, there are several factors that are still holding many firms back when it comes to adopting big data. Chief among these are a failure to develop an overall strategy and a lack of technical skills, both of which were named as issues by 24 per cent of respondents.

Therefore, getting the right personnel on board will be critical in making big data a success. These individuals need to understand the intricacies of big data analytics technologies, as well as take a leading role in preparing the business for the era of data.

Mr Crisciotti said: "The ability to use data to obtain actionable knowledge and insights is inevitable for companies that want to keep growing and profiting. The data analyst or scientist will be crucial in most organisations in the near future."

Telcos turning to Hadoop to help counter fraud


Posted By : admin Comments are off
Telcos, Hadoop, counter fraud, big data
Categories :#AnalyticsNews

Many businesses in the telecommunications sector are turning to Hadoop-based big data analytics solutions in order to tackle fraud, a new survey has found.

Research by Cloudera and Argyle Data noted that fraudulent activities are one of the biggest challenges for the industry, with telcos in the US alone losing around $38 billion a year in revenue to this.

Therefore, any technologies that these enterprises can put in place to help identify suspicious activity and put a stop to it before it becomes a major issue will be hugely valuable, and increasingly, Hadoop is seen as the answer.

Nine out of ten telcos (90 per cent) attending a recent webinar organised by Cloudera and Argyle Data stated they intend to use Hadoop to assist in their fraud prevention strategies.

However, just a third (34 per cent) said they currently have a platform in place for this, which indicates there is still a long way to go for the sector as a whole as they try to identify the best use cases for the technology.

Vijay Raja, solutions marketing manager at Cloudera, noted: "Fraud prevention is a textbook use case for Hadoop-based analytics because the ROI is immediately visible.  Real-time machine learning relies on large amounts of data to detect sophisticated revenue threats."

Platforms that are able to combine real-time analytics, machine learning and graphical visibility tools are essential in countering telecoms fraud. These solutions enable analysts to spot fraud attempts as they happen, which traditional monitoring systems can struggle to achieve.

As today's sophisticated, high volume attacks can cost communication service providers millions of dollars in revenue in a matter of minutes, being able to detect fraud quickly will be essential.

Arshak Navruzyan, vice-president of product management at Argyle Data, said: "Unsupervised machine learning delivers everything telco fraud analysts need to be efficient at and deliver immediate ROI."