University to invest in Big Data Institute

The University of Virginia is aiming to tackle the growing amount of information it has to deal with by investing in a new centre to handle its growing big data analytics needs.

Set to launch in the coming weeks, the Big Data Institute will operate virtually and gather details from faculties, facilities and resources from schools across the organization. Associate vice-president for research Rick Howitz told university publication the Cavalier Daily that this will enable it to keep the establishment competitive with other research-based institutions.

He said this is a solution that all universities are likely to have to adopt in the coming years as they struggle to cope with growing information volumes. Mr Howitz said: "All curricula in the long run are going to have to adapt to big data. It's just like once the internet got started, everyone eventually had to adapt – the question was whether you're late."

The University of Virginia has eleven schools based in Charlottesville, in addition to the College at Wise in the south-west of the state, and it is currently home to more than 21,000 students.

Chair of the Computer Science Department at the university Kevin Skadron explained that big data is actually a catch-all term used to describe as many as four distinct phenomena that are affecting organizations.

Simply having too much data to handle under older systems is one of these, but he added: "Another is that there are many different data sources, where the volume might not actually be that big, but because there are so many sources there is a challenging integration problem."

Researchers are facing new problems because information is coming in at much faster rates and they are also dealing with highly complex models. Not having adequate solutions therefore slows down the process of turning this into insight and may mean users do not get the most relevant or accurate outcomes.

One of the best examples of how good predictive analytics can use this data, Mr Skadron said, was how analyst Nate Silver of the New York Times was able to pull together a large amount of polling data from various sources to forecast the 2012 presidential election.

"He wasn't just pulling together lots of surveys asking the same questions, he was actually pulling together lots of surveys asking different questions and used historical data, etc." This meant Mr Silver was able to correctly predict the outcome for all 50 states on the night of the election.

The University of Virginia's Big Data Institute will seek to accomplish four keys goal in this field. These are facilitating research, sponsoring education, providing support and consulting on ethical issues.

Its Center for Data Ethics will be one of the central components of this, with Mr Skadron saying this is an area where the university will have a lot to contribute to the wider national conversation.

He explained many firms will not know where to turn for help if they have doubts about the ethics of the data they are collecting at the current time, as the sector is still maturing.  "You can't go to a lawyer, because a lawyer doesn't know the answer yet because there isn't a law yet," Mr Skadron said.

The importance of boosting understanding of big data analytics has also been recognized by Kognitio, which has formed its own Analytics Center of Excellence to explore some of the biggest challenges currently facing the sector. Key issues it has been working on include data visualization, peak forecast analysis and the need to upgrade traditional analytical models for the new era of information.