Data Science and Big Data Challenges

71 percent of data scientists believe their jobs have grown more difficult

Interesting study and observations on Data Scientists suffering from increasing difficulty with data, systems and parallelism for analytics. But this is partly brought on by trying to self-build from a diverse set of components that do not always fit particularly well or at least need more than a good nudge to click together. Analytics has to move to parallelism to cope with increasing volume and variety and keep pace with velocity of business needs – but the parallelism in Hadoop and Spark etc. is not refined or complete enough to easily support complex analytics.

Paradigm4 also found that 35 percent of data scientists who tried Hadoop or Spark have stopped using it.

Kognitio has experience in helping businesses get over these hurdles with world leading MPP analytical platform technology that intergrates directly with Hadoop and other big data stores, and with Analytical Services that help business teams solve requirements without a lot of technical fuss – Analytics for all at scale today.

Paradigm4 Study Commentary by Thor Olavsrud