ERP News

Why Is Big Data Analysis So Challenging?

945 0

There’s gold to find in the big data forest, but most companies have no map and no crew.

A new research report from TDWI, titled Data Science and Big Data Enterprise Paths to Success, outlines the state of big data and data science: In short, it’s getting bigger and more difficult. On a scale from 1 to 5, with 5 meaning “completely satisfied” with the current data management strategy, only 3 percent of respondents gave a 5 answer.

Roughly 43 percent were right in the middle, and nearly 40 percent offered a 1 or 2.

Part of that dissatisfaction might be because of the sheer amount of data being collected. Twenty percent of the survey respondents are trying to work with 10-100 terabytes, and 17 percent have anywhere from 100 terabytes to more than a petabyte. Most of this data is structured data right now, but companies understand the need to quickly figure out plans for integrating that reliable data with the more unpredictable new inputs. And Hadoop is the big data platform of choice, generally—30 percent of all respondents use Hadoop on-premises today, but for those managing more than 10TB of data, that jumps to 50 percent.

Among the types of data being managed, some are growing far more rapidly than others. Text/content data from emails, call center notes, and claims is growing extremely fast, as is external social media text data.

For Full Story, Please click here.

Leave A Reply

Your email address will not be published.