As a preamble let’s start with the famous quote from Benjamin Disraeli, prime minister of the UK from 1874 to 1880. “There are lies, damned lies, and statistics” and that is what many people and businesses still think. But if we fast forward to 2017 we see the opposite – enterprise and government see big data and analytics as the single most trusted source of truth and a vital element of almost everything we do.
In fact, the term big data, which first saw its use in the 90s, has seen data sets grow from megabytes (MB) to petabytes (PB)
- 1024 Megabytes = 1 Gigabyte (typically DRAM DIMMs max out at 32GB and SSD at 500GB)
- 1024 Gigabytes = 1 Terabyte (typically hard disks max out at 8TB)
- 1024 Terabytes = 1 Petabyte (currently the largest single data sets)
- 1024 Petabytes = 1 Exabyte
- 1024 Exabytes = 1 Zettabyte
- 1024 Zettabytes = 1 Yottabyte
- 1024 Yottabytes = 1 Brontobyte (and after that Geopbyte).
Typical data growth rates of 20% to 50% annually are attributed to newer data sources, such as web traffic, IoT sensor data, location and telephone call data. Exabyte sized data warehouses will soon be the norm.
And it is no longer about data from one source – it is about the vast data lakes or warehouses created when you mix in your data with other relevant data from social media, finance, government, utilities, Telcos, marketing clouds, even free public Wi-Fi, street cameras and mobile/computer logins.
As Gartner, a research and advisory company puts it, “Big Data represents information assets characterized by such a High Volume, Velocity, and Variety to require specific technology and analytical methods for its transformation into Value.” And a fifth “V” has been added for Veracity – it must be true to be of Value.
For Full Story, Please click here.