Technology_Volume, velocity, variety

Gen-i’s product marketing manager, networked ICT products David Reiss says three characteristics separate big data from everyday data:
Volume: Many New Zealand companies collect large amounts of data that are difficult to store and move using everyday tools. He says it isn’t just data owned or generated by the company. The company can buy databases from external agencies and sift through social media records and similar widely available online material.
Velocity: The rate at which data is generated and captured is important. Companies need timely information. Real time or near-real time processing means marketing campaign can be changed if, for example, there’s negative response to an early advertisement. It’s also important to have up-to-date competitive information.
Variety: Big data typically pulls data from structured and unstructured sources; there can be tweets, blog posts, online comments and video as well as conventional relational databases. Increasingly, data is collected from connected devices such as smartphones, smart electricity meters or embedded sensors.
All of these definitions suggest big data is less about the technologies and tools used to process large amounts of data and more about the approach to analysing the data to understand behaviours and to make better forecasts. Reiss says: “We get lot of questions from customers about what big data is, but most of the conversations we have are more about what our clients are trying to achieve.”
The three Vs are widely understood in big data circles. Reiss likes to add fourth V: veracity. He says there’s an issue of trusting the collected data: “One in three business leaders don’t trust the information they are given.” Traditional databases work on the assumption that all the data is clean and precise. When you throw social media feeds and other unstructured data into the mix there’s question mark over some of the data. That’s where the skill of the analysts – interpreting and applying weighting to the data comes in. And with the 2011 McKinsey estimate of 1.5 million shortfall in skilled analysts in the US alone, those with the right skillsets will be in high demand.