“Big Data” is about volume, but it’s more than that... Big data is also about Variety and a couple other characteristics that will be described in follow up blogs…
CPG companies are no stranger to variety. In addition to their own internal variety
of data residing in databases such as Access, Excel, Oracle, main frames, Teradata, DB2, Netezza so forth. You have multiple
applications such as trade promotion management applications, supply chain, manufacturing, planograms, CRM applications, forecasting and a slew of others.
In addition, the variety of data coming in from point of sale (POS) sources include retailer files that include EDI 852 files, EDI 867 files, AS2, flat files, other EDI files, retailer portal downloads and syndicated data from AC Nielsen, IRI, NPD and others. Most companies are also buying competitive market data, demographic information, surveys, weather trends, currency conversion information, and might even be trying to integrate emerging market data. In addition, you might have space information, displays and diagrams that are unstructured or semi-structured.
Those are all examples of various data sources that have existed over the years. Some of these sources are newer than others. But the newest variety of data is coming in via the web. These sources are coming from various applications that track your “Social Reputation,” clicks, and media presence to name a few.
Marketing teams also have ads, including print, on-line ads, tv commercials, radio spots. They might also have online targeted marketing on social media that include offers on web sites, mobile offers, YouTube videos, etc. All these sources are in different data formats containing different information. All of this adds up to a lot of variety.
Big data just got bigger with more variety from the internet. In these last two blogs we discussed volume and variety, but it's also about velocity and one other key characteristic that will be discussed in the next two blogs. Watch for our next blog, Big Data Part 5 on velocity.