Before the terms, big data, unstructured data, omnichannel, Iot, Internet of Things & SOMOLO were coined, we had mainframes, ERP systems and data warehouses (and by the way, we still do). You could make the claim that big data started in the 1950’s with IBM’s “Big Iron” and “Big Data Processing” to handle mixed work-loads.
Or you could say big data started when Oracle coined the acronym, VLDB back in the 90’s to describe "very large databases." I can't decide which is bigger, “very large” or “big.”
Or was it Teradata, who back in 1992, built the first system over 1 terabyte for Walmart? This was the biggest implementation for its time. Teradata called their platform "massively parallel processing" or "MPP." I don’t know about you, but I definitely think “massive” sounds a lot bigger than “big.”
Big, large, massive…regardless of the adjective used, database companies have been in the “big data” business for years. But “big data” today is NOT the same as “big data” from ten years ago. Today, there has been a full blow explosion of data.
Big Data encompasses many areas, starting with internal data in both transactional and analytical systems as well as unstructured data in hadoop databases and in memory databases like SAP Hana.
In transactional systems, the data is constantly changing and being updated. In analytical
systems, the data warehouse is typically updated once a day or sometimes more frequently. In analytical systems such as the enterprise data warehouse, companies are analyzing data, attempting to learn more about their customer, their buying patterns, their behaviors and how to best market to them.
Simply put, analytical systems are designed to help “manage” your business.
Transactional systems are designed to “run” your business.
The big data explosion started with data from applications designed to run your business. Mainframes were first on the scene. ERP (Enterprise Resource Planning) applications, really took off from the 80's & 90’s. Companies like SAP, Oracle, Microsoft, Sas, Infor, JDA & JDE all offer ERP solutions. They include applications for manufacturing, logistics, invoicing, order placement, call centers, etc. These applications also have reports associated with each of
In the early 90’s companies started getting serious about using the data to improve knowledge, business processes and profits. All the buzz words back them evolved from Decision Support Systems (DSS) to Executive Information Systems (EIS) to data warehousing and business intelligence. Unfortunately, I’m old enough to remember these things.
Over the past ten years, we started seeing cooperation and data sharing between partners. I once felt like a missionary in the consumer goods space back in the 90’s trying to explain the value of sharing point of sale data. Retailers thought I was crazy and manufacturers said it will never happen. But today, they finally get it. Today they understand the value. Some more than others, but it’s finally caught on.
As more and more outside partners and vendors began offering new data and insights, we were
able to leverage that data through an architecture that allows new data sources to be integrated within a company’s existing data warehouse. That’s why Relational Solutions always stress the importance of a solid infrastructure.
Some of these outside data sources include point of sale data, EDI files, syndicated data from companies like IRI, Nielsen and NPD, panel data, demographic data, currency conversions, weather trends and other sources.
New data sources are being shared every including loyalty data and emerging market data. Wholesalers, distributors, brokers and other selling partners are also starting to share data (not just reports).
Big Data is about more than just the size of the database. It's about volume, variety, velocity and veracity. It's about structured and unstructured data working together to bring more value to your business. Combined, it is all big data, but in the pure sense of how software companies refer to Big Data today, they are mainly talking about unstructured data on the web. Making it work together is what we help you do.