For years, there have been a few technology pundits declaring the Data Warehouse to be a dying technology – to be killed off at any moment by “Big Data” and impressive new business analytics technology.
The truth is that the concept of the data warehouse is alive and well, and will be for as long as companies want to use their own operational data to make business decisions.
Long live the Data Warehouse!
Big Data may be the future, but there’s little evidence that the demand for the Data Warehouse solutions will be slowing down anytime soon.
As with all technology, the tools, technology, architecture and processes used to create a Data Warehouse have changed over the years. If you don’t keep up-to-date, YOUR Data Warehouse may become obsolete, and you will have to find a time to upgrade or replace it. The concept itself is here to stay.
The Influence of Big Data on the Data Warehouse
A Data Warehouse is designed to provide “one source of the truth” for all of your company’s operational information. By pulling data from multiple databases, spreadsheets, and business systems, companies can then analyze and track key performance indicators that factor in all aspects of the business – accounting, production, sales, HR, etc.
One of the first steps in creating a data warehouse is to create a data structure. Historical information is then extracted, transformed and loaded (ETL) into the centralized data store following the relational database structure. Aging technology and poorly designed extraction processes can create bottlenecks that limit the value of the Data Warehouse, and may be a sign you need to modernize your Business Analytics systems. Innovations from Big Data are influencing improvements to this process.
Big Data pulls in huge volumes of outside information, from customers, social media, bank records, and other data sources. Big Data systems in the Hadoop ecosystem are designed to span multiple machines, and don’t require the data structuring of a data warehouse. The concept of the Data Lake was born from Big Data. This Data Lake of unstructured data can be combined with structured data to provide additional insight. But without the structured data, how can you run a business?
Companies Need Structured Data
Without the Data Warehouse, there is no officially sanctioned, vetted, governed, agreed to, “one version of the truth.” What we see in our Business Analytics practice is that Big Data trends are reinforcing why companies need a Data Warehouse, and are bringing in technology that makes them faster, easier and cheaper to build.
Author: Mark Hatting, Managing Director – Business AnalyticsOther articles you might be interested in: