What’s the Big Deal about Big Data?

by Robin Harris

I love these “new age” names we come up with, “Big Data” (as opposed to small data?), or “Unified Data Warehouse.” Isn’t technology terminology great?big_data

Dealing with big data is nothing new for the construction industry…it has been around since the dawn of the digital age. What has become the constraint is that it is “too big to understand” – in its native format. It has become so enormous in volume our normal analytical tools fail to deliver data in a manner in which critical decisions can be made (or delivered in a timely manner).  Another constraint in today’s digital age is we need to combine data from many different sources to fully understand what is going on and what to do about it.

We often work with very large enterprise construction firms that have numerous institutionalized data systems over the years that are islands of mission critical data – and because of cost invested, user resistance to change, or excessive disruption to migrate, no one is giving them up!  Industrial Mining, Oil & Gas, large commercial multi-location organizations all struggle with multiple data systems and huge volumes of “big data.”

So how does a company bring all that data together without killing the company?

The concept of “Big Data” drives the need to have an analytical engine that can deliver information to each role in the organization in a meaningful way and in a timely manner. The other mission critical function for companies with locked-in data systems is to bring these enormous “islands” of information into a “Unified Data Warehouse” where it can be combined to deliver even more meaningful and valuable information from different sources.

Several software firms are attempting various approaches to the Unified Warehouse concept and the data mining capability.

Hard Dollar’s LASSO Project Intelligence is an example of a construction industry Unified Data Warehouse where data is imported or connectors are built that integrate data from various sources such as the ERP system’s job cost, Scheduling Software data, Estimating data, Production data, and other sources. The power of LASSO’s tools is that it easily lets an end user build, analyze, and forecast various scenarios – without a custom developer writing code. It might combine a aseline budget, the schedule with its resources and equipment, the costing to date, the production and other data all derived from existing data sources within the company (and sometimes from their vendors and clients as well). This new
combined source of data is then presented through easily defined graphic formats and data grids.

Welcome to the “Big Data new age!”