How to construct a big data strategy

Another example where real-time operational intelligence is especially important is in fraud detection. With more types of data - whether generated through online behavior, social interactions, or transactions - you can start to identify patterns that would have remained obscure before.

When such data is collected in real time, companies can use predictive analytics to flag fraudulent events with greater certainty and avoid false positives.

Yet another example would be predictive maintenance. Cars now have more software embedded in them than in the past. Through sensor devices, manufacturers can collect information and predict the mean time to failure, as well as more easily inform customers when they should bring a car in for a service visit.

Similarly, for aircrafts, companies typically prefer to perform on-wing repairs, which are less costly than sending the aircrafts to the service facility. By collecting data that better indicates when minor service is needed, companies can preempt major repairs and reduce maintenance costs.

Managed data lake

The more data you have, the better you can develop a 360-degree view and operate in real time. But this can also be a double-edged sword. Data is cumulative and huge volumes are created when new data types are added.

Older companies in particular have large quantities of data on legacy systems, as well as mobile and social data that can potentially be used to extract business value. In many cases, you aren't sure what you want to do with the data just yet, but you know there's potential - and you don't want to lose that potential by throwing the data away.

Instead, you want to store it cost-effectively, so you can access it to discover new insights and trends.

This massive repository is called a data lake -- and must be properly managed or you end up with a swamp. Managed data lakes enable you to store all types of data at scale over the years for processing and analysis at petabyte scale.

But even that is not enough. The data must also be easy to search, cleanse, and govern while observing whatever privacy policies may be in place. In addition, you must ensure the data is highly reliable and available. You need to make it easy for information consumers to prepare and analyse it and make it useful.

The final step is to operationalise the insights that you discover in the data lake to create new products and services, improve customer service, and sharpen decision-making.

The business benefits of well-managed big data go beyond these four use cases, of course, but they provide good illustrations of how you can make big data work for your business. By becoming more data-driven, you can shorten the path to achieving your business goals.

  • John has worked in IT software for over 2 decades, with a deep understanding of Big Data, Hadoop, Agile Data Integration, Enterprise Information Management, and Supply Chain Management (SCM).