From discovery to action: big data in the boardroom

As with the adoption of Hadoop in enabling more flexible data storage and analysis, discovery must take place in the context of where there is a clear understanding of how analytics can deliver against the organisation's broader objectives.

Put another way, discovering nuggets of information will only be of real value if they are incorporated as part of new business initiatives. The granularity of the available data enables this to operate both at a strategic (longer-term) and, increasingly, a tactical customer or transactional level.

So how can the business take this process of discovery and turn it into action – in the boardroom and on the shopfloor – as the basis of a more competitive business?

First, the big data roadmap needs board-level support from the outset, from executives who understand the value of data as the driving force behind business improvement. This not only requires that they have a basic familiarity with the concept of analytics, but also they need to recognise the value of experimentation. Many of the nuggets found at the discovery phase need to be tested in a real world environment to assess their potential value, making senior buy-in critical.

Businesses with a web presence tend to be better placed here, as experimentation in the web sphere is far easier. As a simple example: if an online retailer wants to trial an alternative website in a new colour, this can be made available to a controlled subset of customers and the outcome on buying behaviour and profitability assessed, with no impact on the retailer's core customer base. Although this is harder to achieve in a physical environment, in many cases performance improvement will be well worth the time and effort invested.

Again, the ability to experiment is key. Retailers are especially good here, through their willingness to allow individual store or area managers to try out different initiatives. However, unlike much of the distributed power of decades past, the head office usually has full visibility of all activities. Equally important is the impact can also be tracked and monitored at individual shopper as well as at store level.

There are however a number of other hurdles to be overcome: First, it is important for senior management to understand and be sensitive to the practicalities in implementing discovery at ground level.

For example, it may be that all tests have shown that adoption of key phrases as part of a call centre process will deliver more effective results. However, if operatives are uncomfortable with the proposed change and are not persuaded by the scale of benefit, anticipated results will not be realised.

Second, the resulting outcomes and impact must be measurable in order to determine the effectiveness of any initiative, which can then be used to build confidence and secure buy-in for broader rollout across the business.

Finally, in the spirit of fail fast, the business must be brave enough to admit when things aren't working – both in existing parts of the operation and in new initiatives – and take immediate corrective action. In an iterative environment with rapid measurement of results, such decisions can be taken swiftly and repeatedly in order to optimise outcomes.

Visible best practice

A business that has successfully scaled the heights of a seamless strategy – from initial experiment through analysis to final business execution – can be distinguished from a business still in the foothills of Hadoop implementation for example, in a number of ways.

Although much of the effort to deliver this integrated approach takes place behind the scenes, a data-driven organisation is more likely to change visibly over time, as it responds rapidly and effectively to what competitors and customers are doing. The organisation also tends to be more willing to embrace disruptive concepts and introduce radical change within the business in order to differentiate itself and gain a competitive edge.

A good example of an iterative discovery approach is Amazon - to frequent customers, Amazon's site has not changed over time, but if we compare snapshots of the website from five years ago to today, there would be a notable difference.

This perception is because Amazon has taken advantage of incremental discoveries and subtly enhanced their website – delivering service innovation and improvement without dramatically disrupting their customer's experience or routine.

The site has evolved with a continual drive to making the customer shopping experience as intuitive and easy as possible. Changes will be subtle so returning customers know what to do and where to go on the site, but benefit from constant enhancements.

So why is a data-driven approach a key component of business success? It makes for better value-driven decisions by offering facts on which to base comparisons of relative options and accurately measure their effectiveness. Greater granularity enables greater competitive edge, through the ability to measure the value of new offerings.

A Unified Data Architecture ensures the most efficient use of people, processes and technologies. It also creates an agile, future-proofed infrastructure, which can adapt and incorporate new solutions, enabling the business to evolve in time with market demands.

Many of today's businesses have recognised that a good data strategy is no longer about measuring the past but instead enables management to both predict and influence future outcomes. In short, this transparency and control moves the business decisively from being at the victim of circumstance to master of its own destiny.

• Duncan Ross is the director of data science at Teradata UK.