Skip to main content

Making the most of data: the role of discovery

big data
Letting certain data projects fail can be a useful learning process
Audio player loading…

Unlike a traditional project-based approach to business or IT development, which is increasingly perceived as time-consuming and inflexible, a discovery approach to experimentation involves carrying out multiple smaller projects on a test and learn basis. In this way, it's possible to build upon the results as part of a rapid, highly iterative process.

"Fail fast" recognises that the business needs to solve complex problems and addresses this uniquely by viewing problems at a high level from the start, identifying incremental benefits towards achieving a solution. This approach is only made possible with the collaborative efforts of a range of skills and people across the business and because experiments are facilitated in an environment that goes beyond traditional analytics.

Value everywhere

This approach enables businesses to try multiple options and swiftly discard those that don't add value, before moving straight onto alternatives that may do. It is rare that a discovery-based analytical project turns out to be a waste of time, as valuable nuggets of intelligence will almost certainly emerge as the team trials various iterations, and may even solve issues not anticipated at the outset.

Instead of focusing on a single pre-defined objective, this approach to discovery - carrying out multiple smaller projects and building on the results - is more productive in realising business value. Small-scale testing also allows the organisation to check that initiatives deliver the positive business value anticipated promptly and with minimal impact on the rest of the business.

For many fast-moving industries, the potential to adapt quickly to data insights - in some cases moving from basing strategies on a thought-out hypothesis to factual evidence - can dramatically reduce the time and cost of testing ideas and theories in the field. Often, the incremental improvements push large organisations beyond evolutionary product or process improvement, to achieve their goal of a transformational or disruptive change.

One of the reasons why larger organisations can fall short in the area of discovery-led data experimentation is that it can be difficult to prove return-on-investment within the timescales typically set by the business. By its very nature, as innovation involves failure along the way, this requires a more entrepreneurial, 'small company' approach.

A question of culture

As with most major projects, senior level commitment and buy-in will be needed from the outset to ensure an appropriate culture of experimentation. For example, releasing project staff from target-led requirements of the mainstream business will require strong management to champion the potential benefits of this alternative approach.

It will also avoid unofficial initiatives being taken underground within the business, potentially undermining data security and alienating other staff. In some leading organisations, the driving force behind this project will be senior management, whereas in others there may only be one or two stakeholders who see the potential of big data and are prepared to lead innovative change.

The moment it becomes evident a project is not progressing, it is important to have the freedom to recognise this, quickly abandon it and move on. Neither the business nor individual staff should be frightened of failure but should embrace it, as there is much to be learned around what went wrong that can be used to malebetter attempt next time. It is also worth considering that it is much cheaper to fail fast than to fail slowly.

As such, it is essential to document each step of the initiative and design the project in a way that means results are measurable. And if you can't measure it, you shouldn't do it. Here, establishing an initial benchmark will help ensure whether the changes implemented represent success or failure. Recording the process can bring value even when something has been unsuccessful, as it will help others to avoid making the same mistakes and allow them to build on existing analysis.