Forget big data – it's time for big algorithms to change the world


Most examples of algorithms in our daily lives rely on machines interpreting data from humans. But there's no reason why this can't occur where both parties are machines.

"This already happens in financial markets, where programs trade with one another," says Owen, who thinks that machine-to-machine communications are the future. "You can imagine a day when your boiler negotiates a time and price with the gas and electricity provider systems, knowing your likely morning shower time, to most cheaply ensure your morning shower is hot enough," he says.

Sean Owen, Director of Data Science at Cloudera

Sean Owen, Director of Data Science at Cloudera

What is 'deep learning'?

In the world of algorithms there's a lot of hype around deep learning neural networks and machine learning systems, with the goal being artificially intelligent machines that understand as well as calculate.

Cue machines that can play chess like a human, using prediction rather than calculation. "Deep learning – tools built to model nature – are more intelligent and precise," says Turner. "These are the robots and CNC (Computer Numerical Control) machines to build something useful."

However, there's nothing new about deep learning. "It's a decades-old family of ideas," says Owen, who calls deep learning a 'celebrity algorithm'.

"It learns its own intermediate representations of its data, which sounds like thinking," he adds. "With some clever optimisation in the last five plus years, it's been possible to pair it with such brute force computing that it has made amazing progress in particular areas like image recognition."

This isn't about new kinds of algorithms, but more about massively scaling them up – and that's what is about to happen to the economy.

Adrian Carr, Group Vice President for Global Commercial Sales at MarkLogic

Adrian Carr, Group Vice President for Global Commercial Sales at MarkLogic

Is this just semantics?

Absolutely – the tech trends we've mentioned are all part of one, overriding mega-trend; technological progress that gives us a powerful insight. Should we even talk about the 'age of the algorithms' rather than big data?

"This is not an either/or … the age of big data has only just begun, and the Internet of Things will be the second phase of this age," says Adrian Carr, Group Vice President for Global Commercial Sales at enterprise NoSQL platform for big data MarkLogic, who thinks both are rather remote for most businesses.

"Algorithms will provide much needed insight as to where to focus human attention … systematic algorithmic adoption is required to automate decisions, summarise information and focus attention on subjective issues," he adds.

In terms of us humans getting more insight, the development trends in both analytics and visualisation are just as important as the scaling up of algorithms.

Better, faster

Algorithms aren't about anything other than insight and knowledge – and both are critical for maintaining competitive advantage in the coming algorithmic economy. The potential for automation and growth is massive. In fact, it's scary big.

"We need to be mindful of rapidly increasing inequalities – not in financial wealth, for once, but knowledge wealth and capability," says Turner, who thinks we can forget about a 5% increase in efficiency. "These tools can make us ten times better, faster, and more agile."

At its core, that's what the algorithmic economy is all about; speed. "Data offers freedom and opportunities, but it can also clog the system – and this is where moving to a true algorithmic-based economy can help," says Blake. "It keeps the world moving."