The digital doctor will see you now: how big data is saving lives

"According to the data trends, premature babies with more stable heartbeats were more susceptible to infections. These conclusions were most unexpected," big data guru Rick Smolan tells us.

Smolan, co-author of the book The Human Face of Big Data, is a big believer in how the identification of trends within the capacious amounts of data being generated, collected and properly analysed, is our single biggest ally in the battle to extend life.

In the case of 'Project Artemis' at the University of Ontario, smarter use of this so-called 'big data' is already helping to preserve the lives of the planet's most fragile new inhabitants.

You see, from the moment we leave the womb (and before that) we're exuding data with every tiny breath, but up until quite recently, analysis of that heart rate data was rather facile.

Babies would, of course, be connected to heart rate monitors, but they only informed neonatal staff whether the patient was comfortable or in distress. When the babies were sent home, with a presumed clean bill of health, this data was discarded with no further analysis.

Dr Carolyn McGregor decided to look deeper and her initial findings identified a crucial trend; that these babies' hearts were regulating and stabilising at the onset of infection.

Project Artemis

This knowledge gives doctors the opportunity to react before physical signs of the illness manifests themselves… before it's too late.

"About a quarter of babies who are born premature will end up with infections and 10% of those will die – that's quite a significant problem," said McGregor, the lead researcher at the University of Ontario team, backed by IBM.

"When you think about the fact that there is 7,000 heart beats per hour, and more than 2,000 breaths in the hour, then you can start to see that there is a lot of data to a baby."

Project Artemis, capable of processing 1256 readings a second, is now being implemented at neonatal units around the world with hospitals able to hook up to the network over the cloud, sending minute-by-minute readings back to the base in Toronto. It is already making a difference.

However, while big data analysis tools are saving the youngest lives by identifying vital trends, the phenomenon is also assisting those who've been around for significantly longer.

Many folks with elderly relatives face tough decisions when it becomes apparent that they can no longer live alone safely. Big data author Rick Smolan faced this common dilemma when his 90 year-old mother fell and wasn't found for 8 hours. But before too long help may be at hand, thanks to big data.

"I found that General Electric and Intel are working on something called a Magic Carpet," he told us. "It's basically a carpet fitted with sensors that collect data on the movement of your loved ones.

"For the first week it doesn't say 'good' or 'bad' it simply identifies patterns. Then, on a day when that base behaviour changes, for example, if it's 10:30 and the person hasn't stepped foot in the kitchen the system can be set up to send an alert to relatives."

The idea of the fully 'quantified self' is still in its infancy of course. Smolan calls it the 'caveman era.' As great as it sounds, that prototype carpet is likely to cost $30,000 to install. That's if it's ever released.

An Apple a day...

Wearable devices like the Jawbone UP and Nike Fuelband remain our best hope of creating those patterns in the short term.

"The best thing about these devices is that they create the same baseline as the carpet," Slolam added.

"One of my friends, who was one of Steve Jobs' doctors, told me 'years before any of us is in an ambulance on the way to a hospital, our bodies are giving off all kinds of data that something is wrong.' Until now we haven't been measuring it."

By informing users of the quality of their sleep, their heart rates, these wristbands are offering up those data patterns, placing the information in the hands of users. Once Apple enters the fray with its rumoured wristband offering blood sugar and blood pressure sensors that power can become even greater.

However, the real key to extending lives through the use of big data tools may arise through the advent of inexpensive DNA testing and the resulting potential for personalised medicine.

When Steve Jobs was diagnosed with pancreatic cancer in 2003, it cost $100,000 to sequence his DNA. Through those ends, doctors were able to pinpoint the kinds of treatment he would respond best to. Today, that process costs as little as $4,000. However, geneticist Francis Collins predicts that within five years it'll cost just $50 and will be performed at local pharmacies as a routine measure

Why is that so significant, you ask? Well many 'miracle cures,' developed at the cost of hundreds of millions dollars, are never approved by regulators despite proof they would help the overwhelming majority of patients. Analysis of DNA through big data tools can change all that.

"The FDA will never approve a drug that's known to kill people," Sloman says. "Now, with the cost of genetic sequencing plummeting, these cures that exist will come into use because we can know that it'll help me or kill you.

"As a result, we can enter an era of personalised medicine where there is less of this one-size-fits-all approach to administering drugs."

Before it was handcuffed by the aforementioned Federal Drug Agency, another private company was cheaply offering people the opportunity to take control of their own destinies with an affordable genetic testing kit.

Analyse this

By spitting in a test tube and sending it away for analysis Silicon 23andMe patrons were able to discover which illnesses and diseases lay in the cards according to their genetic disposition. Those findings were based on the availability of unfathomably large data sets and the powerful analytical tools used to identify patterns and trends.

23andMe, the company founded by Anne Wojcicki, was shut down for breaching marketing rules.

Although knowing there's a chance you may develop cancer decades from today could constitute a virtual death sentence for some, others were keen for the knowledge so they could make lifestyle and diet changes, in the hope of combating or delaying onset.

Through the company's testing Google founder Sergey Brin discovered he carries the gene LLRK2, a mutation that has been associated with higher rates of Parkinson's disease.

As a result the Google founder rallied a cohort of over 10,000 people (including Muhammad Ali) currently suffering from the disease. The hope is that one day this rich data set can be used to decipher what activates the genes associated with the disease, identify the contribution of environmental factors, and pinpoint how individuals may respond to different treatments.

This data is not being guarded or shrouded in secrecy. The company wants the hive mind of the medical and scientific community on board this big data bandwagon.

"If Sergey is in the database," said Wojcicki in an interview. "I just don't want 23andMe to have access to the data, I want every smart LRRK2 researcher to have access."

So, away from the ever-present fears over government snooping and corporate intrusion big data has brought about, it is also proving to be a force for good in the world.

This relatively new-found desire and ability to capture every data point possible, combined with tools that can process such large masses of information and spit out trends beyond previous levels of comprehension is keeping people healthier and alive for longer – from the youngest to the oldest.

It is creating an appetite and an environment where data-inspired preventative measures can become just as important as the reactive treatments that often come far too late.

Beyond the surveillance society and the need for greed, there is a caring, human face of big data and it's watching your back.

• Big Data deconstructed: how it is keeping us fit and entertained