Why NoSQL is shaking up the database market

The ever multiplying number of ways we consumer data is having a profound affect on the database market.

As such, companies are scrambling to keep up with the changes taking place and adjust their data storage operations to fit the needs of their business and their customers.

Adrian Carr, VP of database specialist MarkLogic, spoke to us about how new technologies like Not Only SQL, or NoSQL, are now becoming essential due to the flexibility they offer when looking after data.

TechRadar Pro: Can you explain the shift that's taking place in the database segment right now?

Adrian Carr: Not so long ago we all subscribed to the theory that you could store anything in a relational database. But now we realise that, although it is technically possible, it's nothing like ideal.

Although you can now store rich content such as documents and social media in relational databases, it's not easy to then do anything with the data. Even basic functions such as search are problematic. Relational databases simply don't perform well unless they are given beautifully structured data and are expensive to boot.

As 80 percent of the world's data is unstructured and more suited to NoSQL (Not Only SQL) databases, these technologies have matured, resulting in the surge of interest we are witnessing in this space.

TRP: How do you predict the database market will evolve?

AC: Rather than widgets and one-project wonders, I believe that platforms control the destiny of most computing. In the NoSQL database world, this means an integrated platform that incorporates not just the database but also the search engine and application services.

With 50 or so NoSQL players jostling for position, we are seeing the first stages of consolidation – such as IBM buying Cloudant - and segmentation.

I predict that all the relational database incumbents will have to make a move into this space as it would take too long for them to build whole new engines themselves. They have been adding extensions to their product but it's like fitting a round peg in a square hole and it certainly doesn't scale.

Part of the challenge for the incumbents is a fear of undermining the huge revenues they receive from their relational database business by launching lower cost NoSQL offerings.

TRP: What impact is big data having on how we architect the datacentre?

AC: Big data has helped to change people's mindsets and appreciate the value of the terabytes of data being created and stored.

However it requires a different approach to building databases and applications. Up to now developers have built out a database to power data centre applications. But for every application you have to load the data from wherever it lives into the application-specific database.

With multiple applications, pretty soon, you have hundreds of data stores with data duplicated all over the place. One day you wake up screaming when you realise the problem is worse than duplication. You have also lost context, security and data provenance.

TRP: Your company discusses the 'data-centred' data centre - what is it and how is it different than a traditional data centre architecture?

AC: The data-centered approach is to focus on the data, its use, and its governance through its lifecycle as the primary consideration. By bringing the applications to the data, instead of taking the data to the applications, you can minimise data duplication resulting in consistent data integrity, more flexible application development, portability to users' devices, and scalability.

Some of these databases can get pretty large and having them in one place makes life simpler when it comes to managing enterprise fundamentals such as security, disaster recovery, tiered storage and archiving.