2014 Predictions vs 2016 Realities: RDBMS, Big Data, Hadoop

How do predictions from our 2014 Data Connectivity survey line up with our 2016 survey realities? Let’s explore these results with data visualization.

In 2014, Rosetta landed in a comet, we all got soaked with the ALS Ice Bucket Challenge and we were all blown away when Oculus Rift took us into the brave new world of virtual reality. Along with these crazy, inspiring events, we reached out to you to help gain a collective understanding of the data connectivity ecosystem, and help de-risk vendor selection amidst future market consolidation.

Results Validated by Third Party Research

Take a look at the top 10 predictions you made and how they align with the realities of 2016. These predictions are based on our 2014 survey, while the “realities” are from our 2016 Data Connectivity Survey.

We thought it would be interesting to compare our results with DB-Engines.com*, a vendor-neutral entity dedicated to collecting and presenting unbiased information on database management systems.

*In the majority of cases their results support ours. It is important to note that DB-Engines.com research is based on demographics that span the entire data management ecosystem while our survey has a focus on data connectivity.

Your Predictions

1. The proliferation of database options leads to inevitable market consolidation

   Reality: This was a broad prediction throughout the 2014 survey, and was a logical conclusion at the time. The reality in 2016 is that some segments of our survey see consolidation while others do not. Overall we do not see consolidation, but rather the current market leaders (especially in the relational space) staying on top and more specialized distributions being created for specific situations and projects.

   The Hadoop market has consolidated a great deal in the last two years with the recent dropout of Pivotal Software. The SaaS ecosystem has grown rather than consolidated, but the 2014 market leaders are still here in 2016 (Salesforce, Microsoft Dynamics CRM and Oracle Service Cloud).

2. SQL Server, Oracle, DB2, and MySQL will decline slightly but remain on top

   Reality: Despite predictions that these databases will steadily decline as companies move into the cloud and start using non-relational technology, they have remained on top with minimal change.

   The diagram below represents the amount of survey respondents (line thickness) that used SQL Server, MySQL, Oracle or DB2 in 2014 and 2016. These systems are still relevant and any change in the last two years is negligible.

3. SAP HANA will grow significantly over the next two years

   Reality: SAP HANA is one of our more compelling stories. Our 2016 survey shows nearly 10% growth and is forecasted to continue to grow in market adoption. Other industry experts validate these findings. For example, the following chart from DB-Engines.com shows an even more dramatic increase in adoption of SAP HANA. 

   

4. Non-relational databases will not impact the growth of established or emerging vendors

   Reality: Established vendors still control the majority of the market. Non-relational offerings continue to innovate and create new technology, but enterprises and application developers are comfortable with SQL-based, relational technology and are not yet ready to leave it behind.

   NoSQL is popular, but the data is hard to integrate with traditional databases without a robust, high performance connectivity solution. MongoDB and Cassandra have climbed in the rankings, with MongoDB overtaking DB2 and Cassandra right behind.

5. Teradata will grow tremendously

   Reality: This prediction is mixed because although Teradata has grown since 2014, it was not at the rate expected. For now, adoption seems to have plateaued and will likely continue to have significant challenges to overcome in 2016. 

   Teradata took a hard hit from Brexit. The U.K. was a large growth market for Teradata’s data and analytics products and the prospect of slow orders from a questionable economic state has investors worried. Amazon Redshift has also taken a significant amount of Teradata’s market share.

   

6. Amazon Redshift will reach new heights

   Reality: Amazon Redshift’s data warehouse is the fastest-growing service in Amazon Web Service’s history. It has grown and will continue to grow with the lower cost of cloud adoption, and shows no sign of slowing down. Our 2016 survey reflects similar, if less dramatic, growth.

   
7. Competition in the Apache Hadoop ecosystem will not significantly change the market

   Reality: Hadoop distributions are getting more technical and more specific, but we do see market consolidation. With Pivotal’s exit, the Hadoop market has moved closer to where it began. It has three independent vendors as primary suppliers of the open-source distributed processing framework. Our 2016 survey shows a similar trend.

8. NoSQL and NewSQL will challenge market leaders

   Reality: NoSQL and NewSQL definitely challenged relational databases, but companies are simply not ready to leave their SQL-based infrastructure. NoSQL and NewSQL were both designed for modern architectures. As such, they are generally being used for new applications. That being said, many applications are still built on SQL and relational databases remain on top. MongoDB has overtaken DB2 in the DB Engine rankings but still has not overcome SQL Server, Oracle or MySQL.

   

   Regarding enterprise migration from SQL and Oracle to NoSQL and NewSQL, Matt Aslett of 451 Research said the following: “It’s not so much about migrating existing workflows from existing Oracle and Microsoft databases as much as it is the development of new applications which, over time, will come online and the older applications may be retired.

   “What the NewSQL vendors are not doing is saying, ‘Port your CRM and ERP databases over to our database,’ because nobody is going to do it." Matt Aslett (Research Director, 451 Research)

9. Within 3 years, ISVs will be building pure cloud applications

   Reality: The technology is there for pure cloud applications, but companies are hesitant to move 100% into the cloud. Solarwinds’ IT Trends Report 2016 explored the barriers to all-in cloud adoption as well as the reality that the majority of organizations use a hybrid IT environment and will continue to do so. 

   In their survey they state, “92% of IT professionals say adopting cloud technologies is important to their organizations’ long-term business success,” but, “60% say it’s unlikely that all of their organizations’ infrastructure will ever be migrated to the cloud.”

   Hybrid connectivity is now a must. This solution provides instant connectivity across the firewall and enables a unified view across multiple data sources, connecting your data source directly to your application.

10. The SaaS ecosystem will consolidate

    Reality: “The general consensus is that at some point the SaaS ecosystem will begin to consolidate. And yet, we have seen very little evidence that this is happening" (Villi Iltchev, Why SaaS Consolidation Is Not Happening). In our survey, we see many of the same names and a similar distribution of our customers across the SaaS ecosystem. 

    You can see our survey data below from 2014 on the left and 2016 on the right. As you can see, rather than consolidation, we see customers experimenting and using a vaster array of SaaS data sources.

Overcoming Database Challenges Together

If you have any further insights or questions, please share them in the comments section. We want to empower you to succeed in all your data connectivity ventures. If you want more detailed information regarding our 2016 survey, download your free copy here. If you would like more information on any of the topics we discussed, click the related box below!

Big Data Frameworks	Non-Relational Database Connectivity
Relational Database Connectivity	SaaS Data Connectivity