Creating an Advanced Data Refinery with Hadoop and DataDirect

Manny Vegara shows you how to get the most out of your HDFS deployment with Progress DataDirect drivers.

It doesn’t matter how big your data file is, what format it's in or how many large files it contains. Regardless of those factors, Apache Hadoop enables data to be segmented and distributed in real-time using HDFS (Hadoop Distributed File System) data architecture and infrastructure.

But even Hadoop’s success is dependent on seamless data connectivity. That’s where DataDirect completes the process.

Three SQL Snafus

Companies building their applications on SQL-based databases tend to run into three major issues:

• One of the biggest problems companies face is how to accommodate database vendors when they use company proprietary libraries to access data sources. Here, CPU and RAM consumption increase due to the requirement of more threads of execution between the application and database. And here’s the hitch: Adding more CPU and RAM to applications will not solve the memory leak or consumption of libraries. Your application throughput will remain the same.
• SQL database vendors are getting competition from NoSQL databases like MongoDB and Cassandra. These databases can perform better SQL queries while using open sources to handle large numbers of non-structured data, and can scale in a cluster easily.
• Another limitation of SQL is the 1 TB (terabyte) of data in the databases while using a join statement. Results indicate that applications are very inefficient and run very slowly while trying to access large amounts of data from the database.

What You Need to Know

DataDirect is engineered to provide superior connectivity between your BI, Analytics applications, Hadoop Hortonworks Data Platform, Hive SQL Queries and Spark SQL.

DataDirect uses Wire Protocol architecture through standard ODBC and JDBC protocols to increase your application performance in milliseconds. This accelerates execution and makes better use of your CPU and RAM. Bottom line? Best throughput results in the industry.

Have a Big Data ​Project with Hadoop in ​Place? Call Us.

DataDirect brings the best data performance and value to your BI and analytics applications in Hadoop HDFS database connectivity including:

• Real-time query and analysis with superior throughput
• Highly secure access with user authentication
• Fast performance with multiple driver tuning options
• Ensured reliability with full standards compliance

Data Direct offers the best ODBC and JDBC drivers aligned to your needs to provide the following benefits;

• Unlock real-time predictable analytics
• Quickly turn big data (like Hadoop) into actionable insights
• Mitigate risks by providing a standard interface to all big data sources
• Reduce operational cost and offer a better user experience

Get started with our advanced DataDirect connectors today, and don’t hesitate to reach out with any questions you might have.