Progress DataDirect for JDBC for Apache Spark SQL

    An asterisk (*) indicates support that was added in a hotfix or software patch subsequent to a release.

    For a list a list of issues resolved since general availability, refer to the fixes.

    Version 6.0.1

    • Certified with Azure Databricks Spark Cluster*
       Driver version (F000843.U000414)
    • Certified with Apache Spark SQL 3.0*
       Driver version (F001064.U000500) 
    • Certified with Oracle JDK 11*
       Driver version (F000101.U000050) 
    • Certified with OpenJDK 11*
       Driver version (F000101.U000050) 
    • Certified with OpenJDK 8 on Windows and Linux*
       Driver version (F000101.U000050) 
    • Certified with Apache Spark SQL 2.2*
       Driver version (F000101.U000050) 
    • Certified with Apache Spark SQL 2.0*
    • Certified with Apache Spark SQL 1.4 and 1.5
    • The installer program has been enhanced to support Azul Zulu JRE 11. As a result of this change, the installer has ended support for some of the old operating systems. The installer supports the following operating systems after the enhancement:*
      • UNIX/Linux:
        • Any operating system on a machine using a JRE that is Java SE 8 or higher (LTS version), including Oracle JDK, OpenJDK, and IBM SDK (Java) distributions.
      • Windows:
        • Windows 8.1 or higher
        • Windows Server 2012 RC2 or higher
      Important: These changes do not affect the operating system requirements of the driver.
    • The driver has been enhanced to include timestamp in the Spy and JDBC packet
      logs by default. If required, you can disable the timestamp logging by
      specifying the following at connection: For Spy logs, set
      spyAttributes=(log=(file)Spy.log;timestamp=no) and for JDBC packet logs,
      set ddtdbg.ProtocolTraceShowTime=false.
    • Interactive SQL for JDBC (JDBCISQL) is now installed with the product. JDBCISQL is a command-line interface that supports connecting your driver to a data source, executing SQL statements and retrieving results in a terminal. This tool provides a method to quickly test your drivers in an environment that does not support GUIs.*
    • The driver has been enhanced to support the Statement.cancel API, which allows you to cancel running queries. The Statement.cancel API is supported only on Apache Spark SQL 2.0 and higher.*
    • The driver has been enhanced to support the Binary data type for Apache Spark SQL 2.0 and higher, including the following two new connection properties:*
      • MaxBinarySize allows you to specify the maximum length of fields of the Binary data type that the driver describes through result set descriptions and metadata methods.
      • BinaryDescribeType allows you to specify whether Binary columns are described as VARBINARY or LONGVARBINARY.
    • The driver has been enhanced to support HTTP mode, which allows you to access Apache Spark SQL data stores using HTTP/HTTPS requests. HTTP mode can be configured using the new TransportMode and HTTPPath connection properties.*
    • The driver has been enhanced to support cookie based authentication for HTTP connections. Cookie based authentication can be configured using the new EnableCookieAuthentication and CookieName connection properties.*
    • The driver has been enhanced to support the Decimal and Varchar data types.
    • The ArrayFetchSize connection property has been added to the driver to improve performance and reduce out of memory errors. ArrayFetchSize can be used to increase throughput or, alternately, improve response time in Web-based applications.
    • Java SE 7 has reached the end of its product life cycle and will no longer receive generally available security updates. As a result, the drivers will no longer support JVMs that are version Java SE 7 or earlier. Support for distributed versions of Java SE 7 and earlier will also end, including IBM SDK (Java Edition).*
    • The driver no longer registers the Statement Pool Monitor as a JMX MBean by
      default. To register the Statement Pool Monitor and manage statement pooling
      with standard JMX API calls, the new RegisterStatementPoolMonitorMBean
      connection property must be set to true.

    Version 6.0.0

    GA Release Features
    • Supports read-write access to Apache Spark SQL version 1.2.0 and higher
    • Supports SSL data encryption
    • Supports Kerberos authentication
    • Supports connection pooling
    • Returns result set metadata for parameterized statements that have been prepared but not yet executed
    • Includes a set of timeout connection properties which allow you to limit the duration of active sessions and how long the driver waits to establish a connection before timing out
    • Includes the TransactionMode connection property which allows you to configure the driver to report that it supports transactions, although Spark SQL does not support transactions. This provides a workaround for applications that do not operate with a driver that reports transactions are not supported.
    GA Release Certifications
    • Certified with Apache Spark SQL 1.2 and 1.3

Connect any application to any data source anywhere

Explore all DataDirect Connectors

A product specialist will be glad to get in touch with you

Contact Us