Progress DataDirect for ODBC for Apache Spark SQL

    An asterisk (*) indicates support that was added in a hotfix or software patch subsequent to a release.

    Refer to the following resources for additional information:

    Version 8.0.2

      Enhancements
      • The driver is now compiled with a Visual Studio 2022 compiler for the Windows platforms. As a result, you must have Microsoft Visual C/C++ runtime version 14.40.33810 or higher on your machine to run the driver.
      • For AIX platforms, the ICU library files that are installed with the product have been upgraded to version 74.1. As a part of this upgrade, the ICU library file names have changed. For the 32-bit driver, the ICU file name has changed from libivicu28.a to libivicu.a. For the 64-bit driver, the ICU file name has changed from libddicu28.a to libddicu.a. This upgrade does not apply to the other UNIX platforms. This upgrade is available starting in build 08.02.1072 of the ICU library files.*
      • For Windows platforms, the ICU library files that are installed with the product have been upgraded to version 74.1. As a part of this upgrade, the ICU library file names have changed. For the 32-bit driver, the ICU file name has changed from ivicu28.dll to ivicu.dll. For the 64-bit driver, the ICU file name has changed from ddicu28.dll to ddicu.dll. This upgrade is available starting in build 08.02.1072 of the ICU library files.*
      • For Linux platforms, the ICU library files that are installed with the product have been upgraded to version 74.1. In addition, the ICU library file names have changed for Linux platforms. For the 32-bit driver on Linux, the ICU file name has changed from libivicu28.so to libivicu.so. For the 64-bit driver on Linux, the ICU file name has changed from libddicu28.so to libddicu.so. This upgrade does not apply to UNIX platforms. This upgrade is available starting in build 08.02.0965 of the ICU library files.*
      • Note: OpenSSL library 3.0.9 has been replaced with a later version. For the latest information about the third-party components that are installed with the product, refer to the Third-party Components Compatibility Matrix.
        The default version of the OpenSSL library has been upgraded to version 3.0.9, which fixes the security vulnerabilities listed on the following page: https://www.openssl.org/news/vulnerabilities-3.0.html.*

        Notes:

        • The driver supports the following OpenSSL 3.0 providers: Default and FIPS.
        • The FIPS provider is supported only on the following platforms: Windows 64-bit, Linux 64-bit, and AIX 64-bit.
        • When installing a new version of the product, the installer program will automatically replace the OpenSSL 1.1.1 library files with the OpenSSL 3.0 library files in the install directory, which will impact all the DataDirect ODBC drivers installed on a machine. Therefore:
          • If you are using multiple 8.0 drivers, upgrade all your drivers to the latest version.
          • If you are using both 8.0 and 7.1 versions of the driver, copy the xxtls27.dll/libxxtls27.so[.sl] file to a different location before installing a new version of the 8.0 driver. Copy it back to the install directory once the installation is complete.
          Refer to TLS/SSL server authentication and TLS/SSL client authentication for details.
      • The driver has been enhanced to allow you to override the default value of the User-Agent header when required by a service. Using the new User Agent (UserAgent) option, you can specify the string value of the User-Agent header to be used in HTTP requests. Refer to User Agent for details.
      • A Password Encryption Tool, called ddencpwd, is now included with the product package. It encrypts passwords for secure handling in connection strings and odbc.ini files. At connection, the driver decrypts these passwords and passes them to the data source as required. See Password Encryption Tool (UNIX/Linux only) for details.*
      • The driver has been enhanced to support connections to Azure Databricks Spark clusters. Refer to Azure Databricks Spark clusters for details.
      • The driver has been enhanced to support HTTP mode, which allows you to access Apache Hive data stores using HTTP/HTTPS requests. HTTP mode can be configured using the new Transport Mode and HTTP Path connection options. Refer to HTTP mode, Transport Mode, and HTTP Path for details.*
      • The driver has been enhanced to support session cookie based authentication for HTTP connections. Cookie based authentication can be configured using the new Enable Cookie Authentication and Cookie Name connection options. See Enable Cookie Authentication, and Cookie Name.*
      • The drivers using base version B0649 and later have been enhanced to include timestamp in the internal packet logs by default. If you want to disable the timestamp logging in packet logs, set PacketLoggingOptions=1. The internal packet logging is not enabled by default. To enable it, set EnablePacketLogging=1.*
      • The Driver Manager for UNIX/Linux has been enhanced to support setting the Unicode encoding type for applications on a per connection basis. By passing a value for the SQL_ATTR_APP_UNICODE_TYPE attribute using SQLSetConnectAttr, your application can specify the encoding at connection. This allows your application to pass both UTF-8 and UTF-16 encoded strings with a single environment handle. 
      • The valid values for the SQL_ATTR_APP_UNICODE_TYPE attribute are SQL_DD_CP_UTF8 and SQL_DD_CP_UTF16. The default value is SQL_DD_CP_UTF8. Refer to Driver Manager and Unicode encoding on UNIX/Linux in the Progress DataDirect for ODBC Drivers Reference for details.*
        This enhancement is available in build 08.02.0449 of the driver manager.
      Changed Behavior
      • The installer program now requires you to install a JRE that is Java SE 11 or higher before running the installer. In earlier versions, the JRE used by the installer program was included in the product. However, to avoid potential security vulnerabilities, the installer program no longer includes a JRE. Instead, the installer program uses the JRE in your environment to allow for the most secure version of a JRE to be used.

        Notes:

        • This change does not affect the JVM requirements for the driver. For the latest driver requirements, refer to the Product Compatibility Guide.
        • The installer program cannot remove the already installed JRE files from the install directory automatically. Remove them manually.
      • The product package no longer includes the ODBC Cursor library file (odbccurs.so) because it has some known security vulnerabilities that could potentially expose you to security risks.*
        Note: The installer program cannot remove the ODBC Cursor library file automatically while installing a new version of the driver. Remove it manually.
      • The product no longer includes version 1.1.1 of the OpenSSL library. The library will reach the end of its product life cycle in September 2023 and will not receive any security updates after that. Note that continuing to use the library after September 2023 can potentially expose you to security vulnerabilities.*

        Note: As a result of this change, when installing a new version of the product, the installer program will automatically remove version 1.1.1 of the library from the install directory, which will impact all the DataDirect ODBC drivers installed on a machine.

      • The product no longer includes version 1.0.2 of the OpenSSL library. The library has reached the end of its product life cycle and is not receiving security updates anymore. Note that continuing to use the library could potentially expose you to security vulnerabilities.*
        Note: As a result of this change, when installing a new version of the driver, the installer program will automatically remove version 1.0.2 of the library from the install directory.
        Notes, Known Issues, and Limitations
        • The driver does not support specifying values for the same connection option multiple times in a connection string or DSN. If a value is specified using the same attribute multiple times or using both long and short attributes, the connection may fail or the driver may not behave as intended.
        • If your database has not been fixed to address the Logjam vulnerability, you can still use the most recent version of the OpenSSL Library by disabling the Diffie-Hellman cipher suites. To disable the Diffie-Hellman cipher suites, specify the following value for the undocumented CipherList connection option: CipherList=DEFAULT:-DH. If you have already specified a value for CipherList, add :-DH to the existing value. The option-value pair for undocumented options can be specified in the odbc.ini, connection string, or the Extended Options field of the GUI. For details, refer to "Configuring and Connecting to Data Sources" in the driver's user's guide.
        • When returning result set metadata for Varchar columns, the Spark Thrift server reports the column type as (12) STRING and the precision as 2147483647. This issue can cause the settings of the following connection options to produce unexpected results: MaxStringSize, MinLongVarcharSize, StringDescribeType, and VarcharThreshold. For the latest information about this issue, refer to the Apache JIRA SPARK-5918 issue Web page.
        • For Spark SQL versions 1.5 and earlier, the Spark SQL Thrift supports only a single connection from a single application (client) per instance. Multiple connections can result in a number of unexpected behaviors, including corrupting data or crashing the Thrift server. While Spark SQL 1.4 resolved some issues related to this limitation, all of the multi-connect issues are not expected to be fixed until Spark SQL 1.6 or later.
        • For Spark SQL version 1.4, the rand() function is not supported. If the rand() function is used, the driver throws the following exception: org.apache.spark.sql.AnalysisException: For input string: "TOK_TABLE_OR_COL";
        • For Spark SQL version 1.5, the Spark Thrift server returns (-17) for the hour() function.
        • All of the Release 8.0 Core Components (header files, etc.) needed for development of ODBC applications on UNIX and Linux are installed into the /include directory created by the product installer.
          Please read the notices.txt file in the installation directory for licensing information regarding the ODBC Core Components. Contact your Progress DataDirect representative if you have any questions.
          Developers using programming languages other than C++: The DataDirect ODBC Driver Manager is a C++ shared library. The ODBC Driver Manager can be used with any language that supports loading a C++ shared library; however, if you are using a programming language other than C++, you must take additional steps to ensure that the C++ libraries are initialized properly when loaded. If you are not using C++, consult your compiler documentation for instructions on using C++ shared libraries with your application.
        • A default installation of the product on UNIX and Linux assumes that the value of the LANG environment setting is en_US. If your locale is not set to en_US, the drivers may not return error messages properly. In this case, you must create a symbolic link in the <Install_Dir>/locale directory that has the same value as the LANG environment setting. This link must point to the <Install_Dir>/locale/en_US directory.
        • The column attributes 1001 and 1002, which were assigned as DataDirect-specific attributes, were inadvertently used as system attributes by the Microsoft 3.0 ODBC implementation. Applications using those attributes must now use 1901 and 1902, respectively.
        • Because of inconsistencies in the ODBC specification, users attempting to use SQL_C_NUMERIC parameters must set the precision and scale values of the corresponding structure and the descriptor fields in the Application Parameter Descriptor.
        • Some IBM AIX C compilers, such as xlc 3.1.4, do not support the use of C++ build binaries. C developers on AIX should consult their compiler documentation and verify that their AIX C compiler supports the use of C++ shared libraries or the application could be unstable.
        • When you first install a driver, you are given the option to install a default data source for that driver. We recommend that you select this option. If you do not install the default data source at this time, you will be unable to install a default data source for this driver later. To install a default data source for a driver after the initial installation, you must uninstall the driver and then reinstall it.

        Version 8.0.1

          Enhancements
          • The driver has been enhanced to support the Decimal and Varchar data types.
          • The new Min Long Varchar Size connection option allows you to fetch SQL_LONGVARCHAR columns whose size is smaller than the minimum imposed by some third-party applications, such as SQL Server Linked Server.
          • The new Varchar Threshold connection option allows you to fetch columns that would otherwise exceed the upper limit of the SQL_VARCHAR type for some third-party applications, such as SQL Server Linked Server.

          Version 8.0.0

          Features
          • Support for Apache Spark SQL
          • DataDirect Wire Protocol technology for improved response time and throughput.
          • Support for all ODBC Core and Level 1 functions.
          • Support for core SQL-92 grammar.
          • Advanced security features, including:
            • Kerberos authentication
            • Data encryption

        Connect any application to any data source anywhere

        Explore all DataDirect Connectors

        A product specialist will be glad to get in touch with you

        Contact Us