4.2.0 archive

Note: This version of Hybrid Data Pipeline has reached end of life. These release notes are for reference purposes.

Security

On-Premises Connector in a Hybrid Data Pipeline Cluster

• Support for the On-Premises Connector in a cluster environment in which multiple Hybrid Data Pipeline nodes run behind a load balancer. The On-Premises Connector allows cloud applications to securely query on-premises data sources without requiring a VPN or other gateway.

Account Lockout Policy

• Support for implementing an account lockout policy. An account lockout policy can be implemented with the Administrator Limits API. An account lockout policy allows the administrator to set the number of consecutive failed authentication attempts that result in a user account being locked, as well as the lockout period and the duration of time that failed attempts are counted. When a lockout occurs, the user is unable to authenticate until the specified period of time has passed or until the administrator unlocks the account.
  With Release 4.2.0, the Hybrid Data Pipeline account lockout policy is by default enabled in accordance with Federal Risk and Authorization Management Program (FedRAMP) low- and medium-risk guidelines. The number of failed authentication attempts is limited to 3 in a 15 minute period. Once this limit is met, a lockout of the user account occurs for 30 minutes.

CORS Filters

• Support for cross-origin resource sharing (CORS) filters that allow the sharing of web resources across domains. While the default CORS setting is off, CORS filters can be enabled with the Administrator Limits API and a list of trusted origins can be enabled with the Whitelist APIs to fully implement CORS filtering. CORS provides several advantages over sites with a single-origin policy, including improved resource management and two-way integration between third-party sites.

JVM Upgrade

• The Hybrid Data Pipeline server and On-Premises Connector have been upgraded to install and use Java SE 8 (8u131).

Tomcat Upgrade

• The Hybrid Data Pipeline server and On-Premises Connector have been upgraded to install and use Tomcat 8.0.46.

Enhancements

Hybrid Data Pipeline Server

• OData 4.0 Support. Support for the OData 4.0 specification. OData 4.0 support includes the following:
  • Support for the $search clause
  • Support for $batch requests
  • The expand clause has been enhanced to support $select, *, $filter, and $top operations
  • The $metadata clause has been enhanced to support the full, minimal, and none arguments
  • Support for the date, dateTimeOffset, and timeOfDay data types
  • Only supports JSON format for payloads
• OData Model Status
  • OData Model status now displays the timestamp of the OData model creation. The timestamp is displayed only when model creation is completed successfully.
  • Users can now view the details of the tables and/or columns that were dropped while generating the OData Model for a given schema map of a Data Source. You can view the warnings through the Web UI as well as through API.
• Version Information. Support for version information to be returned from /api/mgmt/version endpoint. This feature is now accessible via all user accounts. The response is returned in a JSON-style format with the following syntax.

{
"HDPVersion": "major.minor.service_pack.build_number"
"WAPVersion": "major.minor.service_pack.build_number",
"DASVersion":"major.minor.service_pack.build_number"
}

• OData Query Throttling. Support for OData query throttling. OData throttling can be implemented with the Administrator Limits API. When OData throttling is enabled, rows are fetched one page in advance of application requests. In addition, administrators can specify a maximum number of concurrent OData queries to prevent users from exhausting system and database resources.
• Apache Kafka Message Queue. Support for Apache Kafka message queue in a Hybrid Data Pipeline. Apache Kafka allows you to distribute your message queue over multiple nodes for a high-level of availability and fault tolerance. Note that Apache Kafka is not included as part of the product installation. For download information, refer to https://kafka.apache.org/.
• Logging. Support for configuring logging at the data source and user level. Administrators can configure logging using the Web UI or the Administrator Logging API.
• Microsoft SQL Server System Database. Support for SQL Server as an external system database. During the installation process, you are prompted to select either an internal database or an external database to store system information necessary for the operation of Hybrid Data Pipeline. With this enhancement, you can choose Oracle, MySQL Community Edition, or SQL Server as an external database.
• Shared Key Location. During the installation of the Hybrid Data Pipeline server, you are prompted to specify a "Key location" for the generated key. The directory specified serves as the location for additional internal files used in the installation and operation of the server. These files include properties files, encryption keys, and system information. In particular, the files located in the redist subdirectory after installation of the server must be used in the installation of the On-Premises Connector, the ODBC driver, and the JDBC driver. See the Progress DataDirect Hybrid Data Pipeline Installation Guide for details.

• Installation Procedures and Response File. The installation procedures have been modified with the introduction of support for the On-Premises Connector in a Hybrid Data Pipeline cluster, support for Apache Kafka message queue in a Hybrid Data Pipeline cluster, and support for the Microsoft SQL Server system database. New prompts have been added to the installation process. Several of these prompts have corresponding options that appear in a response file generated by the latest installer for silent installation. If you are using a response file generated by an earlier version of the installer, you should regenerate the response file with the latest installer. The new response file should then be used for silent installations. The following table provides the new settings. The settings may differ depending on whether you generate the response file with a GUI or console installation. Further details are available in the Progress DataDirect Hybrid Data Pipeline Installation Guide.

Note: Values for the SKIP_HOSTNAME_VALIDATION and SKIP_PORT_VALIDATION options are now false | true, false for disable and true for enable. These options have the same name in GUI-generated and console-generated response files.

Note: Values for the SKIP_LB_HOSTNAME_VALIDATION option are now false | true, false for disable and true for enable. This option has the same name in GUI-generated and console-generated response files.

New response file options

On-Premises Connector

• Support for the On-Premises Connector in a cluster environment in which multiple Hybrid Data Pipeline nodes run behind a load balancer. The On-Premises Connector allows cloud applications to securely query on-premises data sources without requiring a VPN or other gateway.

Apache Hive

• Certified with Hive 2.0 and 2.1.

IBM DB2

• Certified with DB2 for i 7.3.

Oracle Database

• Certified with Oracle 12c R2 (12.2).

Oracle Sales Cloud

• Support for proxy server connections.

Resolved Issues

Hybrid Data Pipeline server

• Issue 71841. Resolved an issue where the Hybrid Data Pipeline server failed to honor the START_ON_INSTALL environment variable to stop and start Tomcat services.
• Resolved an issue where the installer accepted an SSL certificate only in the PEM file format during the installation of the server for a cluster environment. The installer now accepts the SSL certificate (root certificate) in PEM, DER, or base64 encodings for a cluster installation.
• Resolved an issue where an SSL certificate was required for a cluster installation. An SSL certificate is no longer required for a cluster installation.
• Resolved an issue that prevented the installer from supporting a number of upgrade scenarios.

JDBC driver

• Resolved an issue where the JDBC driver was not connecting to the Hybrid Data Pipeline server by default when running on a UNIX/Linux system.