Step-by-step tutorial to import on-premises DB2 data into Amazon S3 using the Progress DataDirect JDBC Driver.
AWS Data pipeline is a web service that lets you process, transform and move the data securely between AWS Storage and compute services in regular intervals. But, at some point, you would want to import data from your on-premise data sources to your AWS Storage or compute services. AWS data pipeline has a neat little feature which enables you to connect to any database with Progress DataDirect JDBC drivers and import the data into S3 which you can use later to move it across either compute or storage services in AWS.
But how do you access your On-premise database that’s behind a firewall through JDBC drivers?
Progress DataDirect Cloud is our connectivity service that enables you to create secure connections to all of your cloud (SaaS) and on-premise databases (Relational/ Big Data) even if they are behind the firewall. The following tutorial shows how you can leverage DataDirect Cloud JDBC driver to import the data from DB2 which is behind firewall, in to S3 storage engine with minimal effort and by creating a pipeline which you can essentially re-use multiple times.
This On-premise connector when installed gives you a connector id which would be reflected in your DataDirect Cloud account when you are configuring the data source. It is labeled as connector id which is a dropdown from which you can select the connector id that is shown on the machine you have installed the On-premise connector.
Once the pipeline is saved without any errors, you can activate the pipeline. Depending on the schedule that you have configured, the pipeline should run and create a CSV file which contains Accounts Table data read from DB2 database configured on DataDirect Cloud using its JDBC drivers. It should be present in the path that you have configured in the S3DataNode.
We hope this tutorial helped you to import DB2 data into Amazon S3 using Amazon Data Pipeline, which you can use to import it in to various Amazon Compute services. This demonstration is not limited to DB2, in fact you can create Amazon Data pipeline with DataDirect Cloud JDBC drivers to connect and import data from over 50+ data sources which includes your favorite SaaS, Relational and Big Data sources even if they are behind the firewall.
Try Free for 30 Days
Saikrishna is a DataDirect Developer Evangelist at Progress. Prior to working at Progress, he worked as Software Engineer for 3 years after getting his undergraduate degree, and recently graduated from NC State University with Masters in Computer Science. His interests are in the areas of Data Connectivity, SaaS and Mobile App Development.
Copyright © 2018 Progress Software Corporation and/or its subsidiaries or affiliates.
All Rights Reserved.
Progress, Telerik, and certain product names used herein are trademarks or registered trademarks of Progress Software Corporation and/or one of its subsidiaries or affiliates in the U.S. and/or other countries. See Trademarks for appropriate markings.