Home Partners Company

Using the Data Integration Suite

Quick Start: Data Integration Suite

The products included in the Progress DataDirect Data Integration Suite (Stylus Studio) allow you to easily build applications to manage today's data integration challenges. From querying relational data, converting data in numerous non-XML formats to XML, to transforming that data to PDF and HTML, to updating databases and generating code for Java and .NET, DataDirect Data Integration Suite has the power and flexibility you need.

This quick start identifies resources that will give you some ideas about how to use the tools of the DataDirect Data Integration Suite in your enterprise. For specific information about using a particular tool or API, see the product documentation for that component.

What Do You Want to Do?

Getting started with the DataDirect Data Integration Suite is easy - it's just a matter of understanding the problem you need to tackle. We've compiled a short list of common scenarios our users have encountered that show how DataDirect Data Integration Suite products were able to help:

Learn About XML Data Integration

DataDirect is happy to offer a weekly webinar to introduce you to the different components of the DataDirect Data Integration Suite. Drop by and learn how you can use our tools to solve a real-world scenario. Topics covered in this webinar include:

  • An introduction to working with XML files in Stylus Studio
  • Accessing and integrating relational data as XML
  • Converting non-XML files - including CSV and EDI - to XML
  • Using visual drag-and-drop features of Stylus Studio to ease data integration tasks
  • Integrating XQuery generated in Stylus Studio in a Web services environment

At the end of the Webinar, you'll have the opportunity to ask questions about integrating XML, EDI, and flat file data in traditional and SOA environments. See you Thursday!

DataDirect Data Integration Training Webinar

Create and Integrate Web Services
  • XQuery WebService is a framework that allows you to expose an XQuery as a Web service. In this example, you'll see how you can use tools in the DataDirect Data Integration Suite to build powerful data services that query, aggregate, and update multiple data sources.
    Creating a Web Service
  • DataDirect Data Integration Suite can help you design, test, and implement Web services, leveraging the power of XQuery to query an incoming request as XML, and query relational databases or XML configuration files that are available to the implementation running on the middle tier, producing the XML needed for a response.
    Data Aggregation Support

Web Services and Heterogeneous Data Integration

IT organizations have been dealing with a variety of data sources for a long time: relational databases, mainframes, text files, and XML documents, to name a few. SOA and Web services have earned acceptance as a solid approach for systems management - one that allows for the broad reuse of existing software assets, provides a sound architectural model for the federation of disparate IT systems, and supports the automation of abstract business processes via a range of programming paradigms.

This informational Webinar will show you how to:

  • Use XML to create logical views of a variety of physical data sources
  • Aggregate XML documents with relational data
  • Consume Web services in XQuery
  • Expose XQuery as data services

Join XML and Relational Data

One of the main challenges facing many IT organizations today is managing data from disparate sources and in different formats. XML has emerged as a technology that is helping to bridge the divide between legacy systems and current technologies. See how you can use the tools in the DataDirect Data Integration Suite to join XML and relational data.

Integrate XML Data from Your Database

Increasingly, XML is stored in relational databases as structured XML. Some applications will continue to shred XML into a relational format or store XML as CLOB values. Other applications can now take advantage of relational databases that support a native XML data type to represent an XML document or XML document fragment. See how you can use DataDirect Data Integration Suite tools to query XML stored as character data and query XML stored as XML here:

Access LDAP Directory Services

DataDirect XQuery provides an easy and efficient way to aggregate data available in a variety of data sources and formats, like relational databases, XML documents, Web service responses, flat files, EDI files, and more. But data storage, with myriad protocols, data stores, and formats, can present a challenge for even sophisticated data integration tools. Learn about using DataDirect XQuery to access LDAP (Lightweight Directory Access Protocol) directory services:

Generate Non-XML Files

XQuery excels at querying and transforming XML, but did you know that you can also use it to generate a non-XML result? You can! Learn how easy it is to use DataDirect XQuery to leverage DataDirect XML Converters to create a comma-separated values file from XML.

Convert EDI Messages to XML

Conversion modules in the DataDirect Data Integration Suite are able to handle many EDI dialects easily - X12, HL7, and others - managing and converting transaction sets, segments, elements, and code lists to XML. DataDirect XML Converters, one of the products in the Data Integration Suite, use their embedded EDI dictionary to help you create syntactically pure and semantically accurate EDI, converting to/from XML or diagnosing problems with incoming and outgoing EDI data.

Convert Proprietary Files to XML

Companies still frequently use proprietary flat file formats to exchange or circulate information, so relying on standards like EDI is not always an option. But you can rely on tools in the DataDirect Data Integration Suite to help you manage conversion of proprietary file formats to XML.

Convert Tab-Delimited Files to EDI

The DataDirect Data Integration Suite has numerous tools to help you convert all sorts of file formats to XML. But what about converting one file format (say, tab-delimited) to another (in this case, EDI)?

Integrate and Transform Office Documents

Office documents have always been a rich source of data, but their proprietary binary formats have made them all but unusable to businesses seeking to integrate them in their processes. New office document standards like the OpenDocument Format (ODF) and Office Open XML (OOXML) are allowing tools like those in the DataDirect Data Integration Suite to unlock office documents' enormous potential.

Additional Resources

In addition to this quick start, consider visiting the following pages to learn about the many additional resources - video demonstrations, user forums, product documentation, developer blogs, and more - for the products in the DataDirect Data Integration Suite.

DataDirect XQuery
Stylus Studio

Data Integration Suite Example Applications

The following document describes how to install, configure and run DataDirect XQuery® examples. Download these examples for free today and learn how easy it is to integrate, query, and publish heterogeneous data sources using DataDirect XQuery®.

  • designPreviewDemo.zip: Contains all of the example files, including XQuery files, XML Schemas, XSLT stylesheets, Database files and the Stylus Studio project file required to run the XQuery example.

Generating Multiple XML Output Files

A common requirement of DataDirect XQuery customers is the ability to generate multiple XML output files. Imagine a directory with XML files, each of which needs to be transformed and result in a new XML document. Wouldn't it be great to query the complete directory and generate the corresponding results, all from a single XQuery? No need to glue multiple queries together, do external scripting, or write Java or any other proprietary code. Simply a single XQuery!

This use case is not new. And not surprising, people are generating multiple XML output files in XSLT since a long time using the xsl:result-document; instruction. However, despite the library of almost 200 built-in functions in XQuery, there is not such a function in XQuery 1.0.

The good news is that DataDirect XQuery 3.1 adds the following out-of-the-box support to serialize query results in a file.

ddtek:serialize-to-url(
$items as item()*,
$url as xs:string,
$options as xs:string)

ddtek:serialize-to-url(); has three arguments, first the data to be serialized, second a URL specifying the file to be saved and the last argument tweaks the serialization process.


XQuery Output Example

The following XML output example creates a output.xml file in the c:\results directory,

let $item := <message>XQuery rocks!</message>
return
ddtek:serialize-to-url($item,

As you see there is no need at all to write any Java code, this can be executed from within your favorite XQuery editor or using the DataDirect XQuery command line utility

As a URL specifies the output location, you can for example easily upload the result to an FTP server,

let $item := <message>XQuery rocks!</message>
return
ddtek:serialize-to-url($item,
"ftp://uid:pwd@myftpserver/results/", "")

Finally, the serialization of the results can be tweaked. Here we encode the XML output as UTF-16 and specify to include an XML declaration,

let $item := <message>XQuery rocks!</message>
return
ddtek:serialize-to-url($item,
"omit-xml-declaration=no,encoding=UTF-16")

Let's look at a more concrete example, where all XML documents are copied from one directory to another. in this query we use DataDirect XQuery's capability to query a complete directory through fn:collection.

declare function local:get-file-name($document-uri as xs:string){
tokenize($document-uri, "/")[last()]
};
for $doc in fn:collection("file:///C:/input?select=*.xml")
let $filename := concat("file:///C:/output/",
local:get-file-name(document-uri($doc)))
return
ddtek:serialize-to-url($doc, $filename, "")

Assuming this XQuery is saved as C:\xquery\copy-document.xq, it can be executed from the command line as follows:

java -jar C:\ddxq\lib\ddxq.jar C:\xquery\copy-document.xq
How to Generate Multiple XML Output Files with XQuery

In this tutorial you learned how easy it is to generate multiple output documents from a single XQuery application. To try it out yourself, download a free trial of DataDirect XQuery today.

Using XQuery to Produce HTML and Images

Need to produce HTML and image output from XQuery? The following tutorial explains how to create an HTML document referencing JPEG files through XQuery. The DataDirect XQuery built-in function ddtek:serialize-to-url is used to save the JPEG images.

The idea is that all the data, including the images, are stored in a relational database. The queries in this post have been tested with DataDirect XQuery against SQL Server, but as we will demonstrate this works perfectly against any other relational databases supported by DataDirect XQuery - OracleDB2SybaseMySQL, etc.

XQuery is known to query and produce XML, but how can it be used to produce images, JPEG files in this particular example? This is another example where the DataDirect XML Converters come to rescue.

An XQuery HTML Example with Images

The JPEG images are stored in our database, and are retrieved by DataDirect XQuery as xs:hexbinary data. Subsequently the DataDirect XML Converters are used to convert the xs:hexbinary into a binary stream, which is saved as a JPEG file.

<html>{
"My favourite pictures.",<p/>,
(: loop over all images in the database:)
for $image in collection("images")/images
(: the file name of the JPEG file :)
let $filename := concat($image/name, ".jpg")
(: create some virtual XML, which will be
processed by the XML Converters :)
let $jpeg := <jpeg>{$image/image/text()}</jpeg>
return
(<a href="./{$filename}">{
$image/description,
(: create the JPEG file :)
ddtek:serialize-to-url($jpeg,
concat("C:/test", $filename),
"method=binary")
}</a>,
<br/>)
}</html>
Generating HTML from XQuery

In this example we've used JPEG files, but you can apply the outlined technique with any other binary format, and so in a nutshell, DataDirect XQuery provides an easy and scalable solution for generating HTML and images from XQuery. Try it out for yourself today by downloading a free trial.

TUTORIAL

Using the Data Integration Suite

jdbc

DataDirect
XQuery Connectors

XQuery Integration to Various Data Sources

community

DataDirect
XQuery Tutorials

Discussions, tips and tricks for
DataDirect Connect drivers