XQueryWebService is a framework that allows you to expose an XQuery as a Web service. In this section, you'll see how you can use the Data Integration Suite's XQuery product to build powerful data services that query, aggregate, and update multiple data sources using XQuery.
Get hands-on experience running XQuery Web service applications developed using the XQueryWebService framework on a live application server. Test Web service operations exposed by the Employee Lookup Example, run finished applications like the Photo Search Demonstration, use XQuery to query an incoming request as XML as shown in the Travel Reservation Demonstration, or just explore the WSDL used to describe Web service operations that query relational data sources.
XQueryWebService is a framework that allows you to expose an XQuery as a Web service. In this section, you'll see how you can use DataDirect XQuery to build powerful data services that query, aggregate, and update multiple data sources using XQuery.
The application used to illustrate some of the features of the XQueryWebService framework is a simple employee lookup: based on the employee ID you enter, the query returns information such as first and last name, job level, date of hire, and so on.
Before getting started with the example presented in this section, you'll need the following:
You can use any Java servlet container you like; we used the open source Apache Tomcat for this example, so you will have to adjust references to Tomcat to the Java servlet container of your choice in your installation.
To get started with the XQueryWebService example:
Once you've taken care of the basics, you're ready to move on, starting with specifying relational data sources, as described in the following section.
If your queries access relational data, you need to register the database connection with the Java servlet container. This section describes two different ways to do this:
You can use the Web Application configuration file (web.xml) to specify the database connection, as shown in this example, which specifies a connection to Microsoft SQL Server:
The name of the <param-name> element can be any string you like.
Connections specified this way are created and then discarded with each request. A more efficient technique is database connection pooling, which is discussed next.
Connection pooling is a technique for specifying database connections that allows a Web application to create a database connection on demand, and then return it to the pool when it is no longer need, rather than discarding it. Connection pooling can improve response time and help preserve database resources. Web server requests are locked if no connection is available in the pool.
Another benefit of using connection pooling is that it allows for connection recovery in the event that the connection is lost — if the server times out, for example. Dropped or disrupted connections are automatically replaced once the server is returned to service.
All popular Java servlet containers offer a connection pooling framework, and DataDirect XQuery can be plugged into most of them (Apache Tomcat, BEA WebLogic, IBM WebSphere, and JBoss, for example).
Here's how to create a connection pool in Apache Tomcat:
Note that the name= attribute of the <Resource> element has to match the <res-ref-name> element (here it is "jdbc/employee-lookup") in the web.xml configuration file you described previously. This is the name that the Java servlet uses to perform the Java Naming and Directory Interface (JNDI) lookup required to retrieve the connection pool.
As mentioned here.
Once you've specified your relational database connection, you can start to think about the technology you want to use to access the Web services based on your XQuery. We'll discuss two popular technologies — SOAP and REST — in the following section.
Data services — that is, your XQuery exposed as a Web service — deployed on the XQueryWebService framework can be accessed using two techniques:
SOAP is a W3C Recommendation and has been around for nearly a decade. SOAP is usually the more appropriate of the two techniques for complex processing or when security (exposing sensitive data) is an issue. But REST is gaining popularity for a couple of reasons, including minimal requirements on the client, and an interface — the URI — that is straightforward and well-understood.
Let's take a look at a simple XQuery, emp.xquery, which we have saved to our local XQuery directory (c:\MyQueryDir, as defined inweb.xml). When run against the SQL Server pubs sample database, this XQuery returns an employee record given an ID ("A-C71970F").
Using REST, this XQuery can be executed from any Internet browser using just this URL:
Notice that the employee ID ("id=A-C71970F") is visible in the URL.
The result would look something like this:
Using SOAP, on the other hand, requires submitting the following SOAP request (XML):
The result, shown here, is pretty much the same as the one returned using REST, only now it is "wrapped" in the SOAP envelope:
The XQueryWebService framework includes some simple tools that let you test the Web service operations you include in your applications.
The XQueryWebService framework dynamically lists all the operations exposed by the Web service created from the XQuery in your Java servlet container's XQuery folder. This page, for example, was generated by the Employee Lookup example:
To display this page, just point your browser to the XQueryWebService root —http://examples.xquery.com/employee-lookup/, for example.
A simple HTML interface allows you to test an operation. For example, if we click the "emp" operation, the following HTML page is generated:
To test the operation, simply provide the requested value and click the "Invoke" button. Again, the testing interface is generated dynamically, so the form itself varies based on the operation — if an operation does not require a parameter, it is invoked as soon as you select it. This functionality is supported by the REST technology only.
If you choose to use the SOAP transport mechanism in your Web application, you might want to take advantage of the XQueryWebService framework's ability to generate a WSDL document. (A WSDL document can be used to create a set of classes that allow you to manipulate a data service as if it was a local library.) Learn about generating a WSDL document in the next section.
XQueryWebService also automatically generates a Web Service Definition Language (WSDL) document based on the XQuery in your Java servlet container's XQuery folder. The WSDL document describes the services that are exposed by a given XQuery, and this can be useful if you plan to provide programmatic access to one or more of those services.
If we take a closer look, we see that the WSDL document defines a single service (<wsdl:service>), exposed through two ports: SOAP and HTTPGET.
Each query is exposed as a WSDL operation (<wsdl:operation>), with each query's external variables exposed as operation parameters. Further, all built-in schema types are preserved in the parameter declaration.
Consider the following external variable declared in our example XQuery:
In the <wsdl:types> section of the generated WSDL document, you find the following global element:
The input message references the global element, "emp":
Currently, XQueryWebService does not support user-defined types for external variables.
You can use a WSDL document to create a set of classes that can be used to manipulate the data service as if it was a local library. Learn more about using WSDL service references in the next section.
Modern IDEs like Microsoft Visual Studio and Eclipse provide complete support for consuming Web services — for most of them, making the WSDL document available to the IDE is all that's needed to generate a set of classes that can be used to manipulate the data service as if it was a local library. For example, when we open the Employee Lookup WSDL in Microsoft Visual Studio as a Service Reference, the emp and empxsd operations are exposed, as shown in the following illustration:
Such a binding framework works extremely well when the WSDL document makes use of XML Schema to describe the SOAP message payloads. Consider creating a simple C# application using empxsd, as shown here:
If we run our application in debug mode inside Microsoft Visual Studio, we can see that the variables (first name, last name, and so on) are initialized with values from the Web service.
To illustrate how XML Schema can be used to augment a data service WSDL, let's revisit the Employee Lookup XQuery (emp.xquery), and make a few modifications, as shown:
This query is almost identical to the one introduced earlier in this tutorial, except that the root element (<ns:root> ) is now placed in a different namespace. Accordingly, we need to create an XML Schema, let's call it employee.xsd, that describes what the <ns:root> element looks like. We'll also put this XML Schema in the same folder as emp.xquery and empxsd.xquery (c:\MyQueryDir):
Now that this is done, we can place both the query file and the XML Schema file in the Java servlet container folder that contains our XQuery — in our example, this is c:\MyQueryDir.
If we now open our WSDL URL, we can see that the embedded XML Schema contains an import statement referencing the XML Schema associated with our WSDL:
Now, the SOAP message that describes the operation's return types references the global element <ns:root> defined in the WSDL XML Schema:
Now that you know about the DataDirect XQuery XQueryWebService framework, take a look at how it was used to build an application that searches Flickr photos.
The Photo Search demonstration shows you how the XQueryWebService framework can be used with DataDirect XQuery to build a photo database search tool that allows you to query a database, create new user accounts, and perform other administrative tasks.
Note: The Photo Search demonstration uses the Flickr API but is not endorsed or certified by Flickr.
The Photo Search demonstration lets you create a user account and search Flickr for photographs using any keywords that you specify. As shown in the following illustration, to get started all you need to do is login — you can use any value you like for a user name. DataDirect does not record or share any of the information you provide (user name, search terms, and so on).
If you look closely at the URL, you'll see that the file extension is, perhaps, not what you might expect — it's .xquery. The HTML for the login page, as well as for all the other pages in this application, was generated using XQuery. (See for yourself — clicking the View Source button, which you can do from any page of the application, displays static HTML of the XQuery source responsible for generating the page of the Photo Search application you're looking at. You'll learn more about View Source and other tool bar functions in a later topic.)
Once you've logged in, the Photo Search application displays the search page. On this page, you can
When you click the Search button, the Flickr photos are searched using the keywords you specified; after a moment, thumbnails of the photos matching your search criteria are displayed. If we use the search term XQuery, for example, we find several photos of XQuery conferences and XQuery experts such as Michael Kay and Daniela Florescu.
After you've run a few search terms through the Photo Search demonstration, click the Show me my queries button. In a moment, the application displays a list, sorted chronologically, of all the queries you have run.
You can rerun a search by simply clicking the term in the list. Or, if you want to start fresh, you can click Delete my queries to delete all query history.
The DataDirect XQuery Photo Search demonstration application uses numerous technologies — there's XQuery and Web services, of course, but there's also the REST transport mechanism, HTML, and SQL, among others. A high-level view of the application architecture is shown on the following page.
The DataDirect XQuery Photo Search demonstration application uses a number of technologies — there's XQuery and Web services, of course, but there's also the REST transport mechanism, HTML, and SQL, among others. A high-level view of the application architecture is shown here:
In the following sections, we'll take a closer look at the functional components that make up the Photo Search demonstration application.
The Photo Search demonstration is a pretty nifty application. Of course, its real purpose is to show you how the XQueryWebService framework can be used to update a database, search Flickr photos, and generate dynamic HTML. In this section, we'll introduce you to the toolbar, which provides easy-to-use tools that can help you learn about the power of DataDirect XQuery and Web services.
Every screen of every XQueryWebService demonstration application includes the following toolbar:
As you might imagine, you use the View Source button to view the XQuery source code that generated the HTML page. Clicking this button displays static HTML of the XQuery code responsible for generating the HTML of the current page, as shown in the following illustration.
The Get Web Services button displays a list of all the Web service operations exposed by the application. The following illustration shows the list of the Web service operations exposed by the Photo Search demonstration:
Clicking an operation in the list displays a simple interface that allows you to test the operation individually.
Finally, the Learn More button opens a Web browser to the start of these pages.
Click here to start the DataDirect XQuery Photo Search demonstration. Take a few moments to familiarize yourself with the application before continuing with the next section in this discussion.
In the remaining sections, we'll look at the three main functions supported by the Photo Search demonstration application — logging in, searching Flickr photos, and displaying saved queries — and see how DataDirect XQuery and the XQueryWebService framework is used in all of them. We'll start at the beginning, with logging in.
Once you start the Photo Search demonstration, you need to log in. This section describes the XQuery code that's used both to generate the HTML for the Photo Search login page and to perform login functions.
As shown in the following illustration, the Log In page is generated by two XQuery files — login.xquery and toolbar.xquery. The login.xquery file, which you can see in the URL displayed in the browser's Address field, is the file directly responsible for generating the Log In page.
Note, though, that the DataDirect XQuery logo and the toolbar come from the toolbar module, which is imported from toolbar.xquery. (The XQuery code from login.xquery that calls toolbar.xquery is highlighted in the preceding illustration.) Because the toolbar is a modularized component of the Photo Search application, you'll see this import statement at the start of the XQuery for most of the pages in the application. Values for the toolbar functions (View Source, Get Web Services, Learn More) change depending on the specific XQuery that is importing the XML fragments from toolbar.xquery, as you can see in the following code:
So far, what we've seen is pretty straightforward usage of XQuery. Let's take a closer look at the login functionality defined in the HTML inside login.xquery, where we first see how DataDirect XQuery can be used to update a relational database.
The ajax_insertUser() function concatenates the URL with insertUser.xquery?login= with the login value provided by the user — so, we have http://examples.xquery.com/photo-search/insertUser.xquery?login=ohenry, for example. Next, let's look at insertUser.xquery to see what it does with the login value.
The insertUser.xquery is, as the name suggests, responsible for adding new users to the database. This is accomplished using the proprietary DataDirect XQuery ddtek:sql-insert function to insert a single record into a database table.
Here, the ddtek:sql-insert function is being used to insert the user name value ($login) into the "login" column of the "users" table.
Once the user name is added to the database, the ajax_insertUser() function opens a new URL for main.xquery, again creating the URL by concatenating the query name with the user name (login) value — http://examples.xquery.com/photo-search/main.xquery?login=ohenry, for example. The URL is opened in the same browser ("_self"):
It is main.xquery that provides the search and query-listing capabilities of the Photo Search demonstration application.
Now that you have seen how you can use XQuery to update a relational database, let's take a look at the heart of the Photo Search application — XQueryWebService framework was used to implement this search functionality.
Once you have logged in to the Photo Search demonstration, the Web server displays a page created by main.xquery. This section describes how DataDirect XQuery uses the Flickr Web service API to search for photos.
The Search page lets you
This section focuses on the search function. Executing a previously run search is discussed in the following section.
The Keywords and Max hits fields have default values, but you can change them to whatever you like. For the purposes of this discussion, we'll use XQuery as the keyword, and 8 for the number of hits we want returned.
The Search button is part of a typical HTML form:
where the onsubmit="Search()" script event that invokes the "Search()=" function, which in turn calls SearchInternal. Let's take a closer look at SearchInternal to see what it does.
The SearchInternal() function in main.xquery is defined as follows:
After disabling the Search, Show me my queries, and Delete my queries buttons, the ajax_getphotos() function defined in ajax.js is called. Among other parameters, it passes the keywords and maxHits parameters.
These parameters are used to build the URL for getphotos.xquery.
The getphotos.xquery is where we first see a reference to the the Flickr API (http://api.flickr.com/services/rest/):
It is in the XQuery module flickr.xquery that the flickr:flickr_search() and flickr:photoURL() functions called by getphotos.xquery are defined. Both of these functions rely on the Flickr API, which is exposed as a set of Web services. Let's take a closer look at flickr.xquery now.
The flickr.xquery XQuery module defines numerous functions that serve as wrappers for these Flickr Web services. These functions invoke the Flickr Web services using the doc() function. The DataDirect XQuery ddtek:wscall() function is another way to invoke Web services, especially those implemented using the SOAP protocol.
The flickr:flickr_search() function builds the URL of the server hosting the Flickr photo search Web services; the search itself is executed using the values for the $keywords and $maxhits parameters passed from getphotos.xquery.
Once the search is complete, the flickr:photoURL() function returns a URL for each photo matching the search terms. An example of this URL is commented in the example of the flickr:photoURL() function shown here:
If we look again at getphotos.xquery, we see that it returns a thumbnail version of the photo ("small"). This image, along with any others that match the search term (up to the Max hits defined by the user), are displayed in a table inserted by main.xquery when it revises the original search page.
When the user clicks a thumbnail, a new browser instance is launched to display a larger (medium-sized) rendering of the image:
In addition to the photo thumbnails that appear on the search page, the Search, Show me my queries, and Delete my queries buttons are again enabled. We'll take a look at how Show me my queries works next.
We've shown you how DataDirect XQuery can be used to invoke a Web service. We'll conclude this discussion of the Photo Search demonstration application with a look at how photo search queries can be retrieved from a relational database and run again.
We've already shown how you can use DataDirect XQuery to update a relational database. This section describes using DataDirect XQuery and the collection() function to retrieve data from a relational database — in this case, the photo search queries that are saved by the Photo Search demonstration.
Each time you execute a search of Flickr photos, the query — which includes your user name, the search term, the maximum number of hits, and some other information — is saved to the queries table in the database running on the Web application server. When you click the Show me my queries button, the Photo Search application queries the database for your queries and displays the results, as shown in the following illustration:
Clicking the Show me my queries button invokes the ajax_savedqueries function, which in turn calls savedqueries.xquery. Let's take a look at that XQuery code now.
As you can see, the savedqueries.xquery uses the collection() function to query the Photo Search database for saved queries:
Notice that savedqueries.xquery contains only XQuery — the SQL statements required to connect to and query the relational database are created automatically by DataDirect XQuery. Once a user in the users table whose user_id matches the $login, the record containing the query information (the key words used for the search and the max hits specified) is returned and presented in a table, with each record appearing in its own row.
The XQueryWebService framework allows you to expose an XQuery as a Web service. Implemented as a library for Java classes that supports numerous Java servlet containers like Apache Tomcat, JBoss, IBM WebSphere, and BEA WebLogic, the XQueryWebService framework simplifies the design and implementation of Web Servlet Applications.
Using the Flight Reservation use case from the W3C SOAP specification, this example illustrates how you can use XQuery to query an incoming request as XML, and query relational databases or XML configuration files that are available to the implementation running on the middle tier, producing the XML needed for a response
Let's take a look at the XQueryWebService framework architecture before getting into more of the details.
A high-level illustration of the XQueryWebService framework architecture shows all the pieces at work to expose an XQuery as a Web service:
The Travel Reservation XQuery Web service application starts with a user executing
client.xquery, whose purpose is to submit travel itinerary information to the Web service. The
client.xquery code looks like this:
The itinerary element,
<dd:in>, which contains the information that is consumed by the Travel Reservation XQuery Web service application to return a list of airports that provide flights that satisfy the requirements expressed by the itinerary.
Using the <%=ConfigurationManager.AppSettings["DDXQ"]%> Web service call function (
client.xquery invokes the Travel Reservation XQuery Web service using
"flights.xquery" is defined in the Web Service Description Language (WSDL) that describes the operations of the Travel Reservation XQuery Web service application.
On the server, the Travel Reservation XQuery Web service application runs
flights.xquery. This XQuery code defines a function that looks up the airports for a given city in a relational database that is available to the Web service implementation and then looks for all the departing and returning airports in an incoming message.
... If there is more than one airport for a city, or no airport, we return a message which requests clarification ...
... Otherwise, departing and arriving airports are provided in pairs for the departure and return legs of the trip ...
See other examples of XQuery Web service applications built using the DataDirect XQueryWebService framework.
Many web applications exchange data as XML, but that data is usually stored in and queried from relational databases, CRM, ERP, proprietary repositories, and a hodgepodge of other systems. Unfortunately, the languages most commonly used for creating or processing data on the web were designed neither for processing XML nor for integrating data among multiple heterogeneous sources. These are precisely the tasks for which the XQuery language was designed.
This paper shows how to use XQuery for data integration, and how to expose an XQuery as a RESTful data service using a Java servlet. Listing 1 contains the source code for the servlet. This servlet uses the name and external variables of any XQuery to provide a REST interface to the query and deploys the query.
As an XML-oriented data integration language, XQuery can be used to access XML, relational, and flat file formats such as EDI to create complex XML and HTML results. To deploy a query, a developer saves the query into a designated deployment directory in a secure location accessible to the servlet. Subsequently, developers can invoke any query in this directory using its REST interface, which requires nothing more than an HTTP GET or POST operation using a URL that represents the query and its parameters.
Click here to read the full article: An XQuery Servlet for RESTful Data Services