Use XQuery from a Java Environment

While the world of programming—and particularly Java programming—seems to increase, the number of standardized choices is growing as well. In other words, more and more APIs blessed or approved by Sun are available. The result of this standardization is that an increasing number of developers are branching out beyond their core competencies, and learning new technologies.

High on the list of interesting and worthwhile tools and APIs to master are those that deal with data. No matter how cool or clever an application, it's ultimately only as useful as its ability to work with data. And, while the number of APIs constantly expands, the popular and commonly used number of data formats steadily decreases. While some data managers still use object-oriented database management systems, or XML-driven databases, relational databases (RDBMSs) have weathered that storm, and still seem the choice of most data managers. That leaves the Java developer with JDBC (for database connectivity) or perhaps JDO (Java Data Objects) to interact with SQL databases.

SQL databases versus all the others When this article refers to queries and databases, it assumes a SQL database—also referred to as a relational database. However, there are some great applications for XML databases, and even object databases. If you're interested in the world of XML databases, you can check out DB2® Express-C, available for free download . Most notably, XML databases make conversion between XML documents and relational data unnecessary. Also of relevance to this article is that XQuery actually becomes your query language for your database, since XML databases store data as XML data.

Data not in databases has also almost all standardized on XML as a data format. XML is robust, albeit verbose, and there are perhaps more APIs for working with XML than any other non-Java medium in the language. Whether it's parsing, data binding, or transforming, if your application can't deal with XML, then it's considered limited and perhaps even a bit behind the times.

These two seemingly unrelated facts—the propensity for data to live in SQL databases and the popularity of XML for all data outside of a database—has created some unique problems, though. SQL databases are easy to query; XML documents are not. Consumers expect to be able to search through data easily, and while this works nicely with data in databases, it's not so trivial with data in XML documents. Obviously, taking XML-formatted data and dumping it into a database just to make searching easier is the wrong approach. And that's where XQuery—and as a corollary, the XQuery API for Java (XQJ) comes in.

Frequently used acronyms API: application programming interface DOM: Document Object Model GUI: Graphical User Interface IDE: Integrated development environment JAXP: Java API for XML Processing SAX: Simple API for XML W3C: World Wide Web Consortium XML: Extensible Markup Language

XQuery, at its simplest, is a language used to define searches of XML documents. Much as SQL gives specific meaning to SELECT and FROM—in a particular context—XQuery defines a meaning for forward slash (/0 and at-sign (@), as well as a host of other keywords and key-characters.

At its heart, though XQuery is made up of three components:

1. The XPath specification: a means to select zero, one, or multiple nodes in an XML document
2. Additional syntax to select a particular XML document, and to add selection criteria to the nodes returned from an XPath
3. An API—like XQJ, the XQuery for Java API—that evaluates XQuery expressions in a specific programming language

To really master XQuery, you need a solid grasp of all three of these components. For the Java programmer, that obviously means learning XPath, learning the additional XPath syntactical constructs, and then wrapping all of this up in a Java-based API to issue XQuery expressions to an XML document.

The good news is that the XPath and XQuery syntax is fairly intuitive. If you've ever navigated a directory structure in a UNIX® shell, a Mac OS X terminal, or a DOS window, you're way ahead of the game. Add to that basic use of operators like less than (<), greater than (>), and equals (=), and you're over half of the way to being an XPath pro.

XQuery actually depends almost entirely on another XML specification, the XPath spec. XPath isn't about functionality as much as defining a way to create paths that refer to parts of an XML document. For example, the XPath /play/act/scene translates to all of the scene elements nested within an act element, nested within the play root element.

XPaths are relative

Basic XPaths use element names and forward slashes. And by default, an XPath begins with your current location in an XML document. So if you use the DOM, for example, and navigate to a speech element, and then you issue the path speaker, that evaluates to any speaker elements within the speech element that is your current location. So an XPath evaluates relative to your location in a document.

Moving away from the current node

To move to the root of a document, precede your path with a forward slash. So /play looks for the root element called play, no matter where in a document you are. And you can select a parent element of the current element with ../. In fact, this should start to look a lot like a directory structure. So the path ../../personae/title would go two levels up from the current element, then look for a personae element, and then a title element nested within that.

Selecting attributes

You can select a lot more than just elements. Consider an XPath like this: /cds/cd@title. This returns all attributes named "title" that are on cd elements, nested within a root element named cds.

Remember that the @ syntax doesn't return the value of an attribute, but the attribute itself. So @isbn would select all attributes named isbn, not the value of those attributes. Additionally, in XPath, the term attribute refers to the attribute's name and value.

Selecting text

Text is nested within elements The common way to think about an element and the text within that element is to consider the element as "having text." You'll often see things like "the value of the title element is 'You Can't Count On Me.'" But that's incorrect; instead, text is nested within an element. You can think of text as belonging to an element, but certainly not the same way that an attribute has a textual value. An even better way to think about it is to consider that text is truly just nested within an element. That the element is the parent of the text, rather than the value of an element being the text nested within it.

Just as you can select elements and attributes, you can select the text within elements. If your XPath ends in an element name, like /cds/cd/title, then you're selecting elements—and those elements do not include the text within those elements. However, if you want the text within an element, use the text() syntax. So for all the textual titles of a set of CDs, you can use a path like /cds/cd/title/text(). So that path doesn't give you any elements; it gives you the text within the indicated elements.

XPath selects sets of nodes

One of the keys to effectively using XPath is to realize that an XPath always evaluates to a node set. That set might have zero, one, or multiple nodes, but the result of an XPath is always a set. That flies in the face of what most people think when they write XPaths: that the path returns an element, or an attribute, or some text. However, that's not at all the case.

If you use the DOM, you've thought of nodes before. In the DOM, everything in an XML document—elements, attributes, text—is a node. An element is an element node; an attribute (and its value) is an attribute node; even text within an element is a text node. So a path like ../../personae/title, which ultimately results in selecting title elements, actually returns a node set. The set might have zero (there are no matching elements), one, or more nodes. In this case, all of those nodes will be elements named "title."

As your paths get more complex, and potentially select wider sets—perhaps including attributes and elements at the same time, or text and elements—those paths still ultimately are just choosing node sets. Keeping that in mind is key to using XQuery properly. By using XPath, you select a set of nodes; and with XQuery, you then typically select a subset of those nodes with a search criteria, or potentially joining multiple sets of nodes and then applying a search criteria. By keeping in mind that a set can have more than one type (elements, text, or attributes), you'll be able to write those paths a lot better, and ensure they evaluate to what you want.

You've seen how to select sets of nodes based on the names of those nodes (in the case of elements and attributes) as well as the parents of those nodes (in the case of text, or when you select all child nodes of a given parent). On its own, that's pretty powerful; but XPath offers quite a bit more in terms of selectivity, all using what's called a predicate.

Predicate Basics and Syntax

Microsoft® Internet Explorer® (and some other browsers) got it wrong The XPath spec indicates that the predicate [1] refers to the first node in a set; in other words, node sets in XPath are indexed starting with the number 1. However, most versions of Internet Explorer mistakenly implemented XPath with a zero-based index, where [1] refers to the second item in the array, and [0] refers to the first. You'll need to test this within your own browser, but the correct action is for the first item in a set to be indexed with [1], not [0].

A predicate is a partial expression that can be applied to an existing node set. Predicates are placed with square brackets, [ and ]. They're applied to the node set indicated by the path to the left of the predicate. For example, consider the path /cds/cd; this selects all cd elements within the root element named "cds." Then supposed you want the first CD; you can use a predicate to get that CD, like this: /cds/cd[1]. This returns the first of the nodes selected by the path /cds/cd.

Predicates can be applied to any node set

Remember, a predicate applies to the node set to the left of the predicate itself. However, that doesn't mean that a predicate can only appear at the end of a complete XPath. Consider that an XPath itself is really a collection of paths, each returning a node set, and later portions of the path refining or acting on that set. So /cds/cd/title is really three paths:

1. /cds, which returns the root element, named "cds" (a set which contains one element node)
2. cd (relative to the previous node set), which returns all cd elements nested within the previous node set
3. title (again relative to the previous node set), which returns all title elements nested within the previous node set

A predicate must be applied to a node set, but it can be applied to any node set. So here's a completely legal path: /cds[1]/cd[2]/title[1]. This selects the first of the node set selected by /cds; the second of the nodes selected by /cds[1]/cd; and then the first of the nodes selected by /cds[1]/cd[2]/title.

Note: Parts of this path are essentially meaningless; for example, selecting a root element using / and then applying the [1] predicate will always return the first (and only) element in the set. The only time this predicate won't return an element is if the node set itself is empty, when the root element indicated is not named the same as a document's actual root element. But, for illustration purposes—and from a technical standpoint—there's nothing wrong with the XPath itself, or the predicates used.

Going beyond numeric indices in a predicate

Of course, an API that only let you reference sites with numeric values would be pretty limited. You're forced in that model to always know exactly where in the set the items you want are. XPath offers a lot more than that, though. For starters, you can use the last() function in your predicates to select the last item in a set, no matter how many total items are in the set. So /cds/cd[last] selects the last cd in a document.

You can also select all the items that are less than a certain position, or greater than a certain position, using the position() function. The position() function returns the position in a set of a given node. For example, suppose you want the first 5 CDs; you could use the path /cds/cd[position()<6]. This selects all the nodes where position() evaluates to less than 6.

Selecting nodes based on data

Finally—in this short introduction to XPath, although not in a complete discussion of XPath—you can select nodes based on child elements of the nodes, or on attributes of the node. Just as each progressive portion of an XPath is based on the set of nodes determined by the preceding path, the predicate of a set is based on the set that you apply it to. Add to that a predicate's ability to use operators like less than (<) and greater than (>), and you can be quite selective based on data within selected nodes, not just the position of those nodes within an overall set.

So suppose you want all CDs that have an attribute named "style" with a value of "folk." You'd use an expression to first select all the CDs, and then compare the style attributes of those CDs to the value "folk." In XPath, that looks like /cds/cd[@style='folk']. This is pretty self-explanatory based on what you've seen so far. First, a set of nodes is selected through /cds/cd. Then, the attributes of each node named "style" are isolated using the partial predicate @style. Just as discussed earlier, preceding a name with @ indicates an attribute. And that attribute is assumed to be relative to the set of nodes already chosen (in this case, all the cd elements). Then the value of these attributes is compared to a string, "folk." The ones that have matching attributes are returned; everything else is tossed out of the result set.

The same can be done with nested elements of the set selected. Suppose you have a document structured like that in Listing 1.

Listing 1. Structure of a sample CD listing document