XQJ Part X - XML Pipelines

Today's post in the XQJ series is about XML Pipelines. How can we create a pipeline of XQueries or how to integrate XQuery with JAXP based XML processors.

An XML pipeline is a sequence of XML processes, also called transformations, the result of one transformation is passed as input to the next one. In case the question comes up, read Norman Walsh’s essay Why Pipelines? Historically, the transformations coming to mind are XSLT transformation, XML Schema validation, a simple XML parsing, etc. But of course, it can also be an XQuery. This post is about integrating XQuery and XQJ in such XML pipeline.

Let’s start with a pipeline, where the result of a first XQuery is passed on into a second. Given that query execution results in an XQSequence, and an XQSequence can be bound to an external variable, pipelining two xqueries is rather simple.

[cc lang="java"]... XQExpression xqe1 = xqc.createExpression(); XQSequence xqs1; xqs1 = xqe1.executeQuery("doc('orders.xml')//orders"); XQExpression xqe2 = xqc.createExpression xqe2.bindSequence(xqs1);

XQSequence xqs2 = xqe2.executeQuery( "declare variable $orders as " + " element(*,xs:untyped) external; " + "for $order in $orders " + "where $order/@status = 'closed' " + "return " + " { " + " $order/* " + " "; xqs2.writeSequence(System.out, null); xqe2.close(); xqe1.close(); ...[/cc]

In this example, both queries in the pipeline are executed in the context of a single XQConnection object. But this is not required; it is perfectly possible to have two different connection objects, possibly from different XQJ implementations.

The application writer shouldn’t be concerned how the query result from the first is passed to the second xquery. Whether for example a DOM, SAX, StAX, serialized XML or some other (proprietary) mechanism is used, is an implementation detail. The application can assume that the most appropriate mechanism is used.

Let’s now look how an XQuery can be integrated in a pipeline with other XML processors. We'll work out two scenarios, the result of an xquery is passed to an XSLT transformation, and next vice versa, an xquery processing the results of an XSLT transformation. On the Java platform, XSLT transformations are processed through the JAXP api. JAXP makes use of the Source and Result interfaces to query XML documents and handle the results of transformations. As XQJ has built-in support for these interfaces, it is possible to integrate both JAXP and XQJ in a pipeline.

The next example invokes an XSLT transformation, and the results are passed as input to an XQuery. Under the covers, SAX events will be. It is favorable to use SAX from a performance and scalability perspective, rather than when a serialized XML stream would be passed, not to talk about the use of a DOM tree.

First an XMLFilter for the XLST transformation is created. And a SAXSource, the input for the XSLT Transformation.
It is this SAXSource which is bound to the external variable of the XQuery. Subsequently the xquery is executed, and finally the application consumes the results of the xquery.

[cc lang="java"]... // Build an XMLFilter for the XSLT transformation SAXTransformerFactory stf; stf = (SAXTransformerFactory) SAXTransformerFactory.newInstance(); XMLFilter stage1; stage1 = stf.newXMLFilter(new StreamSource("stage1.xsl"));

// Create a SAX source, the input for the XSLT transformation SAXSource saxSource; saxSource = new SAXSource(stage1,new InputSource("input.xml"));

// Create the XQuery expression XQPreparedExpression stage2; stage2 = xqc.prepareExpression(new FileInputStream("stage2.xquery"));

// Bind the input document as a Source object to stage2 stage2.bindDocument(new QName("var"),saxSource);

// Execute the query (stage2) XQSequence result = stage2.executeQuery();

// Process query results result.writeSequenceToResult( new StreamResult(System.out)); ...[/cc]

As you notice in this example, the pipeline is activated starting from the end. Under the covers, the XQJ implementation will invoke the parse() method on the SAXSource bound to the external variable. Which will start invoking SAX callbacks to the XSLT transformation and this will on its turn perform callbacks to the XQJ implementation. These callbacks represent the result of the XSLT transformation, allowing the xquery to yield results back to the application.

Let's now have a closer look at a pipeline where the results of an xquery are passed on to an XSLT transformation.

First we present a utility class, XQJFilter, an XMLFiler based on an XQJ XQPreparedExpression object.

[cc lang="java"]public class XQJFilter extends XMLFilterImpl { XQPreparedExpression _expression;

public XQJFilter(XQPreparedExpression expression) { _expression = expression; } public void parse(InputSource source) throws SAXException { try { XQSequence xqs = _expression.executeQuery(); Result result = new SAXResult(this.getContentHandler()); xqs.writeSequenceToResult(result); xqs.close(); } catch (XQException e) { throw new SAXException(e); } } }[/cc]

Next we’ll use this XQJFilter implementation to build the pipeline,

[cc lang="java"]... // Create an XQuery expression XQPreparedExpression xqp; xqp = xqc.prepareExpression(new FileInputStream ("query.xq")); // Create the XQJFilter, the first stage in our pipeline XQJFilter stage1 = new XQJFilter(xqp); // Create an XMLFilter for the XSLT transformation, the 2nd stage

SAXTransformerFactory stf; stf = (SAXTransformerFactory) SAXTransformerFactory.newInstance(); XMLFilter stage2 = stf.newXMLFilter(new StreamSource("stage2.xsl")); stage2.setParent(stage1); // Make sure to capture the SAX events as result of the pipeline stage2.setContentHandler(...); // Activate the pipeline stage2.parse(""); ... [/cc]

As with the previous example, also here the pipeline is activated through the end, by calling the parse method on the 2nd stage.

In our next post we discuss deferred binding mode, working with external variables, this is essential functionality to handle huge XML documents.

digg_skin = 'compact'; digg_url = 'http://www.xml-connection.com/2007/10/xqj-part-x-xml-pipelines.html';