Introducing the simple mapping operator (!)

September 27, 2012 Data & AI, MarkLogic

MarkLogic supports some of the coolest features of XQuery 3.0, including higher-order functions and switch expressions. This article describes one of my favorites: the simple mapping operator (!). If you like FLWOR expressions, you might just love the mapping operator.

One of the things that has always bugged me about XQuery (and this shows my XSLT background) is that you’re always having to bind variables, even if you don’t really have a good reason to. Here are some good reasons for binding variables:

  • re-use a value in more than one place
  • name a value to explain what’s going on
  • retain access to a value that would otherwise be out of scope

With FLWOR expressions, you’re forced to use a variable for each item you iterate over, whether or not the above reasons apply. In contrast, the “/” operator provides a sort of iteration as well, but it doesn’t require you to bind a variable each time. Instead, it makes use of the “context item” (.) for referring to each item being iterated over. In some cases, it gives you a shorter way of expressing what you might otherwise use a FLWOR expression for. For example, if you wanted to get the lower-case version of the string-value of each node in a sequence, you could write this FLWOR expression:

for $item in $my-nodes return lower-case($item)

Or you could just use a path expression like this:

$my-nodes/lower-case(.)

Not only does this require less typing, but it requires less reading (and I find it easier to read). So “/” is a sort of mapping operator by itself. However, as soon as you get excited about using it this way, you soon find out that it’s quite limited. Let’s say you wanted to also remove extra whitespace in each string. You might be tempted to chain them together like this:

$my-nodes/lower-case(.)/normalize-space(.)  (: illegal :)

But this will result in an XDMP-NOTANODE error. Everything to the left of “/” has to be a node, which means you only get to use “/” once when the right-hand side returns simple values (instead of nodes). After that you have to nest your functions (rather than chain them):

$my-nodes/lower-case(normalize-space(.))

That’s okay, but it would be much nicer to chain these, rather than have to switch back and forth between nesting and chaining. Another limitation of “/” is that it has two additional behaviors when returning nodes:

  • the result is sorted into document order, regardless of the input order, and;
  • all duplicate nodes are removed.

This means that not only is “/” very limited in what you can do with it, but it likely does other things that you wouldn’t want it to do were you to remove those limitations.

Okay, enough about the limitations of slash (/)! Let’s see what this operator (!) can do for us!

Let’s say you want to generate a list of numbers, each prefixed with “#”. You could always use a FLWOR expression:

for $n in (1 to 100) return concat("#", $n)

But now, you can use the simple mapping operator:

(1 to 100) ! concat("#", .)

Unlike “/”, you can you have any sequence to the left of “!”. The original order is preserved, just as with a FLWOR expression.

Here’s a real-world example (for combining multiple .js files into one) that can be rewritten much more nicely with the simple mapping operator:

/html/head/script/@src
/unparsed-text(resolve-uri(concat($js-relative-path-prefix,.), $base-uri))

Nested function calls like the above can be hard to read. In a sense, it reads backwards. When you nest function calls, what you do last (call unparsed-text()) comes first, and what you do first (call concat()) comes last. One way to make this more readable would be to introduce some variable definitions:

for $src in /html/head/script/@src
let $path := concat($js-relative-path-prefix,.)
let $resolved-uri := resolve-uri($path, $base-uri)
return unparsed-text($resolved-uri)

But now we’re back to defining variables just so we can avoid the awkward nesting, not necessarily because the variable names add any particular value. Without further adieu, let’s look at the rewritten expression using the simple mapping operator:

/html/head/script/@src ! concat($js-relative-path-prefix,.)
                       ! resolve-uri(., $base-uri)
                       ! unparsed-text(.)

Much nicer! Now it reads naturally. First you call concat(), then you call resolve-uri() against that result, and finally, you call unparsed-text() against that result.

Interestingly, you’ll note that ! replaces both “for” and “let” in the above example. The first sequence (/html/head/script/@src) potentially returns multiple items; that’s why “for” was necessary. For the sequences that contain only one item, “for” and “let” are effectively equivalent. The thing to remember is that “!” behaves like “for”: the right-hand side is evaluated once for every item in the sequence returned by the left-hand side.

Of course, FLWOR expressions are still necessary for various reasons. To name a few, you need them for sorting (order by), for local variable bindings (“let” expressions), and for joins (where you need to refer to two variables being iterated over, not just one).

With the simple mapping operator, you now have the option to write XQuery code in a very different style than you may be accustomed to. I encourage you to experiment with it, push the bar, and decide for yourself where it helps and where it might hinder the power and readability of your code.

MarkLogic

Evan Lenz