MarkLogic World 2018 Recap: Integrate to Innovate

“The pace of change will never be as slow as it is today,” said Len Hardy, SVP Architecture and Innovation, Northern Trust, paraphrasing media expert Graeme Wood. The San Francisco audience seemingly exhaled in agreement.

It’s exciting to be on the forefront of change. It is also a bit unnerving and exhausting. As entire industries are under threat by disrupting interlopers, corporations respond through innovation. It’s not enough to just innovate though, they must do so at scale, and therefore look for intrepid change agents to lead the charge.

Disruption From Outside — and Within

Those change agents with a mandate to digitally transform realized that to keep pace with agile startups — they needed a disruptive technology. By embracing MarkLogic, they can find themselves at odds with an ingrained culture when they need to disrupt the status quo and existing processes.

In his Day 2 keynote, Geoffrey Moore, author of “Crossing the Chasm” and “Zone to Win,” and self-professed word nerd, explained the dilemma. To ride the wave of an S-curve of innovation – you have to survive the J-curve of research and development. That’s the period of time where you are underwater – investing, incubating, and investing some more – before finally emerging.

Moore explained that in order to advance organizational transformation requires modifying Performance and Productivity metrics. Of course that forgiveness needs to come from the C-suite. And the irony is that few executives have the stomach to sacrifice today’s performance and productivity for tomorrow’s promise.

Most attendees of MarkLogic World (#MLW18) have that C-suite buy-in for change, but often still have to forge the rough waters of colleagues and departments who resist. Transformation, like all change, is hard and unsettling.

As such, the annual MLW conference is as much about learning about this revolutionary database – as it is a mecca for those seeking like-minded, against-the-tide swimmers.

In fact, while months of preparation go into creating the MLW presentations and keynotes, and, under the watchful eye of the event team, everything comes together in a careful choreograph. But the customers bring their own agendas, too – and offer evocative insights like Hardy’s.

In a Financial Services and Insurance luncheon, one of several breakout luncheons, tables were filled with competitors, who, for today, were comrades in disruption. They traded email addresses and tips on getting management and colleagues to buy in – as well as coping strategies for when buy-in doesn’t come. Like the MarkLogic database, the conference was the tableau for enabling innovation and change.

Managing Change: Unlearn to Learn

In his opening keynote, where he interviewed global innovators, Joe Pasqua, Executive Vice President, Products for MarkLogic, asked Northern Trust’s Hardy about challenges in managing change. Citing Alvin Toffler, Hardy said, “Literacy in the 21st century is about learning, unlearning, and relearning.”

Unlearning. That theme, too, popped up throughout the multi-day conference. Unlearning is letting go of the conventional — critically and deliberately looking at what you have done forever and unlearning those patterns.

That sentiment embodies what it takes to learn MarkLogic. The multi-model database provides untold agility because it does not store data relationally, but rather in a tree structure. But that means everything you took as gospel, “how to query,” “how to scale,” “how to prepare data,” really needs to be questioned, and, pardon the pun, uprooted.

For Dan Grecu, a government consultant who traveled from Canberra, Australia, wrapping his head around the document model (not to mention RDF triples) took unlearning and relearning. Now MarkLogic’s data model – and flexibility – makes perfect sense, he told me.

Another area of unlearning was around APIs. David Gorbet, Senior Vice President, Engineering and gifted storyteller, showed how a complex use case like data privacy can be easily addressed – if you flip the current conventions on their heads.

Data Management Made Easy

The traditional way of creating a single schema that lets you bring in customer data from 10s, 100s, even 1000s of silos – (and you will likely forget an attribute that will be needed) – just has you creating another silo. And, you will have lost the metadata along the way – the provenance, lineage, permissions, etc. So all that preparing of the data, ironically, prepares you to fail.

Instead of preparing the data first, said Gorbet, think about the business process. “For our call-center guy, maybe it’s the ability to find a customer, the ability to get his personal information, see its uses, and what consent he’s given, and the ability to update the consent info. Our guy is using a nice UI for this, but that UI is fed through application programming interfaces, or, APIs.”

Of course this is only possible, if all the data is already integrated – with all lineage, permissions and consent (and more) preserved – for both historic (observe the business) and transactional (run the business) data. MarkLogic’s Operational Data Hub (ODH) can do just that.

With the data already in one place, you can focus on building the necessary API to solve the business need. And with the API, it’s easy to build the UI.

Keeping Data Management – in the Database

Having a database that actually manages your data might sound obvious, but in actuality they just allow durable storage, and, if you are lucky, fast query. Everything else is expected to be on the shoulders of application developers. Managing data today means security, privacy, data sovereignty, lifecycle management, entitlements and consent management, said Gorbet. For most databases it’s not “their job” to do these things. “I don’t know about you, but I’m sick of hearing that data management in a DBMS is ‘not my job.’ By today’s definition that is not a DBMS!”

Throughout the conference, this redefinition of data management (to do it instead of ignore it) was a recurring chorus from CEO Gary Bloom on down. You can’t integrate to innovate (the conference’s actual theme) – without a database that was smart enough to actually manage your data. In his keynote, Bloom pronounced the Operational Data Hub the “most common architectural pattern used by our customers,” which set the stage for two days of discussions on the ODH … and its supported open-source framework that gets your hub up and operational quickly.

New in MarkLogic 9.05

SVP of Engineering Joe Pasqua and his team revealed the release of MarkLogic 9.0.5, which includes a number of new features – including:

• Support for Apache NiFi. This feature was enthusiastically received as the integration immediately gives users access to a large and growing number of open source processors and templates that can be assembled using a drag and drop GUI to get data from many sources (such as relational databases) into MarkLogic. NiFi was originally developed by the NSA as part of its first release of opensource software.
• Smart Mastering Smart Mastering is a new feature in MarkLogic that makes it possible to master entity data (like customers) from multiple sources faster than ever before. Smart Mastering consists of matching entities in an operational data hub, then auto-merging for high-scoring matches and recording a notification for a human reviewer for cases where the score indicates a possible, but not definite match. Match scoring rules and merging algorithms, thresholds and actions are configuration driven. APIs are available either through a set of XQuery libraries or a REST service layer. The feature lets you leverage MarkLogic to quickly build your “360 view” — without being forced to buy an entirely separate MDM system.

• MarkLogic Query Service, which is, according to Pasqua, “a really clever way to give customers automatic elasticity in the cloud.” It automatically will add and remove query processing resources (what MarkLogic calls E-nodes) to a MarkLogic cluster as the load rises and falls. This is managed as a service, and makes MarkLogic extremely cost-effective to run in the cloud. No other database has the separation between query and storage (E-Nodes and D-Nodes), giving MarkLogic a distinct advantage when it comes to cost-effective elasticity in the cloud.

Organizations Using the Operational Data Hub

In addition to gaining support from each other, customers like Northern Trust, Credit Suisse, Sony, Boeing, Autoliv, Chevron, Aetna, Erie Insurance, Cisco, LDS Church and many more were on hand to humble brag over their new-found efficiencies and agility with the ODH. Here’s a sampling:

• 4:1 acceleration – Allen Muller, Eaton
• Simplified mainframe migration and reduction of complex distribution challenges – Rob Maxwell, Sony Pictures
• Triple the number of deployments averaging two minutes each,” – David Wilcox, Erie Insurance
• Write to microservice, but read, query and join from MarkLogic — Mike Bowers, LDS Church
• Accelerating digital capabilities – Damian Peloghitis, Chevron
• We ran a multi-state query and achieved a 15-second result – instead of 2.5 days. And the room erupted. – Danny Jackson – Autoliv

And speaking of bragging rights, the annual Christopher Lindblad award for innovation (named after MarkLogic’s founder) went to Hardy and his colleague, Maureen Penzenik, senior solutions architect at Northern Trust. Penzenik and Hardy have led an entire IT organization through the troughs of transformation – from Relational to NoSQL, innovating, incubating and now putting those ideas into production.

Creating a best of breed customer experience is a key goal for the pair. “One of our phrases at Northern Trust is: Achieve Greater,” said Penzenik. “And we believe that MarkLogic will help us to ‘achieve greater’ for our clients.”

There was much, much more over the four-day event, including an inaugural Hackathon, full-day training, and informal pick-the-brain sessions with MarkLogic engineers. It was clear that everyone in attendance shared Lindblad’s view: “Lets really solve the problem – let’s not try to do it the same way everyone else is doing it.”

For More Information

• MarkLogic World Conference Sign up for notifications for conference recordings and information about next year’s event.
• Download MarkLogic Database Join the MarkLogic developer community and discover the power of multi-model — for free!