We announced MarkLogic’s partnership with MuleSoft earlier this year, along with the availability of the MarkLogic Connector for MuleSoft on the MuleSoft Anypoint Exchange. With the Connector, developers can easily use MuleSoft to connect MarkLogic to other applications, data and devices.
We are now excited to announce the newest release of the MarkLogic Connector for MuleSoft. The new release includes multiple new capabilities and enhancements aimed at improving support for data egress, transactions and security.
Together, the combination of MuleSoft and MarkLogic provides a powerful technology stack that helps agile teams manage their changing network of applications and data, and ultimately speeds the delivery of every project.
MuleSoft assists in building out your application network, with the MuleSoft Anypoint Platform providing full API lifecycle management. As the world’s leading integration platform for SOA, SaaS and APIs, MuleSoft makes it quick and easy to connect your whole ecosystem.
The MarkLogic Data Hub plays the role of providing the data persistence layer, acting as a single source of truth to enrich, harmonize and govern your data. Developers can then access and share the data across the MuleSoft API ecosystem.
The Challenge Faced by Developers Today
One of the big challenges that developers face today is gaining access to the data they need in their apps quickly and easily—and being sure that the data is curated and governed for their purposes.
In today’s API economy, it’s easy to make a connection between sources. But, all too often, the network of connections gets overly complicated and unmanageable. It’s no surprise that we have seen such an increase in security problems and unfinished projects due to overwhelming complexity.
The MuleSoft and MarkLogic Solution
From a strategic perspective, MuleSoft and MarkLogic make a perfect combination when building out your tech stack, with MuleSoft providing the ‘smart pipes’ and MarkLogic providing the data integration and persistence layer.
MuleSoft Provides the Integration Layer
MuleSoft is designed to seamlessly integrate into your organization’s existing CI/CD pipeline (there is built-in support for Maven and Jenkins). You can manage your APIs and integration through a single web interface, control access and traffic to your APIs, map dependencies and get customized dashboards.
MarkLogic Provides the Data Persistence Layer
MarkLogic’s Data Hub Platform addresses the complexities of data integration by providing a simpler, more flexible approach to data management, all built on a powerful multi-model database.
Does MarkLogic have ways of ingesting and accessing data alone? Of course. But, for organizations that want the benefit of an API management tool that provides ease of use and agile delivery, MuleSoft is fantastic.
The Hub and Spoke Model
As you can see from the above architecture diagram, what we’re promoting by using MuleSoft and MarkLogic together is a hub-and-spoke approach to data management. We believe this approach solves many of the common challenges with data integration in the enterprise.
Why not use a data lake or data federation architecture?
As you can see in the Data Hub vs Data Lake vs Data Federation comparison, data hubs are the best. The reason is that a Data Hub offers a more agile approach to data harmonization, only modeling what is needed, as it is needed. And, unlike with a federated approach where the data is never actually migrated, a Data Hub indexes all of the data in its raw and harmonized forms.
Securing the Perimeter and the Data
From a security perspective, MuleSoft is designed to keep an enterprise’s perimeter secure. The AnyPoint Platform has out-of-the-box compliance for ISO 27001, SOC 2, PCI DSS and GDPR. And, the platform protects sensitive data through format-preserving tokenization.
Securing the perimeter of a network is only part of the picture, however. Organizations also need to consider data security, where the data is stored. And that’s how MarkLogic complements MuleSoft by really providing the end-to-end security that enterprises need. MarkLogic focuses on data security with encryption at rest, role-based access control and all of the certifications that enterprises require for managing their most critical data assets.
How the Connector Works
The MarkLogic Connector for MuleSoft’s Anypoint Exchange delivers data movement and transformation capabilities for use within MuleSoft Anypoint Studio and Flow Designer. Anypoint Studio and Flow Designer are MuleSoft’s Eclipse- and web-based graphical IDEs used to design, author and run flows.
Under the covers, each flow in MuleSoft is an XML file defining the workflow. This is a perfect fit for MarkLogic, which natively stores XML, and developers can even author flows entirely in XML if they choose. The MuleSoft Connector for MarkLogic is built atop the MarkLogic Java Client API and Data Movement SDK (DMSDK). The Connector is based on the Mule 4 Runtime Engine, Mule SDK 1.1, and is developed and tested with Anypoint Studio 7.
Below is an example of an ingest flow to move data from MuleSoft into MarkLogic:
The Connector can also handle egress flows, which have a similar number of steps but in the opposite direction. A typical egress flow involves an HTTP listener to trigger a MarkLogic query, which then generates a stream of data back to the user.
In the latest release of the Connector, version 1.1.0, we launched many new features and additional enhancements:
- Support for egress of data out of MarkLogic for downstream processing in MuleSoft
- Ability to delete documents in MarkLogic using our Data Movement SDK
- Support for two-way SSL connectivity and MarkLogic certificate authentication
- Support for specifying temporal collections on importDocs operation
- Added dateTime stamps to getJobReport, specifying jobStartTime, jobEndTime and jobReportTime
- Improved UI inputs for Anypoint Studio
- Improved handling of transformation parameters and values
We are continuing to enhance the Connector and welcome your feedback into this initiative. For more information on how to use the Connector, check out this in-depth user tutorial.
Key Resources
Download the Connector on the Anypoint Exchange
View the MarkLogic World presentation
Read the Mulesoft Quickstart tutorial
Matt Allen
Matt Allen is a VP of Product Marketing Manager responsible for marketing all the features and benefits of MarkLogic across all verticals. In this role, Matt interfaces with the product and engineering team and with sales and marketing to create content and events that educate and inspire adoption of the technology. Matt is based at MarkLogic headquarters in San Carlos, CA and in his free time he is an artist who specializes in large oil paintings.