Model Driven Mapping Now Available with Data Hub 5.1

Data Hub 5.1 is the latest release of our software interface for ingesting, harmonizing, mastering, and accessing data. Powered by MarkLogic Server, Data Hub 5.1 provides a unified Data Hub platform for mission-critical data integration use cases.

In this release, we’re excited to announce Model Driven Mapping that makes it easier for non-developers to be involved in the data integration process. With Model Driven Mapping, users can quickly define the mapping expressions to harmonize data from source to target entities right in the Data Hub UI, without writing any code.

In addition to Model Driven Mapping, the release includes a number of performance and usability improvements to the Data Hub’s Smart Mastering capability. Users can now simply split their Smart Mastering configuration into two discreet steps, Matching and Merging, to prevent deadlocks and boost performance due to parallel processing. This is important in scenarios where performance SLAs are not being met.

These capabilities continue our roadmap of simplifying data integration for all users. Stay tuned to learn more about exciting new data exploration capabilities in the first quarter of 2020.

Improved Data Harmonization with Model Driven Mapping

In Data Hub 5.1, Model Driven Mapping provides a UI-driven approach to mapping entities from as is source data to harmonized data.

When using the Data Hub, a user creates a number of data curation steps, which includes steps to ingest the data, harmonize it, and master it. In the process of data harmonization, users map the data model of the incoming source data (source entities) to the model the user defines for its harmonized end-state (canonical entities). With Model Driven Mapping, we made it possible for users to easily define the mapping and harmonization logic right in the UI with an extensible set of declarative expressions.

Previously, non-technical users of the Data Hub had to write custom code or use a separate tool to handle data mappings. With this new functionality, non-technical users can be part of the agile data integration process right in the Data Hub. They can easily harmonize data in various formats, as and when available, without complex ETL.

Data harmonization is a crucial data curation step to transform, translate, and enrich source data into harmonized data. Model Driven Mapping not only simplifies the user experience but also tracks provenance and lineage for data governance.

Improved Smart Mastering

Data Hub 5.1 includes a number of important enhancements to Smart Mastering, MarkLogic’s built-in capability for Master Data Management (MDM). The improvements include the following:

• Meet your performance SLAs by simply splitting Smart Mastering configuration into two discreet steps – Matching and Merging. The discreet steps prevent deadlocks and use parallel processing to boost performance.
• Users can now merge and un-merge specific entity instances, which makes it easier to revert changes during the mastering process (which is somewhat common with MDM). This means that mastered data can be easily kept up-to-date and the system can run real-time operations. This solves a common MDM problem in which inaccuracies live on and the business starts to lose trust in the system
• Users now get more detailed tracking information about when, how, and why properties are matched and merged during a mastering step

Key Resources

Download Data Hub 5.1 – https://developer.marklogic.com/products/data-hub/

QuickStart Tutorial for Data Hub – https://developer.marklogic.com/learn/data-hub-quickstart/

Documentation for Model Driven Mapping – https://docs.marklogic.com/datahub/flows/about-mapping.html