How Lawrence Livermore National Laboratory (LLNL) Supports Its Mission with the MarkLogic® Data Hub

Come to MarkLogic World 2019 in Washington D.C. to hear Valerie Noble’s firsthand account of how LLNL is using MarkLogic to support its mission to use applied science and technology to make the world a safer place.

Lawrence Livermore National Laboratory

(LLNL) understands the challenges of aggregating large amounts of data across multiple sources.

With sponsor expectations for more formal project and information management, the program is now required to demonstrate end-to-end traceability of data from design to production for the lifetime of the system.

The challenge was that by keeping the data in silos, the engineers weren’t considering the downstream use and need of the data.This includes analysts who validate simulation results, or further downstream where if there was a production issue, it could be traced back in the original test data. Data discovery is next to impossible beyond 6-12 months after generation.

Valerie Noble, Director of Applications Management and a 25-year veteran in the weapons program, concluded that they needed a better way to manage their data.

On her search for a new solution, Valerie was introduced to MarkLogic as a possible choice while collaborating on a different software project with Matthew Jacobsen, Configuration and Data Manager, U.S. Air Force Research Laboratory.

“The NoSQL framework and MarkLogic have the ability to handle high variability, whether you want to store structured data that is indexed or have the metadata and the multi-terabyte file on Tier 2 storage. It’s also flexible with its built-in search and retrieval functionality,” Valerie says.

LLNL explored other tools before choosing MarkLogic, but Valerie says they required tremendous effort to stitch and link together manually. MarkLogic fulfilled the requirements for the project in a single platform.

“The best part of MarkLogic is the data hub concept. We have a lot of other COTS tools, and we want that data to be discoverable from one location without migrating all of the data into a single application,” says Valerie. “We select the best tools to manage different types of data and will rely on MarkLogic to serve as the glue that holds it all together.”

LLNL has now hired more than 100 new engineers for its life extension program, and they have made great strides in rapid data ingestion for time series data and making it searchable. They’ve also connected Tableau to MarkLogic. “I was surprised how easy it was to do that,” said Valerie. They are using Tableau as a significant component of their UI for immediate visualization capability of the data, leveraging the built-in filtering and faceting necessary to plot and manipulate time series data.

Learn how  the U.S. Air Force Research Laboratory used MarkLogic to develop an Operational Data Hub that provides real-time access to high volumes of research data.