Eaton: From a Data Lake to a Data Hub

Eaton Corporation, the global manufacturing giant that provides energy-efficient solutions to help customers manage electrical, hydraulic and mechanical power more safely and sustainably, had a complex IT landscape and resulting major data-management problems on its hands.

All the data from its source ERP systems, numbering upwards of 200, flowed into real-time, Oracle-based ODS systems. The data would then go through up to six phases of transformations in business logic in Informatica—six different service environments. After it went into Exadata and the Oracle big data appliance, the data then went into data lakes, and from data lakes into its downstream consuming apps.

As explained at MarkLogic World 2018 by Allen Muller, Manager, Enterprise Information Architecture at Eaton, the company is “a very typical manufacturing firm. When we buy a company, small or large, we assess whether it makes sense to change out the ERP system and move it to a centralized ERP system. Most of the time, an ERP system is typically not a business differentiator, so we leave it.”

The big challenge was that the company has over eight million products in its catalog and nearly 110 divisions. And due to its IT complexity, if divisions are making similar products, they can’t tell how much product they have and how much revenue they’re making because the data is in different systems.

Eaton wanted a centralized, near-real-time, standardized view of its operational data to see how better to run its business. Because the company couldn’t consolidate its data, it couldn’t get a clear view of it in order to understand the distribution of its products through its channel partners in a real-time way.

The First Concept

Eaton’s first pass at solving this problem was a typical architecture that they called a “conceptual ERP”—an attempt at one ERP system. “We started off with our standard technologies we had in house because, why not? They’ve been working all these years, they should work now, right?” said Muller. “So we started off with ETL and a relational database … and we started with one subject area: sales and orders.

“We set off down the road, and it didn’t work very well. It was very difficult to design, it was inflexible, we couldn’t change it and it was very costly in terms of time, resources and I would say, missed opportunities. We still stand by the fact that our idea was good; we were just going about it the wrong way.”

When the team had to make a change to the model, they had to go change all the business logic in the ETL. Figuring out where those changes in the data were made was almost impossible because different groups were doing different things in ETL. The localized data was totally gone, and the model couldn’t hold all of the local data that was needed. So business consumers were disappointed because the special fields that came from their ERP system, which helped them run their business and were very important to them, were gone.

“The data model is what I would call brittle,” said Muller. “So all of this resulted in a situation where we were just not able to deliver. It took us more than two years to get sales and orders into a form where it was working, and then the first people who saw it said, ‘Hey we want this,’ and it wasn’t there. And we had to go back and change it.”

Exploring Solutions

So the team stepped back, changed course and met MarkLogic Corporation to discuss its Operational Data Hub (ODH) pattern. “We’re very cautious at Eaton; we’re not a leading-edge company, we’re a later adopter of new technology,” said Muller. “We went to a very simple PoC—a simple architecture. We didn’t build a whole lot of infrastructure, we just wanted to prove the central piece. Could MarkLogic do what we needed it to do?”

The Eaton team took hydraulics and aerospace data for invoices, factored in a number of ERP variants and then added 50% to PoC time. “We put our finger on the scale against MarkLogic because we figured we’re new to this, we’re two people. We also factored in the formality of processes that would take place at Eaton to make this military grade,” explained Muller. Even after they did that, they had a 4.2 to 1 acceleration using MarkLogic. Instead of the project taking two years, it took just six months!

The Eaton team appreciated MarkLogic’s deployment simplicity. “You just install it, and it’s ready, compared to Oracle where you have people working for weeks to get things all tuned up,” said Muller. He said they like the clustered architecture because they are tired of buying massive servers. “We want to go to something we can scale incrementally, horizontally, with smaller commodity servers. So we follow the MarkLogic® data hub pattern, pull data out of tables, put it into MarkLogic and we keep track of it as we go. So that makes things a lot simpler.”

MarkLogic’s ODH pattern was able to change the way the team approached its data-management challenges. The flexible data model and the ability to do data model versioning allowed them to work in an agile fashion and release data very quickly, even before the requirements were complete. Most importantly, it was usable data that customers could get at, and their ability to reach the data fields they needed wasn’t disrupted.

Next Steps

The Eaton team is planning to build a program to address finance data, manufacturing sales and orders data and supply chain data. “We absolutely needed this data hub,” said Muller. “Having all our data in one hub in a conformed way is very important. MarkLogic is the tool for us. Our traditional tools just didn’t work. Oracle is a good database, but it just doesn’t do what we need it to do. We looked at other NoSQL databases—MongoDB and some of the others out there—and they don’t have the suite of tools that comes with MarkLogic that allows you to do these things. It’s well suited for quicker delivery methods, and the canonical and local data support is a big advantage.”

Listen to Allen Muller’s full presentation at MarkLogic World 2018 to hear all about Eaton’s journey from lake to hub.

Read MarkLogic’s ebook, “Introducing the Operational Data Hub,” to learn everything about our innovative and proven enterprise pattern.