Working with something revolutionary can be both exhilarating — and frustrating as you don’t yet know all that you need to know. You also know that stubbornly refusing to learn it – is only going to put you at the mercy of those who do. After being impressed with the product I recently joined MarkLogic as their newest product marketing manager. I’ve had to do a deep dive in order to learn enough to write about the new features of MarkLogic 9. I thought it might be helpful for other newbies to see this disruptive multi-model database through a newbie’s lens.
MarkLogic is a database management system, what’s wrong with the databases that already existed?
With the technological advances and the ever-changing market today, companies need a foundation that is adaptable and fast. There is a need for organizations to use data in an integrated manner in real time, and databases need to allow seamless use of data out of the box.
In traditional, relational databases, data resides in rows and columns with very little flexibility and context. To add to the problem data today resides in different systems and in different formats. To fulfill a specific data need, organizations need to create a new unified view of the data by creating schemas and using extract, transform, load (ETL) which can be difficult because relational databases are rigid. Not only does this create a new data silo, but makes the data out of date as soon as the integration starts because business needs are always changing.
Figure 1: What a complex relational model might look like (Source:)
NoSQL solves a lot of the issues with the rigidity in traditional databases and there’s many NoSQL options so why MarkLogic?
“NoSQL” used to mean a non-traditional database. It later “transmogrified” to mean Not Only SQL. So it became the collection of every type of database that didn’t just require SQL. MarkLogic is not only a NoSQL database, it’s the only enterprise NoSQL database.
This means that it comes with all of the features that traditional databases have—features that enterprises need. This includes ACID transactions, government-grade security, high availability and disaster recovery to keep data safe and performance predictable. In addition, MarkLogic indexes data at ingestion and provides a powerful and quick search engine for data. Other NoSQL options require costly add-ons that continue to pile on until you have a complicated mess before you can use your data.
How does MarkLogic work?
Of all the NoSQL databases, the document store is the most flexible. It organizes data in a self-describing hierarchal formats like JSON and XML. Marklogic is a document store. In addition, there’s no need to create one uber data schema upfront because MarkLogic can ingest data as is. You can massage the data into the format you need – as you need it. We call that harmonization. We also allow you to ingest Triples – another data model that typically lives in a type of Graph database (another type of NoSQL). So MarkLogic is actually a document store and a triple store – or what is now being called a multi-model database!
Handling so many types of data natively, with MarkLogic it is possible to constantly adapt to what information and data types are chosen. This save enormous amounts of time and energy that would ordinarily be invested in ETL processes, and you also gain agility later on with future development. This is important because you can’t predict what data you’ll need in the future. In a sense, MarkLogic ensures that databases are future-proof.
Figure 2: Example of a JSON document representing a surgical procedure at a hospital with context
The Secret Sauce: Comprehensive Indexing
A database will create an index – as will a search engine. So typically when you have mixed models – you need to store the corresponding indexes in different “engines.” With MarkLogic, all the indexes are stored in one place – range, full text, geospatial, triples. So when you build a query – you only have to build it through one interface – to hit all the indexes! We call this “composable queries.” So no more joins at the application layer – just one query at the data layer!
Melar’s Path to Learning MarkLogic
If you’re looking to learn more about MarkLogic, here are some of the resources I found most helpful ordered from the most basic, quick overview to content with more detail. MarkLogic also has fantastic, free self-paced or instructor-led classes on various topics for different audience types.
- MarkLogic Done with Bottles Ha! Beer is used to explain MarkLogic’s ACID compliancy, security features, and search capabilities in comparison to traditional, SQL databases. This is actually one of the videos I first watched while learning about the product for my interview. It helped convince me that MarkLogic would definitely be an interesting place to work if their employees explain technical concepts with beer.
- Escape the Matrix In this 47 min. keynote David Gorbett, SVP of Engineering does an awesome job at explaining some of the issues with relational databases. This was one of the first videos I was pointed to after joining MarkLogic to get a graphical representation of what SQL and NoSQL looks like. This is a great intro into learning what semantics is and how MarkLogic semantics help organizations make more sense of data.
- Rethink Data Modeling This whitepaper explores the problems with data modeling in relational databases and provides more detail on the consequences of using relational databases as data evolves. This is also a great read to learn about specific cases where MarkLogic was able to solve a database problem.
- Inside MarkLogic Server Curious about what exactly is going in MarkLogic and the technical details? This eBook will provide the technical capabilities that developers, programmers and data architects will appreciate as well as providing an easy to understand overview for those without technical expertise in the first chapter.
- Building Multi Model Databases This O’Reilly book provides the most detail on databases and what multi-model databases entail and ultimately why they are necessary. Reading this took a bit more time, but I enjoyed learning more about the history of databases and the personal anecdotes on working with relational models.
MarkLogic has helped so many companies build amazing things that wouldn’t have been possible. Ever hear of the Saturday Night Live app that was nominated for an Emmy? Built on MarkLogic. Ever take your car into a shop for repair? Virtually every mechanics shop uses Prodemand by Mitchell 1
There’s so much more for me to learn – and hopefully, my viewing list above will give you the short cut you need to really gain from this revolutionary technology.