ACID Transactions…Check!

I remember the first time I wrote to a MySQL database. It was a simple JDBC program for test purposes, a bit more than Hello World but nonetheless designed to merely kick the tires of Java and JDBC, which were both relatively new at the time. With the Pet Store app not yet in existence, I wrote a simple command line program which inserted a record into a Person table, followed by an Address record insert into another table. I intended to do this as a single transaction. I was quickly disappointed as I was met with a SQLException at the first run. The accompanying message was something akin to “Transactions not supported.” This threw me for a bit. After all, even the early incarnations of the JDBC spec had facilities to turn off AutoCommit, as well as commit and rollback methods at the Connection level. My try-catch-finally block was well formed and there was nothing at compile time to warn me of what eventually happened.

For those familiar with the history of MySQL, the reason for my failure was simply because MySQL did not support transactions in its early days. Since the JDBC specification was a general contract for driver implementers, the only thing the creators of the MySQL JDBC driver could do was simply cough up a RuntimeException when any of the transaction-based methods were called. Because I knew that I was using something that was free, and was excited enough to have my own personal database on my laptop (which was considered cool back then…), I was willing to wait things out a bit, so I checked the MySQL website for roadmap information. What I found was surprising. Instead of a message to the tune of “hang in there, transactions are on the way” what I found was a lot of strongly worded opinions about why transactions weren’t really necessary because of the super performance you could get without them. While I certainly didn’t agree with these opinions, I admired MySQL for their pluck (and success) in presenting their own weakness as a feature.

We know today how things ended up. InnoDB was eventually included as a transaction engine for MySQL (you know, assuming you wanted such a silly thing) and the “transactions are bad” messaging began to wane. I even remember attending a conference around the time that this happened, where I was given a MySQL T-Shirt. On the front was the MySQL logo and on the back was a “selected checkbox” graphic followed by the word “transactions.” Apparently it was a check-list item all along and I finally had the T-shirt to prove it.

Today, in the NoSQL space we’re seeing more of the same. Except now, unlike in the relational era, there is not only less obvious precedent for the need for transactions in the NoSQL space, but the tech industry seems to be propagating the “no transactions” messaging. It seems everywhere I look, the term NoSQL is associated with a lack of transaction support. A Big Data article from Wikibon as recently as March 29th 2013 posted the following when discussing NoSQL: “The downside of most NoSQL databases today is that they traded [ACID] compliance for performance and scalability. Many also lack mature management and monitoring tools.”

Interesting. And yes, while that is technically true, the key word to remember here is “most” (you can probably guess where this is going…).

As for the vendors themselves, for the many that do not provide ACID transactions, they happily point to the contrived concept of BASE (basically available, soft state, eventually consistent) to provide cover for what’s missing. And then there’s BASE’s kissing cousin, the CAP theorem, which seems to add some intellectual credibility to the domain by almost suggesting that ACID and distributed NoSQL may not be possible; more welcome FUD for the transaction-less NoSQL product vendors.

So if you’re running mission critical applications, all of this messaging might suggest that one can only go so far with a NoSQL database. After all, a quick look at the BASE adverbs and adjective in isolation – basically, soft, eventually – are not exactly words which inspire confidence (I’ve heard the less-than-kind call these weasel-words). From a product standpoint, it’s even gotten to the point where with one particular NoSQL database, a recent upgrade boasted “better default settings that should help prevent data loss.” Yes, those are the exact words of the post. And while I can certainly appreciate the candor, I have to feel like something just isn’t right there. Call me old-fashioned but I was and still am under the impression that the whole point of a database is to actually save data.

What exactly have things come to?

Though I can’t say this for certain, I truly believe that the concept known as BASE will also wane as the non-enterprise NoSQL vendors play catch-up in the maturity space. I do believe that they actually want to have transaction capability as well as other enterprise features but , much like the early days of MySQL, they are either downplaying their weaknesses or somehow positioning them as either a collective strength or necessary trade-offs for “performance.”

At MarkLogic, we respectfully disagree with these notions.

Here we are, a 10+ year old company with a NoSQL database that supports transactions, has government-grade security, has built in search, scales to hundreds of terabytes, has actual production installations of 100+ node clusters, and leads the NoSQL pack in revenue by a country mile. Yet despite this, the NoSQL expectation-threshold is set so surprisingly low and the very definition of the term gets unnecessarily fused to the contrivances of BASE and CAP. Perhaps there’s a thinking that the super-majority defines the category or perhaps it’s more fun to write about new acronyms.

Whatever the reasons, we will continue along our path of improving our Enterprise NoSQL database (yes, with ACID transactions), rolling out Enterprise-grade installations and making our customers happy.

As for the T-shirts, we still have a bit of work to do. After all, completing our transactions checklist is so last-decade.