Big Data Nation Chicago: Situational Awareness

Volume, velocity and variety of weather had hit Chicago on Wednesday but the Blackhawks had won in triple overtime and the sky was blue for us when the Big Data Tour stopped at Spiaggia Restaurant downtown on Thursday morning. It was a terrific audience — and the first one I can say that had gender parity. It was great to see the ladies get in on the Big Data scene.

John Shap kicked off the event explaining the “art of the possible,” and instantiated it by sharing stories about the BBC, Conde Nast, FAA and Mitchell1. All the guys love the Mitchell1 story, which boasts virtually every auto shop across the US and Canada as its clients. There are tens of automobile manufacturers who have manufactured many thousands of automobile models since Mitchell 1 creating service instructions in 1960. In order to service a car effectively an automobile technician needs access to the technical manuals originally produced for the manufacturer/model in addition to any maintenance manuals, and technical service bulletins (which includes recall notices). So Mitchell1 put all that paper in one searchable directory. Ben Johnson, product management over at Mitchell1 loves to share that all those manuals today would fill a warehouse.

I’m a story teller and if you sell technology — you start by telling the story. So when we brought our panel of Mark Rodgers, Dan McCreary and Fernando Mesa, I immediately asked Mark to tell the audience what the issues were when he was looking for a solution for Reed Elsevier’s LexisNexis. Lexis is part of the Reed Elsevier family and Mark runs the technology group that supports Lexis Advance. About 10 years ago, Lexis was choking on the amount of content it needed to ingest – roughly 200,000 different data sources. “Onboarding” that content source would be somewhere between 1 and 48 hours, noise when you think of a single source, more than several lifetimes otherwise. So faced with some ridiculously impossible task of moving more content sources into a single format – Lexis was thus impacted on how fast it could get new products to market, and was finding that competition was nipping at its heels. When Mark and his colleagues discovered MarkLogic and heard the promise it could ingest “any type of content as is”– he created a small proof of concept. Fast forward a decade and Lexis has ingested all 200,000 sources and is rapidly creating new products on top of MarkLogic, regaining its competitive edge.

“Making Sense of NoSQL” Author Dan McCreary listened to Mark and gave some great insights as to why a “document-centric data store” versus all the other types of NoSQL databases, have proven to be most flexible in handling different data types. Having the ability to store text, metadata, json and even relational, it means that querying across all these data types is simplified, explained MarkLogic Enterprise CTO Fernando Mesa. I mentioned that Brian Bishop from Springer once said to me that if you can create a query — you can sell it. And all three agreed in unison. Queries are the basis of any application — of any report. And reverse queries or stored queries — are sitting waiting for information to come in and match it. We also call those alerts.

So with a dynamic conversation with these 3 guys, we get the crowd going and what-iffing. It’s my favorite part; we have managed to tickle the imagination and now they are throwing out for the crowd to embellish: well I have traffic lights with sensors and maps of roads, and i know a storm is coming in — can i create a situational awareness dashboard that will tell me … 

And yes, we can help you do that. It’s the Art of the Possible.

Duel appearing in Ft Huachuca, AZ and Boston tomorrow.