Using MarkLogic for Cyber Threat Analytics and Awareness

Cybersecurity and enterprise IT leaders know data is critical to an effective cybersecurity operation. But how effectively are they leveraging their data?

Large government and commercial organizations often think of cyber data simply in terms of managing event logs: combing through large volumes of event logs from potentially tens of thousands of network nodes to gain visibility into potential configuration, hardware, or security issues.

But a modern, data-driven approach to cybersecurity relies on more than just storing and analyzing machine logs and responding to hardware notifications. It involves:

• collecting the many varieties of data that are generated by a wide array of hardware, software, and network products across the IT enterprise
• centralizing and integrating that data to gain a full-dimensional view of what is happening
• acting on that aggregated data in an automated, real-time way so it serves as a proactive participant in your cyber defense operation

Many products perform specific cybersecurity functions, but many typically don’t work well together to promote a comprehensive, unified cybersecurity operation that fully leverages the potential of the data being generated.

The best way to pull all this information together and use it effectively is to ingest it into a highly versatile, highly secure, centralized data platform that can support any data format at scale to power extremely fast, advanced analytics and alerting operations that defend against both internal and external cyber threats.

MarkLogic Advances Critical Data Functions for Cybersecurity

MarkLogic is a highly secure, multi-model, agile data platform that quickly and easily connects many disparate data sources to provide a single, consistent, composite view of something — whether it’s a person, place, event, or thing — for the entire enterprise. It then enables people or applications across the enterprise to query that information in real time to suit their specific data access needs while also abiding by relevant access and security protocols.

For modern cybersecurity operations, a data platform should perform three critical functions, and MarkLogic advances all of them with industry-leading capabilities.

Making Data Available

To effectively leverage enterprise IT data, it must be centralized, reliable, cohesive, and quickly accessible — regardless of its format, source, or volume — so that the complex dynamics of a large-scale network can be well understood, and vulnerabilities and threat actors are apparent and addressable as they arise. Data availability also requires high security and the ability to transcend organizational or technological silos that can impede needed information-sharing. Finally, data must also be curated through effective metadata management and creation so cyber teams are not overwhelmed by too much data or by the alerts and warnings that data generates.

Individual cyber defense tools can have great capabilities and generate large volumes of log information, but they typically don’t accommodate the size and scale of large government enterprises and their abundant varieties of data. This is a core strength of MarkLogic, which can amass any and all data from a large-scale IT enterprise and make it immediately usable, through responsive search and analytics, for cyber defense operations. While large commercial networks, such as those of banks and retail brands, may have thousands of nodes and end users on their networks, at federal agencies, network nodes typically number in the tens of thousands — and MarkLogic comfortably accommodates the volumes and varieties of data those nodes and end users generate.

In addition, MarkLogic uniquely manages all varieties of metadata — passive, semantic, and active metadata — and applies robust semantic AI capabilities to enrich data with its full value and context so it can be fully leveraged in operations across the enterprise.

Using Enterprise IT Data and Active Analytics

Once IT data is made available, it can be used to create automated alerts and warnings of suspicious internal and external activity that prompts automated or human responses. Machine learning applications can be used to model data, link data, construct baselines, and spot anomalies. And, importantly, that data can be analyzed to quickly understand the nature and extent of an attack once it has commenced: Is the attack internal or external?How was access achieved?What is affected and how can guardrails be adjusted to mitigate the damage and protect the enterprise?

Insider threats are particularly critical for many large federal government and commercial organizations. Data such as login tracking, user search behavior, and data retrieval, can be baselined and converted into early warning alerts to spot potential breaches.

Because MarkLogic centralizes more types and greater volumes of enterprise IT data, these alerts, warnings, and analytics can respond to a much wider set of potential scenarios. Moreover, MarkLogic’s robust interactive abilities to manage metadata and search data means it can find needles of critical information within vast haystacks of data in seconds — instead of hours or days — so cyber teams can conduct analytics on the fly as situations demand.

Conducting Threat Analysis

Data is also critical for incident response activities and post-incident analysis. These are critical functions that can mitigate threats and improve understanding of attacks after they have occurred to prevent similar attacks or mitigate damage. Analytics enable large organizations to respond to attacks with more localized or nuanced countermeasures — as opposed to broader responses, such as shutting down an entire network, that could unnecessarily sideline mission-critical systems.

MarkLogic’s ability to ingest and manage exceptionally large volumes and wide varieties of data at lightning speeds allows cyber teams to develop a granular understanding of attacks to mitigate their effects in real time and install early warnings for similar attacks in the future. This means that MarkLogic is not subject to the types of data caps that often limit the amount of data other solutions can ingest and manage — allowing the platform to reproduce a full picture of what happened for maximum benefit. Having a complete data picture of a cyber incident is particularly critical when employing machine learning algorithms as part of a cyber defense operation.

Moreover, MarkLogic is a multi-model database that does not require organizations to perform ETL (extract, transform, and load) when ingesting data. This means an organization can employ data it ingests immediately for true machine-speed automation and analysis. Also, MarkLogic’s ability to robustly manage metadata means it can be programmed so that access to data is tightly controlled based on an individual’s job, security clearance, or role, thus mitigating inside threats and building in countermeasures. For example, MarkLogic administrators can do their job without having to access data.

A Unified Data Platform to Solve the Most Difficult Cyber Challenges

MarkLogic employs industry-leading innovations to bring disparate data together, enrich that data with value, and present government agencies with a comprehensive, actionable data picture of their cyber environments — all within the most robust security framework possible. In this way, MarkLogic helps agency decision-makers connect to their IT data quickly, make better decisions, become more agile, and solve their most difficult cyber challenges.

To learn more about MarkLogic capabilities, visit the MarkLogic data platform page.