Streamlining Algorithms for Artificial Intelligence

In part 1 of this series, we discussed challenges the Department of Defense (DoD) is up against when dealing with performing algorithmic tasks around different data. In this post, we’ll discuss how we can streamline this process with a solution built using MarkLogic.

Today’s automation techniques are difficult to implement and often result in delayed deployment. This is largely because each new algorithm needs to be implemented with a complex technology stack including security, data access, integration with downstream systems, and operational infrastructure, all while addressing cross-domain challenges. At scale, this model is neither cost-effective nor sustainable.

A new means is desperately needed to operationalize artificial intelligence (AI) across mission capabilities to realize the expected cost and time savings. To solve these challenges, sharing the same data integration and operational infrastructure across automation workloads eliminates the need for single-purpose systems. Fielding new sensors and algorithms becomes easier by sharing foundation intelligence access to secure data sources and leveraging the same dissemination tools and integration with downstream systems.

This shared infrastructure approach (see Figure 1 below) is highly effective across a portfolio of capabilities. Let’s say you are tasked to identify vehicles in an object detection example, new objects (vehicles) are discovered by multiple, separate AI algorithms running on the same platform. This AI can also be seamlessly used to detect other objects and perform specialized tasks such as object identification (identifying what type of vehicle it is) or vehicle track management.

To dive a little deeper, multiple AI algorithms can be used together to cooperate on the same mission. For example, using two different sensors, one infrared the other visible satellite imagery (VIS), two specialized algorithms could collaborate to track the same object during the day and night. Each algorithm would access the MarkLogic database, and the product of the two would create a clear picture to be displayed via transactional applications, analytical tools or downstream systems all via APIs.

Benefits of an Automation Infrastructure

As an enabler for machine learning and AI systems (ML/AI systems), MarkLogic is an ideal database for curating quality data. MarkLogic’s flexible, multi-model approach is perfect for integrating and storing the rich entities that ML/AI systems needs from various data silos, along with all of the contextual metadata. MarkLogic’s sophisticated indexes help the developers and data scientists that are curating the data find what they need quickly and precisely, and then send that data to the ML/AI system. Also, as the most secure NoSQL database, MarkLogic has all of the data governance required to understand and control what data is used for ML/AI.

Consider the example of how MarkLogic might improve a customer service experience using AI. Imagine that MarkLogic is storing data about customers, products, and orders. The key data about a customer entity is all represented as a single document, and audio clips from customer support calls are also stored in MarkLogic. But, the audio clip alone is not that valuable. So, MarkLogic communicates with a third-party cognitive service AI system to transcribe the support call audio and communicate the results back to MarkLogic using industry standard APIs. The results are then stored, indexed, and can be searched in MarkLogic. Then, it becomes easy when the business wants to closely analyze their support calls or help a customer service agent more quickly address a customer need. Using MarkLogic to develop an automation infrastructure able to support multiple sources and algorithms provides the following benefits:

• Improved Tradecraft – The ability to quickly add new algorithms to build recognition patterns and create new intelligence improves flexibility and knowledge management for industry and academia. As new products are identified and created via automation, they become added to the mass of information, creating a smarter data set.
• Mission Focus – Analysts will have more time to focus their efforts on advanced analytics, providing improved situational awareness to the warfighter in a timelier manner. Secure dissemination of these new products, objects and activities can be performed with unified alerting and tasking based on areas of responsibility (AORs) and areas of interest (AOIs).
• Increased Efficiency – The most current automation capabilities will consistently be available via processing, exploitation, and dissemination (PED) tools, which in turn increases analysts’ productivity. Operationalizing AI across mission capabilities on one platform, with a single point of security and access logs drastically reduces cost and overhead, and maximizes collaboration opportunities across a portfolio of specialized sensors and algorithms. MarkLogic can be deployed from the cloud to disconnected, intermittent and latent (DIL) environments ensuring users are always connected.

Proven Technology

MarkLogic’s proven platform is currently in production on various DoD and Intelligence Community (IC) programs providing the main capabilities described above. MarkLogic supports all source data as is – allowing faster ingestion and easier data manageability. Finally, a database that lets you manage change and innovate as new agency needs arise, while staying operational and secure.