How Not to Fail with Generative AI

Generative AI is in the news almost daily. This technology has emerged as a powerful tool with applications across industries and use cases. While generative AI can generate impressive outputs, businesses often forget that it comes with flaws and that careful planning is required during the implantation for success.

Whenever I think about companies trying to leverage generative AI, I am reminded of an old English joke in which the answer to the question, “What’s the best way to get from here to Tipperary?” is “I wouldn’t start from here.” In my opinion that sums up the bulk of efforts to date: trying to achieve the goal without putting in the necessary foundation to make that goal achievable.

Back in 2015, I was working for IBM in the Watson Group and IBM was betting its future on “Cognitive Computing.” One of the reports we were using to support this effort was a piece of work by Gartner from 2014 titled, “Smart Computing and Analytics Rejuvenate the Enterprise Search Market” which showed a progression from basic search to the brave new world of cognitive assistants. What follows is a simplified view of this evolution.

Figure 1: Technologies for increasing business value and semantic outcomes, enabling Cognitive Assistants

Unfortunately, IBM had not read the report closely enough because while semantic search was a key step in that progression, they deliberately tried to circumvent it. Watson required large-scale human intervention, with workers manually converting Wikipedia articles into triples that could be fed directly into the engine—ignoring years of research in natural language processing and knowledge organization. Not surprisingly, when IBM tried to apply this to business problems, it failed. After months of training, it produced less impressive results than the search-driven chatbots I had been selling at Verity in 2004.

IBM’s problem was the same as that faced by organisations trying to exploit large language models today. In fact, it was the same as that faced by our fabled traveler to Tipperary: they started from the wrong place.

To progress on the path to better semantic outcomes and higher business value, you need to start by establishing the necessary foundations:

1. To leverage all the knowledge in your organization, you need to enable access to the outputs of that knowledge which—in most cases—is unstructured data. This requires enterprise search. With enterprise semantic search your organization can derive more value from its siloed information across disparate systems and reveal new insights.
2. To support semantic search, you need to be able to define the concepts that are meaningful to your organization and the relationships between them. This knowledge model allows the meaning of documents and the outputs of text analytics to be understood across your organization. Semantic knowledge models enable context and meaning for enterprises at scale and form the underpinning of every enterprise knowledge graph. 
3. A knowledge graph is the application of the knowledge model(s) to the data. Enterprise knowledge graphs enable organizations to represent their own subject matter. They systematically connect knowledge sources of different types and make it easy to define different business concepts.

Figure 2: Enterprise Search, Knowledge Models and Knowledge Graphs underpinning the technologies driving business value and semantic outcomes.

Despite having respectable offerings in the space, IBM failed with Watson because they tried to jump straight to Cognitive Computing without thinking about how search and text analytics could support it. Even if they had factored them in, they still would have fallen short without considering the management of knowledge necessary for implementing semantic search and deducing intent.

Organisations trying to jump straight to generative AI will fail twice. The first failure will be to assume that a vector database can replace enterprise search and organized knowledge models; it can’t. They will spend a lot of money implementing another repository that will only serve the LLM (and only that LLM) but will need to continue maintaining the file system and content management systems currently in use to run the business. Should they get past that challenge, they will find that the system cannot take advantage of the institutional knowledge that has been built up over the years. They will be in the same situation as IBM Watson in 2014, and they, too, will fail.

The coda to the IBM story is that after a year of trying I couldn’t get past the need for semantic search and knowledge models, so I left for Smartlogic and have been here (through the acquisition by MarkLogic in 2021 and Progress in 2023) ever since. Our solution for Retrieval Augmented Generation (RAG) combines search, knowledge models ad knowledge graphs. Finally, we have a solution that is built from the ground up, and with strong foundations, should give organisations a platform they can rely on.

To discover what Progress Semaphore can do for your organization, visit our website or contact us directly.

Explore Semaphore