Generative AI will be on every organization’s digital transformation roadmap. The astounding capabilities of ChatGPT have been grabbing all of the headlines, but the question now is, how can these powerful tools be used to optimize the customer or employee experience?
The OpenAI website (the company behind ChatGPT) features a case study from Morgan Stanley[1]. The headline is “Morgan Stanley wealth management deploys GPT-4 to organize its vast knowledge base.” The case study quotes Jeff McMillan, Head of Analytics, Data & Innovation, saying that “the model will power an internal facing chatbot that performs a comprehensive search of wealth management content and “effectively unlocks the cumulative knowledge of Morgan Stanley Wealth Management.”
McMillan is further quoted as saying “You essentially have the knowledge of the most knowledgeable person in Wealth Management–instantly… Think of it as having our Chief Investment Strategist, Chief Global Economist, Global Equities Strategist, and every other analyst around the globe on call for every advisor, every day. We believe that is a transformative capability for our company.”
This is the holy grail of knowledge management — the ability to embody the organization’s knowledge and expertise in the systems, processes, and tools that customers interact with.
Are we really there? Is Generative AI the answer to knowledge access, retrieval, and application? Before claiming victory over information chaos, it is important to consider a few foundational elements and caveats.
First, there is an assumption underlying the perception that generative AI can solve knowledge management challenges that the knowledge exists in explicit, documented form[2]. In most organizations, however, knowledge is locked in people’s heads, and, if is stored in digital form, it is fragmented in silos across the ecosystem of departments, technologies, and repositories. The Openai.com site further states that Morgan Stanley “publishes thousands of papers annually covering insights on capital markets, asset classes, industry analysis, and economic regions around the globe…This wealth of knowledge creates a unique internal content repository for Morgan Stanley to process and parse using GPT-4 while also being subject to the firm’s internal controls.” Morgan Stanley has knowledge that can form the basis for questions using the LLM of ChatGPT. If the enterprise content and knowledge resources are not available for ingestion, are of poor quality, or are not aligned with customer and employee needs, ChatGPT will not be able to access organization-specific knowledge that responds to those needs.
Second, Generative AI creates content; it is not a retrieval mechanism. So how is the original knowledge base being used? This is a tricky area. ChatGPT is looking for patterns in content and concept relationships so that it can predict what text should be presented based on a prompt. A prompt is a signal, just as a search term is a signal. A search engine predicts what information should be presented based on not only the term, but also other signals related to the context of the query, for example, the industry, or the role of the searcher. It is possible to provide context to ChatGPT in the form of facts or documents within the prompt, or programmatically by pointing to specific information as the basis for responses.
A large language model (LLM) is a mathematical representation of terms, concepts, and relationships that are contained in a body of information. The power of LLMs lies in their ability to understand the user’s intent — what the user is looking for, regardless of how the request is phrased — and in predicting the patterns of words that have the greatest probability of occurring in response to the user’s intent. The model “understands” what the user is asking for and makes a prediction regarding what should be returned. Search engines also make a prediction based on a user’s query, albeit by a different mechanism. Search engines can be used for retrieval in a generative AI context. The content is retrieved using a semantic search or neural search engine and the response is formatted for the user using the LLM.
A thesaurus maps non-preferred terms to preferred terms (for example, “SOW” and “Statement of Work” are mapped to “proposal,” the preferred term that documents are tagged with). Think of one aspect of an LLM as a “thesaurus on steroids,” but for phrases and concepts, not just words. Users can ask questions in many different ways that resolve to the same intent. This intent classification is not new and is the basis for chatbots that resolve phrase variations to specific actions. Language models are the foundation for intent resolution and classification functionality.
The LLM also understands the patterns of words that should follow a prompt. This is how the conversational fluency of ChatGPT is enabled. The key to making them practical for the enterprise is to tune the model based on a specific body of content or knowledge,(which is what Morgan Stanley is doing with its implementation of ChatGPT and to ingest the terminology that may be unique to the organization.
Many tutorials with example code are available that illustrate how to use LLMs with specific content. For example, this video walks developers through the process of using a language model such as GPT-4 and pointing the chatbot to specific knowledge and content.
After reviewing a number of these tutorials, I have a few observations:
Custom, knowledge-specific chatbots can use large language models to understand what the user is asking and then return results from a designated knowledge source. Developers point to the need to “chunk” content into “semantically meaningful” pieces. Componentized content designed to answer specific questions needs to be complete and in context. It is important to note that knowledge does not generally exist in this state. To be componentized, large documents and bodies of text have to be broken down into chunks. A user manual, for example, could be broken into components by chapter, section, paragraph, and sentence. In the tech docs world, this is already done — a standard such as DITA (Darwin Information Typing Architecture) uses a topic-based approach, which is perfect for answering questions!
Developers talk about “semantics” and how semantics are important. What does this mean? Semantics is about meaning. Semantically enriched content is tagged with metadata that helps to precisely retrieve needed information and — again — the context of the information. For example, if a user has a particular model of router and that router throws an error code, content that is tagged with those identifiers can be retrieved when a support bot is asked for help. This process is also referred to as “slotting” in the chatbot world.
Custom content is ingested into what is referred to as a “vector space” — another mathematical model of information that places documents in multi-dimensional space (again, a mathematical construct) that allows for similar documents to be clustered and retrieved. This is referred to as an “embedding.” Embeddings can contain metadata and identifiers such as reference sources that are useful for documenting why a particular answer was presented to the user. This is important for legal liability and regulatory purposes as well as providing assurance to users that the correct, most authoritative information has been presented.
There are a couple of perspectives around “training.” We hear that ChatGPT and LLMs are trained on vast amounts of content in a way that allows them to understand user queries and respond to them with the best answers which are well-formed and conversational. One way to train the tool is to include content in the prompt. “Answer this question based on the following information…” There are two problematic issues here.
The first is that ChatGPT can only handle a certain amount of content in its prompt, and this way of asking questions would be very limited. It is possible to ingest content into the tool, which will enable additional training. However, adding content into ChatGPT also incorporates that content into the public model. Therefore, company IP will be compromised. This risk has led many organizations to ban ChatGPT and others that have lost IP through the inadvertent uploading of company secrets.
But there is another way to train on content. The LLM can use company-specific knowledge as part of a training corpus, but this would require a version that is behind a firewall. Fortunately, LLMs are rapidly becoming commoditized, and some can even be run locally on a laptop. This type of training can also be computationally expensive. Another mechanism is to use the LLM to interpret the user’s objective (their intent) and then programmatically provide context from a specific data or content source using vector embeddings, as explained above.
The language model then processes and formats the response to be conversational and complete. In this way, the knowledge stays separate from the LLM, and trade secrets and IP are not compromised.
All of these factors point to the need for knowledge management and knowledge architecture that structures information into components so users can get answers to specific questions. The revolutionary nature of LLMs and ChatGPT can provide the conversational fluency needed to support a positive customer experience with an almost human level of interaction. The critical element is access to well-structured knowledge in the organization. ChatGPT seems like magic, but it is based on the statistical processing of information and pattern prediction. With the correct structuring and integration of information, it will be an essential component of enterprise digital transformation.
