Share: LinkedIn ↗
Large Language Models (LLMs), by themselves, are powerful tools. Recently, they have been used to great effect in building custom chatbot services and creating intelligent enterprise search over all data sources in your company.
However, in order to harness that power we must build infrastructure around them to support their capabilities. In fact, this is often much of the code that gets written to support custom LLM based applications. In this article, we’ll discuss some of the more pivotal aspects of system architecture that make LLM applications feasible.
Modern LLM Application Flow (double click to zoom)
As you can see, the code that actually interacts with the language model is just one part of what makes an LLM application come to life. The lower branch of this diagram details the main flow of the LLM invocation, which generates and optimizes a prompt for your use case, and then inspects the output and applies any data processing and business logic relevant to your application. The upper branch of the chart details the “retrieval” process, which is present in all but the most basic LLM applications. This process supplies the language model with the ability to guide its responses by selecting relevant excerpts from your specialized knowledge base. By combining all of these pieces together, we can create a chatbot that can interact with a user in a natural way, while:
The user interface is the front-end component of an LLM application where users interact with the system. It should be designed to both capture user input clearly and present system outputs in an understandable manner. For businesses, the UI could be a simple text box on a website, a chatbot integrated into a customer service platform, or even a voice-activated interface. The key is to make user interaction intuitive and seamless, reducing the complexity of what happens behind the scenes to simple commands and responses that feel natural to the user.
At PressW, NextJS is our frontend technology of choice (as well as the current industry standard). It provides a lightweight and flexible way to build highly responsive LLM interfaces with a high degree of customizability and a modern user experience.
Data orchestration in an LLM system refers to the processes and systems responsible for managing data flow and integration. This involves sourcing data from various inputs such as company databases, marketing materials, and developer documentation. We then must process that data and channel it to the right components at the right time. Efficient data orchestration ensures that the LLM has access to the necessary data in a format that it can use, which can include pre-processing steps like tokenization, anonymization, and formatting to fit the LLM's needs. This component is critical for performance and accuracy, ensuring that data is current and synchronized across the system.
For your LLM project, we pick from a variety of staple technologies to facilitate data orchestration based on the desired system functionality and constraints. Often, it’s necessary to choose a Vector Database in order to facilitate fast document retrieval based on “semantic search.” These vector databases are used in any application where it’s necessary to provide the LLM with external supporting documentation to help it make decisions and answer questions, and are the backbone of intelligent enterprise search. Further, we leverage a variety of more traditional data storage solutions, such as Relational Databases to power functionality such as output storage, result cacheing, integration with existing production systems, and more.