Table of Contents

Understanding Large Language Models (LLMs)

Imagine trying to create a recipe from all the cooking shows you've ever watched. Your brain sifts through everything you've learned, combining bits of information to help you make a dish. Large Language Models (LLMs) work similarly, but in the world of words and information.

An LLM is a type of artificial intelligence that reads and absorbs vast amounts of text from the internet, books, articles—virtually anywhere you find words. It's like an incredibly well-read friend who's digested more books and sites than you could in a lifetime.

But what does an LLM do with all this information? It learns the art of conversation, writing, and even understanding human language. When you ask an LLM a question or seek its assistance in writing, it isn't pulling responses from thin air. Instead, it recalls all the text it's studied, understanding the context and the way language is used, to provide a response that makes sense.

So, LLMs are not creators of new ideas but rather expert communicators. They synthesize everything they've "read" and assist us in various tasks—from writing emails, generating reports, to answering complex questions, making our digital communications smoother and more efficient.

<aside> 🤖 That breakdown was written by an LLM, ChatGPT 4.

</aside>

In shorter and simpler terms

At its base, a language model is a program that uses a probability-based algorithm to decide what words it should use to reply to its input (often called a “prompt”), based on its previous training.

A large language model, or LLM for short, is a powerful language model that has been trained on a massive amount of data in order to accurately understand any input and respond accordingly.

<aside> 🗒️ How does a language model use probability to decide how it should respond to our messages? By training itself on example data, which can resemble anything from an article to someone’s social media message history, language models start to see a trend in what words follow others. Thus, when a language model takes in our messages, it calculates what words it would be most likely to see as a response based on its previous experience, and gives us that response. The fact that LLMs can respond to just about anything is a testament to just how much data they’ve trained on - no matter how uncommon your question might be, they will generally have a pretty good guess as to how to respond.

</aside>

Why are LLMs such a big deal?

Large Language Models are incredibly powerful

If you ask ChatGPT what information it was trained on in order to be able to converse with humans, it won’t hesitate to tell you: in its training process, ChatGPT has taken in information and human interactions from all kinds of books, articles, papers, and websites. It also has the bandwidth to remember a large portion of that data - since ChatGPT does not have a live connection to the internet, all the questions it is able to answer are from its own training. Whenever we ask an LLM to do something, it is able to use the unimaginable amount of data at its disposal to decide exactly how to respond in a way that helps us the most. No other tool in human history has been able to aggregate and make use of this much information in a way that can be utilized as easily as sending a message.

Large Language Models are versatile

Because LLMs are trained on all types of data, it is also much more flexible than any other machine learning program on the market. LLMs are capable of responding to our requests in any way we choose: we can tell one to give us only one-word answers, or structure its output in a table format, or even output code. This allows us to incorporate LLMs into our code and use their flexibility to our advantage, so that a program that would take an intensive amount of computer logic or formatting can instead be tackled as a no-code solution by requesting an LLM to process it. An LLM also takes a much more human approach to problems that we give it, which makes the resulting program much more robust - an oddly written user input or even a malicious attempt to break normal code with a malformed input doesn’t pose a problem to an LLM.

Large Language Models are only getting better

As LLMs have become the new darling of the tech industry, the race to make them even stronger and more accessible has continued over the last few years. This means as time goes on, the LLMs that are being used in our programs will get even better at understanding what we want them to do and performing their work in a fast, cheap way.

The Pros and Cons of Large Language Models

What Large Language Models are Great at

Processing data for computer programs and databases

• While a user’s answer to even a simple question such as “What’s your name?” can vary wildly, an LLM is able to handle whatever response it gets back as well as a human would.
  • For example, if a user only gives us their first name and we need their full name, an LLM could recognize this and ask, “May I have your last name as well?”
  • If a user asks “Why do you need my name?”, we could even provide the LLM with context on why we’re asking for their name that the LLM could then give back to the user. (ie, “I’m trained to collect your name so that I can respond to you in a more polite manner and to make it easier to schedule an appointment with you later if you choose to do so!”)
• LLMs can also transform an answer to fit our needs in the code, which makes things much easier for an end user.
  • For example, if we need what year the user was born, we don’t have to constrain the user into answering in a specific format (”please write your birthday as MM/DD/YYYY”). Even an answer such as “I was born 55 years ago” would suffice for an LLM, since they have the problem solving capabilities to deduce when that would be.

Performing a wide variety of tasks, using tools we provide it as needed to do the job

• One common application of LLMs is a question and answer bot, where an LLM will go through a database for you and answer any questions you have about the data.
• A wide variety of tools are now available for the most popular LLMs, which give the LLMs the ability to do much more than just talk: for example, they can search the internet or save any important information a user provides into a database.

Taking in constructive criticism and improving its output over multiple interactions

• Most LLMs have the capability to “remember” their previous interactions with a user in a conversation, and use that information to help the user as the user gets more specific about what they need.
• While this may seem like a process that requires an active back-and-forth from a human user and an LLM, there are also many tools now that utilize this process to the fullest without any further input from the user.
  • One example that’s become increasingly popular is a summarizing tool, which will provide an LLM with something to summarize at first (a meeting, a video, an article, etc). It will then ask the LLM to continually revise its previous summary while bringing in any missing context or important information, a process that creates summaries with much more detail without making them significantly longer.

<aside> 🗒️ Are there more applications than these? Of course! These are just a few of the high-level strengths we’ve been able to put to work in our products. As we build more and more tools that attempt to utilize LLMs to their fullest, we’ll change and add to these pros and cons to give you a better sense of what they can do. If you don’t see your use case listed here, let us know and we can give you a better sense of how we’d approach it with LLMs

</aside>

What Large Language Models Struggle With

Completing complex tasks in a single call

• The powerful computing power behind LLMs is ultimately aimed at making them much more capable of understanding and responding to a user. An LLM’s “problem solving” capabilities are more of a happy coincidence than an intended feature: in its efforts to respond to our input as humanly as possible, it can give us the information we need. However, at the same time, no part of the LLM is double checking the answer it gives us - it doesn’t use a calculator if we give it a math question or fact check any question we have for it.
  • Thus, if we’re not careful with how we ask for information from an LLM, it can often give us the wrong answer.
    • Most commonly, this occurs in the form of an answer formatted in a way that’s incompatible with our code. Since we have code written to catch any answers that weren’t outputted correctly, we can re-ask the LLM the question to see if it’ll perform better the next time. If this happens too often, it’s often a sign we have to change how we’re asking the LLM for information, as too many repeat calls can significantly slow down a program.
    • Sometimes, if an LLM is really struggling, it’ll make up completely wrong answers, in which case we have to take a different approach to how we’re working with the LLM. These situations very rarely occur outside of early testing.
• However, this issue has some workarounds inside the LLM space, which means that LLM-based technologies can still do a lot of complex and amazing things!
  • The simplest one is simply upgrading the model of LLM we’re using, which comes at a small performance cost and a much larger relative monetary cost.
    • For example, ChatGPT’s current base model (Chat GPT 3.5) has a premium version called Chat GPT 4, which costs 20x more per call.
  • Alternatively, breaking up the complex problem into multiple LLM calls can greatly improve accuracy and keep the costs down, with the only major downside being the time necessary to make multiple calls.
    • Usually, some of the steps in a complex problem can also be done using code instead of LLMs. Most applications of LLMs that we’ve built to this day have used a maximum of two or three LLM calls in order to perform any one particular task.

<aside> 🤠 If you need a break from all this technical talk, click here for a particularly entertaining demonstration of ChatGPT attempting to play a word game!

</aside>

Performing creative tasks without help or examples

• Another aspect that LLMs fall short on is the creative side: since LLMs are designed to replicate what humans have written in the past, they can often struggle significantly when asked to do something that humans have never done before.

  • An LLM can easily tell you a joke that you’ve never heard before, but if you ask it to make a joke that nobody has ever told before, it’ll often give you something nonsensical or half-baked, or just tell you a joke that already exists.

  

  • While this is a good joke, a quick Google search confirms that the joke is one that’s been around a lot longer than its creator ChatGPT.

• However, LLMs can perform well at creative tasks with some initial inspiration and additional context in the form of examples, since that requires it to be less “creative”. Instead, it can use the basis of a previous example and the topic provided to fill in the gaps between, resulting in a much more satisfactory result.

  

  • While it didn’t incorporate the llama aspect too much, at least it created a new joke! If I keep working on my input, maybe ChatGPT still has a future as a comedian.

Speed

• For each individual call of an LLM, getting a response takes at least a second as of writing this. More complex calls can take even more time, sometimes up to a full ten seconds depending on if we need to give the LLM special tools to do its job.
  • While we do all we can to offload any LLM calls to code that’s not actively part of interacting with the user, many LLM calls must be done before our programs respond to the user in order to provide a satisfactory response.
• However, just like LLMs will only get more capable over time, they will also get faster - which makes technology built with them faster over time with no additional dev time.

Are LLMs safe to use in my product?

Of the concerns surrounding the LLM space, there are a few that, in a vacuum, could reasonably surface in an LLM-based product:

• Misinformation, where the LLM generates information that is factually incorrect or misleading
• Bias, where LLMs perpetuate biases present in their training data
• Inappropriate content, where the LLM generates a response that is offensive or inappropriate
• Data privacy or security risks, where the LLM is able to relay sensitive internal information

In order to ensure that these will never be a problem in our end products, we take a multi-pronged approach:

• We only use LLMs in our products that we have personally tested and experimented with to the degree that we’re comfortable using them in our products.
• We carefully engineer the ways we ask LLMs to perform work for us such that any content that does not apply to our use case (whether offensive, malicious or simply irrelevant) does not cause the LLM to respond in unexpected ways that could spread misinformation, return inappropriate content, or reveal sensitive internal data to an end user.
• We rigorously test our products with a constantly-evolving testing harness to ensure that all LLMs and their respective configurations meet our standards of safe usage.

Wrap-Up

In summary, large language models are extremely powerful tools for us as programmers. By leaning on their strengths and accounting for their weaknesses, we can build incredibly potent tools that work intuitively for both end users and other programmers who want to review or modify our products in the future. If you have any more questions about LLMs or how we use them to build products for you, please don’t be afraid to contact us!