Table of Contents

Motivation

Effective prompt engineering is a crucial skill for leveraging the full potential of large language models (LLMs). By crafting well-structured prompts, users can significantly improve the quality and relevance of LLM outputs. This guide draws from a variety of sources to present best practices and advanced techniques in prompt engineering.

The Basics of Prompting

Prompting involves providing a structured input sequence to an LLM, which then generates a response based on this input. The essence of successful prompting lies in understanding the target application, formulating clear instructions, and selecting the appropriate input formats.

Best Practices for Effective Prompting

1. Model Selection: Always opt for up-to-date, feature-rich models to ensure improved performance. Check recent benchmarks to see what models are performing best in your target domain.
2. Prompt Clarity: Start with concise and straightforward prompts and refine them over time. Instructions should be placed at the beginning or end of prompts to exploit LLMs' attention biases.
3. Specificity and Detail: Ensure prompts are explicit about the desired outcomes, including aspects like format, length, style, and language. Avoid vague or ambiguous phrasing.
4. Structured Formatting: Clearly distinguish instructions from the provided context or text to enhance the model's understanding and output relevance.
5. Use of Demonstrations and Instructions: For complex tasks, consider incorporating few-shot learning or Chain-of-Thought (CoT) prompting to guide the LLM toward the desired behavior.
6. Define Strict Output Formats: It’s generally best to have the model output into a predefined output format. Whether that’s as simple as {”answer”: “insert chatbot answer”} or as complex as a nested JSON structure, it’s good to let the LLM know what you expect to ensure the final format is understood by the rest of your system.

Tips and Tricks

Enhancing the quality of responses from large language models (LLMs) requires more than just following best practices. Here are several effective tips and tricks, gleaned from various expert sources, to help you elicit better, more accurate, and more creative responses from LLMs.

Incorporate Examples

<aside> 📣 Demonstrate Desired Output Including examples within your prompt can guide the LLM towards the type of response you're seeking. This is particularly useful for tasks with complex requirements or when seeking creative content.

</aside>

Optimize Prompt Structure

<aside> 📍 Strategic Placement Place critical instructions at the beginning or end of your prompt to leverage the LLM's attention biases towards these areas.

</aside>

<aside> 🧱 Break Down Complex Tasks For complicated queries, breaking down the task into smaller, manageable steps can help the LLM process each component more effectively. This is a key aspect of Chain-of-Thought (CoT) prompting.

</aside>

Leverage Model Variants

<aside> 🧩 Choose the Right Model Variant Utilize the "base" or "instruct/chat" variants of models appropriately based on your task. "Instruct/chat" models are fine-tuned to better handle instructional prompts and conversational contexts.

</aside>

Iterative Refinement

<aside> 💬 Prompt Tuning If the initial response doesn't meet your expectations, refine and adjust your prompt based on the output. This iterative process can help you home in on the most effective way to communicate your requirements to the LLM.

</aside>

Avoid Negative Instructions

<aside> 🔎 Focus on What to Do Instead of telling the model what not to do, specify what it should do. This approach is more direct and reduces the chance of generating undesired content.

However, do note that if your model consistently exhibits a certain unwanted behavior, adding a “Rules” section to your prompt that explicitly forbids such behaviors has been known to work.

</aside>

Advanced Techniques

<aside> 📚 Few-Shot Prompting This involves integrating exemplar inputs and outputs within the prompt, helping the model recognize patterns and generate similar outputs for new inputs.

</aside>

<aside> 🔗 Chain-of-Thought (CoT) Prompting CoT prompts encourage LLMs to unfold intermediary reasoning steps, thus improving the quality of responses for complex reasoning tasks.

</aside>

<aside> 🔢 Hyperparameter Adjustment These are the model parameters that affect the output outside of the prompts themselves

<aside> 🌡️ Temperature This parameter controls the randomness of responses. A higher temperature might yield more creative outputs, while a lower temperature results in more predictable and conservative responses. Adjust this setting based on your needs for creativity versus accuracy.

</aside>

</aside>

What about Retrieval Augmented Generation and Fine-Tuning?

The general flow of building an LLM application is this:

1. Create a prompt to represent your task
2. Optimize the prompt with the best practices outlined in this article
3. Consider if your task would benefit from examples or external information. If only a few examples/supplemental facts are needed, you can put them directly into the prompt. If the best examples and docs to use vary based on the user’s input, consider Retrieval Augmented Generation (RAG)
   1. While RAG can be helpful out of the box, these pipelines really shine when they are optimized to your use case. There are many many knobs and levers to pull here. Our guide on this is coming soon!
4. Finally, if the above steps are not enough to reach your desired accuracy, consider Fine Tuning your LLM. This involves creating a task specific dataset of inputs and desired outputs so the LLM can get a better idea of how it is supposed to respond in certain cases.
   1. We want to perform this task last, as updating the prompting or retrieval can sometimes “undo” some of the progress you make with fine tuning, depending on how far you depart from the original prompt or retrieval formulation.
   2. This option can be expensive, so it can be good to confirm you’ve done the best you can with the first steps before doing this!
   3. Not all models are available for fine tuning. At the time of writing, GPT-4 fine tuning is in limited access alpha, for instance.

Remember Prompt Engineering is Always First. Don’t jump the gun and start providing supporting examples or fine tuning on a task that isn’t even well defined! Often, as you add in RAG pipelines you will have to go back and adjust your prompting, but always make sure your task is clearly defined in your prompting approach before introducing complicating factors.

Conclusion

Mastering prompt engineering is essential for unlocking the capabilities of LLMs across various applications. By adhering to the outlined best practices and engaging with the community for shared learning and collaboration, practitioners can enhance their prompt engineering skills. The field of prompt engineering continues to evolve, with ongoing research and development offering new insights and techniques for effective interaction with LLMs.

If you’re looking for a deeper dive into Prompt Engineering, reach out to us over here at PressW!