Prompting for Large Language Models

Now that Large Language Models (LLMs) like ChatGPT have gotten so popular, a question that arises is how to ensure that they understand our requests correctly. Considering that LLMs have no innate understanding of text, but instead are based on statistical associations between words, this question is especially important. Choosing the right words, providing the right amount of context, and emphasizing the right part of the prompt is crucial. This is what's called 'prompt engineering'. In the following sections, we provide some tips on how to better engineer your prompts to LLMs. Of course, what makes a prompt 'good' depends on the task, and on the model - these are some basic concepts. If you're looking for tips on how to prompt image generators, see here.

prompt engineering noun COMPUTING:

The process of designing prompts (= instructions giving to an artificial intelligence by a human using natural language rather than computer language) that will give the best possible results or answers.

Basic Tips

Length and Precision

To start, two terms are important: tokens and context length. Tokens are the individual words or punctuation marks a LLM breaks text up into, in order to understand it in its statistical way. Context length is the amount of tokens a LLM can remember at once. (For example, if a LLM's context length is 15.000, it can remember 15.000 words and punctuation marks before it begins to forget some.) Often this context length includes both input and output. Because of this, it is important to keep prompts efficient to keep up as little of the context length as possible; make them too long, and the quality of the output diminishes. Writing efficient prompts might also make the model run faster. However, make sure to not make your prompts 'too' efficient, since important information that makes the model understand your prompt better might be missing.

Besides modifying the input length, it is also possible to modify the length of a LLM's output. Since LLMs are poor at mathematics and counting, specifying a certain text length (e.g. a word count) might not produce the result you want. Instead, you could try using words like 'concisely' or 'extensively', or to provide a 'Top 10' of answers.

Clarity

Because LLMs don't have an inherent understanding of meaning, it might be helpful to use more specific words, since they can less easily be misinterpreted. For example, you might ask a model about 'domestic dogs' instead of just 'dogs' if you're only interested in the pet kind. In addition, the output will be improved by outlining what type of response you're looking for.

Vague prompt: “Tell me about dogs.”

Clear prompt: “Provide a detailed description of the characteristics, behaviour, and care required for domestic dogs.”

Context

Besides specifying the output wanted, it can also be helpful to clarify the input provided.

Vague prompt: “Translate the following text: ‘Hello, how are you?’”

Clear prompt: “Translate the following text from English to French: ‘Hello, how are you?’”

In the first example above, the desired output is clear: we want a translation of the sentence provided. However, context is missing on the language the sentence is in, and what language we want to translate it to.

This contextualising of input becomes especially important now that LLMs are developing and are gaining access to more information sources. For example, if we upload an excel file and want a model to analyse some information from it, it is important that it understands the file's content (e.g., what do the columns represent, what are the units).

Role-play and audiences

One tip we give to responsibly use a LLM is to use it as a study buddy. These models are quite good at modifying the style of their output to match a certain role they're given.

Example: “You are a friendly and helpful tutor in the field of genetics at a university. Can you explain the process of mitosis to me?”  

Besides giving the model a role, you could modify its output by specifying a target audience.

Example: “Explain the concept of circular agriculture, using words a 12-year old can understand.”

Highlighting specific information

There are two ways to ensure a greater weight is given to the most important information or instructions from your prompt. You could use capital letters ('tell me about DOMESTIC dogs'), or pay attention to the order of your paragraph. As humans we pay most attention to the first and last sentences, thus a LLM tends to do the same. Therefore, it might help to repeat the most important instructions on e.g. text length at both the start and the end of the prompt.

Using custom instructions

In ChatGPT, you can set specific instructions for your account. You can provide information about you as a user, and about the tone and role of ChatGPT in its output. This can help improve the context and style of responses.

Advanced Tips

Providing examples

Sometimes it is helpful to provide a model with some extra 'training data' by giving an example of the type of output you expect. Providing one example is called 'one-shot prompting'. In 'few-shot prompting', you provide multiple examples.

Example prompt: “The slogan for McDonalds is ‘I’m Loving It’. What could be a slogan for Wageningen University?”  

Chain-of-thought prompting

LLMs are often criticised for their lack of reasoning capacity, e.g. in logic puzzles. One way of increasing your chances of getting an accurate output is by asking about their chain of thought. For example, you could add a sentence like “Explain your reasoning step-by-step”.

Reflection

A final technique you could apply is reflection. If, after providing clear and precise instructions, the output is not what you expect, you might ask the model to evaluate its own output based on the original prompt given. Adding a sentence like “Could you verify that your output matches all the criteria from the previous prompt, and correct the output if it is found not to be in agreement?” could improve the output's accuracy.

The text of this page is based largely on a document written by Tijmen Kerstens.