Prompt Iteration and Evaluation

Sometimes, if we write a complete prompt for Llama 3 using a combination of different prompt engineering techniques with appropriate values of prompt engineering parameters, Llama 3 is still unable to generate the required response. This happens because of differences in the underlying training of the model. Each LLM is trained differently to respond to the query. Sometimes, we get our required response in just one query, and sometimes, we need to prompt the model iteratively to reach the point where we get the desired output.

Iterative prompt engineering

Writing a prompt that gives the desired result with accuracy is a challenge in prompt engineering. Iterative prompt engineering is the solution. Iterative prompt engineering is a process of writing and refining the prompt multiple times until the desired response is generated.

The following steps are involved in iterative prompt engineering:

1. Writing an initial prompt: The first step is to clearly define the task we want the model to perform. After identifying the task, write an initial prompt with clear and specific instructions using elements of prompts.

2. Prompting the model: Now, prompt the model with the initial prompt and see how the model responds.

3. Analyzing the response: Analyze the model response in detail and observe where it lacks our objective. Once we identify the area for improvement, we are ready for refinement.

4. Refining the prompt: Refine the prompt to address the identified shortcomings and prompt the model. This process is repeated unless we achieve an accurate and relevant response that meets our requirements.