Generating a List of Words

Behind the scenes

Generating text might seem like magic at first, but really, it’s just clever statistics. LLMs are trained on massive amounts of text. When given a prompt, these models generate the most probable sequence of words based on the prompt and already generated sequence of words. It then uses that predicted word to inform the next guess, and so on, building a sentence one word at a time. It’s like having a super-powered autocomplete that can take your ideas and turn them into full-fledged conversations.

Most LLMs also have attention mechanisms that allow them to focus on specific parts of the input text that are most relevant to the current generated word. Attention works by assigning weights to different parts of the input. These weights indicate how important each part is for predicting the next word in the sequence. Let’s explore this with an example. Take the word “chip” in the two sentences:

• That’s a big bag of chips.

• This is the fastest chip on the market.

In the first sentence, we are talking about potato chips. This can be inferred by the use of the words “big bag,” as potato chips are usually sold in bags. In the second sentence, we are discussing computer chips, which can be inferred by the use of the word “fastest.” Similar to how we make these inferences, attention allows the LLMs to analyze the surrounding words (“big bag” or “fastest”) and assign weights to them. The LLM then uses these weights to create a nuanced understanding of “chip” in that specific context.

Let’s now look at an example of how an LLM might try to complete the partial phrase, “The quick brown fox jumps over …”