OpenAI introduced the generative pre-trained transformers (GPT) models in 2018. These models provide unsupervised pretraining, which enables us to leverage heaps of text on the internet without spending our resources on annotations. GPT was succeeded by GPT-2 in 2019 with 1.5 billion parameters. GPT-3 is the latest model in the GPT family. It has 175 billion parameters and allows us to develop excellent applications.

While there are a lot of tasks available in the GPT family of models, its tendency to generate long text from a brief preamble is unparalleled.

Press + to interact

Models for Text Generation

Model	Description
`gpt2`	Transformers-based model trained on large-scale datasets. Trained on English datasets without any labeled data. Has the ability to manipulate and generate the text from the provided instructions and the text.
`Michau/t5-base-en-generate-headline`	Trained on around 500 thousand articles. Highlights the headings from the articles.
`EleutherAI/gpt-j-6B`	Trained using `Mesh Transformer JAX` on the around six billion parameters on the Pile dataset. The `gpt-j-6b` model is used for text generation.
`facebook/opt-350m`	Based on the pretrained `OPT` models and fine-tuned on the five English datasets of around 800 gigabytes. Main purpose is to generate text.
`bigscience/T0`	Trained on large-scale datasets from different sources of English. Outperforms `gpt-3` in some cases and is very small compared to `gpt-3`. Note: The model size is around 41.5 gigabytes, and it takes too long to load it.

Parameter	Type	Category	Description
`inputs`	String	Required	String that instructs to generate the new string
`parameters.top_k`	Integer	Optional	Specifies how many tokens will be considered during text creation from the input text
`parameters.top_p`	Float	Optional	Specifies the probability of the token to be added in summary from the most probable to less probable until the sum of the probabilities is higher than `top_p`
`parameters.temperature`	Float	Optional	Specifies a sampling technique, and value ranges from 1.0 to 100.0. Setting `temperature` to `0` takes tokens with the highest probability. If we set `temperature` to `1`, it does regular sampling. At `100.0` `temperature`, it selects the tokens with uniform probability.
`parameters.max_length`	Integer	Optional	Specifies the number of maximum tokens to be included in the output text
`parameters.num_return_sequences`	Integer	Optional	Specifies the number of results we want to be returned.
`options.use_cache`	Boolean	Optional	Hugging Face Inference API has a cache mechanism implemented to speed up the requests. Use it for the deterministic models. Default value is `true`.
`options.wait_for_model`	Boolean	Optional	Hugging Face Inference API models takes time to initiate and process the requests. If the value is `true`, it waits for the model to get ready instead of returning an error. Default value is `false`.

Press + to interact

// Endpoint URL
const endpointUrl = "https://api-inference.huggingface.co/models/gpt2";
const headerParameters = {
  "Authorization": "Bearer {{ACCESS_TOKEN}}"
};
// Input text to classify
const data = JSON.stringify({
  inputs: "Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive \
  language model that uses deep learning to produce human-like text.",
  parameters: {
    max_length: 32
  },
  options: {
    wait_for_model: true
  }
});
const options = {
  method: "POST",
  headers: headerParameters,
  body: data
};
async function textGeneration() {
   try {
    const response = await fetch(endpointUrl, options);
    printResponse(response);
  } catch (error) {
    printError(error);
  }
}
textGeneration();

Let’s have a look at the highlighted lines shown in the code widget above:

Line 2: We specify the gpt2 model for text generation.
Lines 8–17: We set the inputs with a text to make text generation and set max_length for the output text.
Lines 25–32: We create a function, textGeneration, to make the API call and handle the exceptions.
Line 34: We call the textGeneration function to invoke the endpoint.

Response fields

The API call above returns a dictionary object or a list of dictionary objects, depending on the inputs. The response contains the following field.

Press + to interact

// Endpoint URL
const endpointUrl = "https://api-inference.huggingface.co/models/Michau/t5-base-en-generate-headline";
const headerParameters = {
  "Authorization": "Bearer {{ACCESS_TOKEN}}"
};
// Input text to classify
const data = JSON.stringify({
   inputs: "Context: Richard Feynman was a Physicist. Being one of the most famous scientist ever, he is still remembered in the scientific society.\
  Question: Who was Richard Feynman?",
  parameters: {
    max_length: 50,
    temperature: 0.1
  },
  options: {
    wait_for_model: true
  }
});
const options = {
  method: "POST",
  headers: headerParameters,
  body: data
};
async function textGeneration() {
   try {
    const response = await fetch(endpointUrl, options);
    printResponse(response);
  } catch (error) {
    printError(error);
  }
}
textGeneration();

Introduction

NLP

Computer Vision

APIs Integration in React

Conclusion

Text Generation

Generate text using the API

Models for Text Generation

Request parameters

Response fields

Examples