Machine Translation

Get an overview of machine translation, and learn to perform it using the Hugging Face Inference API.

We'll cover the following

Another excellent application of NLP is machine translation, where a text written in a natural language is automatically translated into another natural language. There has been a significant and continuous improvement in the results produced by some common translators, like Google and Bing. This improvement can be attributed to bigger datasets and better models.

Press + to interact

Machine translation was one of the earliest intended applications of AI. Despite its initial success in games and some trivial tasks, AI was unable to perform machine translation. This was one significant reason behind the first AI winterSimilar to nuclear winter, AI winter refers to the era of reduced funding and interest in AI research..

Hugging Face gives us the luxury of choosing among several translation models. As of November 2022, there are more than 1,600 models on translation alone. This number is progressively increasing.

Language translation

The Helsinki-NLP/opus-mt-en-fr model is recommended for language translation tasks. Helsinki-NLP is trained in many languages and provides many models to translate in different languages. The model name opus-mt-en-fr shows that en is the source language and fr is the targeted language, translating from English to French. The short form of the languages is used.

However, what about a language like Lhasa or even an Indo-European language, like Punjabi or Pashto? These languages have many speakers but are impeded by a lack of trained models. Fret not! We can use a pretrained multilingual model in these scenarios and fine-tune it to the desired language.

There are many models available for the language translation task. Let's limit our discussion to a few Helsinki-NLP models for specific language translations:

Models for Language Translation

Source → Target

Model

French → English

Helsinki-NLP/opus-mt-fr-en

German → English

Helsinki-NLP/opus-mt-de-en

English → German

Helsinki-NLP/opus-mt-en-de

Arabic → English

Helsinki-NLP/opus-mt-ar-en

English → Arabic

Helsinki-NLP/opus-mt-en-ar

We can call the following endpoint via the POST request method for language translation task by replacing the path parameter {model} with any model mentioned above:

https://api-inference.huggingface.co/models/{model}

Request parameters

The request parameters for this API call are as follows:

Parameter

Type

Category

Description

inputs

String

Required

Specifies a string or string [ ] to be translated

options.use_cache

Boolean

Optional

Hugging Face Inference API has a cache mechanism implemented to speed up the requests. Use it for the deterministic models. Default value is true.

options.wait_for_model

Boolean

Optional

Hugging Face Inference API models takes time to initiate and process the requests. If the value is true, it waits for the model to get ready instead of returning an error. Default value is false.

The code below translates the inputs from English to French.

Press + to interact
// Endpoint URL
const endpointUrl = "https://api-inference.huggingface.co/models/Helsinki-NLP/opus-mt-en-fr";
const headerParameters = {
  "Authorization": "Bearer {{ACCESS_TOKEN}}"
};
// Input text to classify
const data = JSON.stringify({
  inputs: "All the variety, all the charm, all the beauty of life is made up of light and shadow.",
  options: {
    wait_for_model: true
  }
});
const options = {
  method: "POST",
  headers: headerParameters,
  body: data
};
async function translate() {
   try {
    const response = await fetch(endpointUrl, options);
    printResponse(response);
  } catch (error) {
    printError(error);
  }
}
translate();

Let’s have a look at the highlighted lines shown in the code widget above:

  • Line 2: We specify the Helsinki-NLP/opus-mt-en-fr model for translating from English to French.

  • Line 9: We set the inputs for the translation.

  • Lines 21–28: We create a function, translate, to make the API call and handle the exceptions.

  • Line 30: We call the translate function to invoke the endpoint.

Response fields

The API call above returns a dictionary object or a list of dictionary objects, depending on the inputs. The response contains the following field.

Parameter

Type

Description

translation_text

String

Specifies the translated string or string [ ] of inputs

Examples

Now, try the following examples to translate to other languages using the different models in the widget above.

Translate Arabic to English:

Press + to interact
inputs: "عندما ذهبت إلى المكتبة"

Translate English to Arabic:

Press + to interact
inputs: "It's a pleasant day."