Search⌘ K
Home
Courses
Mastering spaCy

Solution: Customizing spaCy Models

Explore customizing spaCy models by preparing annotated training data, creating and training a blank NER model with specific entity labels, and visualizing predictions. This lesson guides you step-by-step to build effective custom named entity recognition tailored to your dataset.

We'll cover the following...

Solution

Here is the solution to the previous exercise:

Python 3.5
import random
import spacy
from spacy.training import Example 
from spacy import displacy
train_set = [
  ("I love to eat apples and bananas for breakfast.",{"entities": [(14, 20, "FRUIT"), (25, 32, "FRUIT")]}),
    ("Oranges are a good source of vitamin C.", 
     {"entities": [(0, 7, "FRUIT"), (28, 38, "VITAMIN")]}),
    ("I need to buy some grapes at the grocery store.", 
     {"entities": [(19, 25, "FRUIT")]}),
    ("Pineapple is my favorite fruit.", 
     {"entities": [(0, 9, "FRUIT")]}),
    ("She made a delicious fruit salad with strawberries and kiwis.", 
     {"entities": [(38, 50, "FRUIT"), (55, 60, "FRUIT")]}),
]
entities = ["FRUIT","VITAMIN"]
nlp = spacy.blank("en")
ner = nlp.add_pipe("ner")
for ent in entities:
  ner.add_label(ent)
epochs = 25
other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner']
with nlp.disable_pipes(*other_pipes):
    optimizer = nlp.begin_training()
    for i in range(epochs):
        random.shuffle(train_set)
        for text, annotation in train_set:
            doc = nlp.make_doc(text)
            example = Example.from_dict(doc, annotation)
            nlp.update([example], sgd=optimizer)
doc = nlp("I love to eat strawberries and blueberries for dessert.")
print(doc.ents)
print(doc.ents[0].label_)
colors = {'Fruit': "#00ff00"}
options = {"ents": ['Fruit'], "colors": colors} 
spacy.displacy.render(doc, style="ent", jupyter=True, options=options)

Solution explanation

Lines 1–4: We made the necessary imports. We imported spacy and spacy.training.Example. We also imported random to shuffle our dataset and imported displacy to visualize our test sentences later on. ...