Master Knowledge Graph Retrieval-Augmented Generation with Neo4j/

...

/

Improved Entity and Relationship Extraction Using LLMs

Improved Entity and Relationship Extraction Using LLMs

Learn how LLMs are used to extract accurate entities and relationships from the raw text.

We'll cover the following...

Importing OpenAI for using LLMs

First, we need to import the OpenAI library to access its LLM services. OpenAI provides APIs that allow us to interact with their models, such as GPT-3 or GPT-4, to perform various natural language processing tasks, including entity and relationship extraction.

from openai import OpenAI

Creating OpenAI client

Before making requests to the OpenAI API, we need to create a client object using our OpenAI API key. This client will be used to communicate with the API.

client = OpenAI(api_key=OPENAI_API_KEY)

Note: OPENAI_API_KEY will be your personal API key required for authentication with the OpenAI service. If you don't have an API key, you can still use the same prompts as provided in the code below by visiting the official website of any LLM-powered service, such as ChatGPT, and entering the prompts directly into their interface. The responses might differ a bit due to the nature and type of the LLM being used.

Prompting LLM to extract entities and relationships

To instruct the LLM to extract entities and relationships, we create a prompt that guides the model in generating the desired information specified in the prompt. Our desired information is entities and relationships in the given text. In the following code snippet, we write the prompt for the LLM model.

messages = [
  {"role": "system", "content": """"
  You are a helper tool for a knowedge graph builder application. Your task is to extract entities and relationships from the text provided by the user. 
  Format the output in such a way that it can be directly parsed into Python lists. 
  The format should include:
  
  1. A list of **Entities** in Python list format.
  2. A list of **Relationships**, where each relationship is represented as a tuple in the format: (Entity 1, "Relationship", Entity 2).
  
  Here is the format to follow:
  
  Entities: ["Entity 1", "Entity 2", ..., "Entity N"]
  
  Relationships: [("Entity 1", "Relationship", "Entity 2"), ..., ("Entity X", "Relationship", "Entity Y")]
  
  Example Input:
  Extract entities and relationships from the following text:
  "Michael Jackson, born in Gary, Indiana, was a famous singer known as the King of Pop. He passed away in Los Angeles in 2009."
  
  Expected Output:
  
  Entities: ["Michael Jackson", "Gary, Indiana", "Los Angeles", "singer", "King of Pop", "2009"]
  
  Relationships: [
    ("Michael Jackson", "born in", "Gary, Indiana"), 
    ("Michael Jackson", "profession", "singer"), 
    ("Michael Jackson", "referred to as", "King of Pop"), 
    ("Michael Jackson", "passed away in", "Los Angeles"), 
    ("Michael Jackson", "date of death", "2009")
  ]
  """},
  {"role": "user", "content": f"Extract entities and relationship tuples from the following text:\n\n{text}\n\n"}
]
A prompt for the ChatGPT model to extract entities and relationships from the given text

  • messages contains a list of message objects.

    • Lines 2–31: The system message sets the context for the LLM.

    • Line 32: The user message contains the actual prompt with a placeholder for the raw text provided by the user.

Now, we will create a chat using the OpenAI client. We will specify our preferred large language model and provide the prompt mentioned above.

response = client.chat.completions.create(
  model="gpt-4",
  messages=messages,
  )

  • client.chat.completions.create() sends a request to the OpenAI API to generate a completion for a given prompt.

  • model="gpt-4" specifies the model to use. ...