Before going any further, it is worth pausing to think about how we will store the conversation history in the graph.
The chatbot will use two methods to retrieve information from the graph, but both need to store memory consistently. As the particular implementation may change based on the data retrieval method, the chains themselves will be responsible for persisting the memory, keeping the logic close to the chain.
Let’s walk through an example conversation with a user asking for a recommendation.
-
Human: Can you recommend a movie about robots?
-
AI: Sure, iRobot is a 2004 sci-fi film where a homicide detective with a distrust of robots investigates a homicide with the help of a robopsychologist.
-
Human: That doesn’t sound very age-appropriate. Can you recommend a film suitable for children?
-
AI: Oh, sorry. WALL-E is a 2008 animated film where a waste-cleaning robot falls in love and saves the world. It has a 4.2-star rating.
-
Human: That sounds interesting, tell me more…
-
AI: The film is a Disney Pixar production directed by Andrew Stanton. Ben Burtt voices WALL-E, while Jeff Garlin plays Captain B. McCrea.
Retrieval Tool
When the human asks for a recommendation based on its plot, the chatbot will embed the rephrased question and search the vector index for similar embeddings. The text associated with those embeddings will be appended to the prompt, allowing the LLM to use the information to generate a natural language answer.
At the same time, the chain will be responsible for creating a :CONTEXT relationship from the (:Response) node to the node that contains the embedding.
Cypher Tool
As the human asks more complex questions, the chatbot must fall back to the Cypher tool, which will generate and execute a Cypher statement capable of answering the rephrased question.
The Cypher generation prompt must include instructions to return the elementId of the node along with any relevant properties.
This will provide the chain with the information needed to create the :CONTEXT relationship, which, in turn, will provide additional clarity on the response.
The (:Response) node should also store the generated Cypher statement.
Note also the addition of the .cypher property on the (: Response), which will clarify the Cypher statement used to query the database.
Saving the conversation history
The chatbot must create a (:Response) node that contains the original input, the rephrased question, and the output from the LLM.
As users receive one or more responses within a session, this forms a natural hierarchy of (:Session) nodes, each containing relationships to many (:Response) nodes.
Regardless of which chain creates it, the (:Response) node should hold the following properties:
-
.createdAt- The creation date of the node -
.input- The original input that comes from the user -
.rephrasedQuestion- The standalone question rephrased by the LLM using the conversation history -
.output- The output subsequently generated by the LLM following the RAG stage
Creating a Response Chain
Creating a chain of :NEXT relationships between responses will provide an optimal way to find the conversation history.
When adding a new response, the chatbot must create an additional :LAST_RESPONSE relationship from the (:Session) node. This relationship offers a shortcut to locate the latest response in the session, from which one can follow the :NEXT relationship to trace the recent response history.
Using the :LAST_RESPONSE relationship to obtian recent message history will be more performant than collecting all responses in memory and ordering by the .createdAt timestamp.
From there, you will follow the path of :NEXT relationships in a variable-length traversal to obtain a list of previous messages.
This :LAST_RESPONSE pointer will need to be actively maintained, removing any old relationships before creating a new one to ensure the pointer always points to the latest response.
Modeling Decisions
You can learn more about modeling nodes and relationships in the Graph Data Modelling Fundamentals course.
Saving responses
You will use the following Cypher statement to save conversation history to the database.
Unresolved directive in lesson.adoc - include::https://raw.githubusercontent.com/neo4j-graphacademy/llm-vectors-unstructured/main/cypher/save-response.cypher[]The statement performs the following actions:
-
Find or create the user’s session
-
Create a new
(:Response)node with properties -
If it exists, remove any existing
:LAST_RESPONSErelationship from the session -
Use the
lastnode in the chain to create a:NEXTrelationship to the newly created node -
Create a
:LAST_RESPONSErelationship from the session to the new Response node -
Create
:CONTEXTrelationships to any element IDs used in the response
Rephasing Questions
In the next lesson, you will build a chain that utilizes the conversation history to rephrase a question into a standalone question.
Check Your Understanding
Contextual Relationships
What does the :CONTEXT relationship do in the conversation model?
-
❏ It links the user’s session to the AI’s response
-
✓ It creates a connection between the
(:Response)node and the nodes which were used to construct the response -
❏ It stores the original input from the user
-
❏ It provides a rating for the movie recommendation
Hint
In Retrieval Augmented Generation (RAG), contextual information is provided along with the prompt to improve the accuracy of the response.
Solution
The :CONTEXT relationship creates a connection between the (:Response) node and the nodes that were used to construct the response.
Chaining Responses
What does creating a chain of :NEXT relationships between responses accomplish?
-
❏ It increases the complexity of the database
-
❏ It decreases the efficiency of retrieving conversation history
-
✓ It provides an optimal way to find the conversation history
-
❏ It stores the movie recommendations more securely
Hint
Neo4j is designed to follow relationships in a connected dataset.
Solution
The :NEXT relationship provides an optimal way to find the conversation history.
Last Response Relationship
What is the purpose of the :LAST_RESPONSE relationship?
-
❏ To point to the first response in the conversation
-
✓ To provide a shortcut to find the latest response in the session
-
❏ To link the user directly to the movie recommendation
-
❏ To store the AI’s response for future reference
Hint
The last response in a relationship is the most contextually important in the conversation.
Solution
The purpose of the :LAST_RESPONSE relationship is to provide a shortcut to find the latest response in the session.
Summary
In this lesson, we explored the data model you will use to store conversation history in the graph.
In the next lesson, you will create a Neo4jGraph object for interacting with Neo4j.