Even though you have provided an LLM with specific instructions, mistakes can still be made.
Few-Shot prompting is a technique where you provide the LLM with an example of how to respond or generate a response to a specific scenario. For example, you could include example Cypher statements in the prompt that answer a specific type of question.
Specific examples
If you know of recurring questions that the LLM struggles with, you can provide it with a few-shot example to help it generate a better response.
For example, how to find actors or directors of specific movies.
CYPHER_GENERATION_TEMPLATE = """
You are an expert Neo4j Developer translating user questions into Cypher to answer questions about movies and provide recommendations.
Convert the user's question based on the schema.
Use only the provided relationship types and properties in the schema.
Do not use any other relationship types or properties that are not provided.
Do not return entire nodes or embedding properties.
Fine Tuning:
For movie titles that begin with "The", move "the" to the end. For example "The 39 Steps" becomes "39 Steps, The" or "the matrix" becomes "Matrix, The".
Example Cypher Statements:
1. To find who acted in a movie:
```
MATCH (p:Person)-[r:ACTED_IN]->(m:Movie {{title: "Movie Title"}})
RETURN p.name, r.role
```
2. To find who directed a movie:
```
MATCH (p:Person)-[r:DIRECTED]->(m:Movie {{title: "Movie Title"}})
RETURN p.name
```
Schema:
{schema}
Question:
{question}
"""
Experiment with the chatbot, try to identify challenging questions, and develop a few-shot example to help the LLM generate a better response.
Six Degrees
The examples don’t have to be simple. They can be complex and deal with very particular scenarios.
Let’s play the Six Degrees of Kevin Bacon game. Pick an Actor at random and ask the Chatbot to identify the connections.
How many degrees of separation are there between
Viola Davis and Kevin Bacon?
Console Output
> Entering new AgentExecutor chain... ```json { "action": "Cypher QA", "action_input": "How many degrees separation are there between Viola Davis and Kevin Bacon?" } ```
> Entering new GraphCypherQAChain chain... Generated Cypher: cypher MATCH p = shortestPath( (viola:Person {name: "Viola Davis"})-[:ACTED_IN|DIRECTED*]-(kevin:Person {name: "Kevin Bacon"}) ) RETURN length(p) as degrees_of_separation
Full Context: [{'degrees_of_separation': 6}]
> Finished chain.
Observation: {'query': 'How many degrees of separation are there between Viola Davis and Kevin Bacon?', 'result': 'There are 6 degrees of separation between Viola Davis and Kevin Bacon.'} Thought:```json { "action": "Final Answer", "action_input": "There are 6 degrees of separation between Viola Davis and Kevin Bacon." } ```
> Finished chain.
The agent provides the correct answer, six, but the game also requires that you name the movies that connect them. The output in the console shows that the path length is returned, but there is no information about the movies in that path.
To help the LLM, you can provide an example in the prompt to demonstrate how Cypher queries should be written.
Here is a query containing all of the information required to help the LLM solve the problem.
MATCH path = shortestPath(
(p1:Person {{name: "Viola Davis"}})-[:ACTED_IN|DIRECTED*]-(p2:Person {{name: "Kevin Bacon"}})
)
WITH path, p1, p2, relationships(path) AS rels
RETURN
p1 {{ .name, .born, link:'https://www.themoviedb.org/person/'+ p1.tmdbId }} AS start,
p2 {{ .name, .born, link:'https://www.themoviedb.org/person/'+ p2.tmdbId }} AS end,
reduce(output = '', i in range(0, length(path)-1) |
output + CASE
WHEN i = 0 THEN
startNode(rels[i]).name + CASE WHEN type(rels[i]) = 'ACTED_IN' THEN ' played '+ rels[i].role +' in 'ELSE ' directed ' END + endNode(rels[i]).title
ELSE
' with '+ startNode(rels[i]).name + ', who '+ CASE WHEN type(rels[i]) = 'ACTED_IN' THEN 'played '+ rels[i].role +' in '
ELSE 'directed '
END + endNode(rels[i]).title
END
) AS pathBetweenPeople
I can’t imagine how much training it would take to get an LLM to write a reduce
function like that…
Add the above Cypher statement to the CYPHER_GENERATION_TEMPLATE
string:
CYPHER_GENERATION_TEMPLATE = """
You are an expert Neo4j Developer translating user questions into Cypher to answer questions about movies and provide recommendations.
Convert the user's question based on the schema.
Use only the provided relationship types and properties in the schema.
Do not use any other relationship types or properties that are not provided.
Do not return entire nodes or embedding properties.
Fine Tuning:
For movie titles that begin with "The", move "the" to the end. For example "The 39 Steps" becomes "39 Steps, The" or "the matrix" becomes "Matrix, The".
Example Cypher Statements:
1. To find who acted in a movie:
```
MATCH (p:Person)-[r:ACTED_IN]->(m:Movie {{title: "Movie Title"}})
RETURN p.name, r.role
```
2. To find who directed a movie:
```
MATCH (p:Person)-[r:DIRECTED]->(m:Movie {{title: "Movie Title"}})
RETURN p.name
```
3. How to find how many degrees of separation there are between two people:
```
MATCH path = shortestPath(
(p1:Person {{name: "Actor 1"}})-[:ACTED_IN|DIRECTED*]-(p2:Person {{name: "Actor 2"}})
)
WITH path, p1, p2, relationships(path) AS rels
RETURN
p1 {{ .name, .born, link:'https://www.themoviedb.org/person/'+ p1.tmdbId }} AS start,
p2 {{ .name, .born, link:'https://www.themoviedb.org/person/'+ p2.tmdbId }} AS end,
reduce(output = '', i in range(0, length(path)-1) |
output + CASE
WHEN i = 0 THEN
startNode(rels[i]).name + CASE WHEN type(rels[i]) = 'ACTED_IN' THEN ' played '+ rels[i].role +' in 'ELSE ' directed ' END + endNode(rels[i]).title
ELSE
' with '+ startNode(rels[i]).name + ', who '+ CASE WHEN type(rels[i]) = 'ACTED_IN' THEN 'played '+ rels[i].role +' in '
ELSE 'directed '
END + endNode(rels[i]).title
END
) AS pathBetweenPeople
```
Schema:
{schema}
Question:
{question}
"""
Braces
Braces in prompt templates are treated as placeholders.
You must escape any braces that you use in example Cypher statements.
When escaping braces, {
becomes {{
and }
becomes }}
.
Adding Additional Examples
You can add as many few-shot examples to the prompt as necessary. But beware, most LLM providers charge you are charged per token, and the larger the prompt, the higher the cost.
If you require many few-shot examples, you can always create multiple tools that call an instance of GraphCypherQAChain
with a different Cypher generation prompt.
Testing the Change
If you now ask the bot the same question, you should get a greatly improved answer.
Console Output
> Entering new AgentExecutor chain... ```json { "action": "Cypher QA", "action_input": "How many degrees of separation are there between Viola Davis and Kevin Bacon?" } ```
> Entering new GraphCypherQAChain chain... Generated Cypher: cypher MATCH path = shortestPath( (p1:Person {name: "Viola Davis"})-[:ACTED_IN|DIRECTED*]-(p2:Person {name: "Kevin Bacon"}) ) WITH path, p1, p2, relationships(path) AS rels RETURN p1 { .name, .born, link:'https://www.themoviedb.org/person/'+ p1.tmdbId } AS start, p2 { .name, .born, link:'https://www.themoviedb.org/person/'+ p2.tmdbId } AS end, reduce(output = '', i in range(0, length(path)-1) | output + CASE WHEN i = 0 THEN startNode(rels[i]).name + CASE WHEN type(rels[i]) = 'ACTED_IN' THEN ' played '+ rels[i].role +' in 'ELSE ' directed ' END + endNode(rels[i]).title ELSE ' with '+ startNode(rels[i]).name + ', who '+ CASE WHEN type(rels[i]) = 'ACTED_IN' THEN 'played '+ rels[i].role +' in ' ELSE 'directed ' END + endNode(rels[i]).title END ) AS pathBetweenPeople
Full Context: [{'start': {'born': neo4j.time.Date(1965, 8, 11), 'link': 'https://www.themoviedb.org/person/19492', 'name': 'Viola Davis'}, 'end': {'born': neo4j.time.Date(1958, 7, 8), 'link': 'https://www.themoviedb.org/person/4724', 'name': 'Kevin Bacon'}, 'pathBetweenPeople': 'Viola Davis played Carol Barrett in Blackhat with Chris Hemsworth, who played Nicholas Hathaway in Blackhat with Chris Hemsworth, who played The Huntsman in Snow White and the Huntsman with Charlize Theron, who played Queen Ravenna in Snow White and the Huntsman with Charlize Theron, who played Karen Jennings in Trapped with Kevin Bacon, who played Joe Hickey in Trapped'}]
> Finished chain.
Observation: {'query': 'How many degrees of separation are there between Viola Davis and Kevin Bacon?', 'result': 'There are three degrees of separation between Viola Davis and Kevin Bacon. Viola Davis co-starred with Chris Hemsworth in Blackhat, Chris Hemsworth co-starred with Charlize Theron in Snow White and the Huntsman, and Charlize Theron co-starred with Kevin Bacon in Trapped.'} Thought:```json { "action": "Final Answer", "action_input": "There are three degrees of separation between Viola Davis and Kevin Bacon. Viola Davis co-starred with Chris Hemsworth in Blackhat, Chris Hemsworth co-starred with Charlize Theron in Snow White and the Huntsman, and Charlize Theron co-starred with Kevin Bacon in Trapped." } ```
> Finished chain.
Did you get a better answer? Once you have completed the steps, click the button below to mark the lesson as completed.
Summary
In this challenge, you provided the LLM with examples of Cypher statements to answer specific questions.