Answer Generation Chain

When implementing Retrieval Augmented Generation (RAG), you provide information as part of the prompt that will help the LLM generate an accurate response.

In this challenge, you must modify the initGenerateAnswerChain() function in modules/agent/chains/answer-generation.chain.ts to return a chain that instructs the LLM to generate an answer to a question based on the context provided. The chain should not be concerned with where that information comes from.

The chain will accept the following input:

typescript

Chain Input

export interface GenerateAnswerInput {
  question: string;
  context: string;
}

The chain will need to:

Format a prompt based on the input to the chain. The prompt template must include {question} and {context}
Pass the formatted prompt to the LLM
Convert the output to a string

Open answer-generation.chain.ts →

Modifying Functions

Each challenge in this course will follow the same structure. The repository contains helper functions that will accept the appropriate arguments. You will complete the implementation using the arguments passed to the function.

Later in this course, as you build out the agent, you will use these functions to create instances of the chains and register them as tools.

Create a Prompt Template

Inside the initGenerateAnswerChain() function, use the PromptTemplate.fromTemplate() method to create a new prompt template. Use the following prompt as the first parameter.

Prompt

Use only the following context to answer the following question.

Question:
{question}

Context:
{context}

Answer as if you have been asked the original question.
Do not use your pre-trained knowledge.

If you don't know the answer, just say that you don't know, don't try to make up an answer.
Include links and sources where possible.

Due to the nature of semantic search, the documents passed by the application to this prompt may not fully answer the question. Therefore. this prompt instructs the LLM to do its best to generate an answer based on the input or respond with "I don’t know" if it cannot answer the question.

Later in the course, you will build a chain to provide a definitive answer.

Your code should resemble the following:

typescript

Prompt Template

const answerQuestionPrompt = PromptTemplate.fromTemplate(`
  Use only the following context to answer the following question.

  Question:
  {question}

  Context:
  {context}

  Answer as if you have been asked the original question.
  Do not use your pre-trained knowledge.

  If you don't know the answer, just say that you don't know, don't try to make up an answer.
  Include links and sources where possible.
`);

Create the Runnable Sequence

Use the RunnableSequence.from() method to create a new chain. The chain should pass the prompt to the LLM passed as a parameter, then format the response as a string using a new instance of the StringOutputParser.

typescript

return RunnableSequence.from<GenerateAnswerInput, string>([
  answerQuestionPrompt,
  llm,
  new StringOutputParser(),
]);

Use the return keyword to return the chain from the function.

Working Solution

Click here to reveal the fully-implemented answer-generation.chain.ts

export default function initGenerateAnswerChain(
  llm: BaseLanguageModel
): RunnableSequence<GenerateAnswerInput, string> {
  const answerQuestionPrompt = PromptTemplate.fromTemplate(`
    Use only the following context to answer the following question.

    Question:
    {question}

    Context:
    {context}

    Answer as if you have been asked the original question.
    Do not use your pre-trained knowledge.

    If you don't know the answer, just say that you don't know, don't try to make up an answer.
    Include links and sources where possible.
  `);

  return RunnableSequence.from<GenerateAnswerInput, string>([
    answerQuestionPrompt,
    llm,
    new StringOutputParser(),
  ]);
}

Using the Chain

You will be able to initialize and run the chain in your application with the following code:

typescript

const llm = new OpenAI() // Or the LLM of your choice
const answerChain = initGenerateAnswerChain(llm)

const output = await answerChain.invoke({
  input: 'Who is the CEO of Neo4j?',
  context: 'Neo4j CEO: Emil Eifrem',
}) // Emil Eifrem is the CEO of Neo4j

Testing your changes

If you have followed the instructions, you should be able to run the following unit test to verify the response using the npm run test command.

Running the Test

npm run test speculative-answer-generation.chain.test.ts

View Unit Test

typescript

speculative-answer-generation.chain.test.ts

import { config } from "dotenv";
import initGenerateAnswerChain from "./answer-generation.chain";
import { BaseChatModel } from "langchain/chat_models/base";
import { RunnableSequence } from "@langchain/core/runnables";
import { ChatOpenAI } from "@langchain/openai";
import { PromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

describe("Speculative Answer Generation Chain", () => {
  let llm: BaseChatModel;
  let chain: RunnableSequence;
  let evalChain: RunnableSequence<any, any>;

  beforeAll(async () => {
    config({ path: ".env.local" });

    llm = new ChatOpenAI({
      openAIApiKey: process.env.OPENAI_API_KEY,
      modelName: "gpt-3.5-turbo",
      temperature: 0,
      configuration: {
        baseURL: process.env.OPENAI_API_BASE,
      },
    });

    chain = await initGenerateAnswerChain(llm);

    // tag::evalchain[]
    evalChain = RunnableSequence.from([
      PromptTemplate.fromTemplate(`
        Does the following response answer the question provided?

        Question: {question}
        Response: {response}

        Respond simply with "yes" or "no".
      `),
      llm,
      new StringOutputParser(),
    ]);
    // end::evalchain[]
  });

  describe("Simple RAG", () => {
    it("should use context to answer the question", async () => {
      const question = "Who directed the matrix?";
      const response = await chain.invoke({
        question,
        context: '[{"name": "Lana Wachowski"}, {"name": "Lilly Wachowski"}]',
      });

      // tag::eval[]
      const evaluation = await evalChain.invoke({ question, response });

      expect(`${evaluation.toLowerCase()} - ${response}`).toContain("yes");
      // end::eval[]
    });

    it("should refuse to answer if information is not in context", async () => {
      const question = "Who directed the matrix?";
      const response = await chain.invoke({
        question,
        context:
          "The Matrix is a 1999 science fiction action film starring Keanu Reeves",
      });

      const evaluation = await evalChain.invoke({ question, response });
      expect(`${evaluation.toLowerCase()} - ${response}`).toContain("no");
    });

    it("should answer this one??", async () => {
      const role = "The Chief";

      const question = "What was Emil Eifrems role in Neo4j The Movie??";
      const response = await chain.invoke({
        question,
        context: `{"Role":"${role}"}`,
      });

      expect(response).toContain(role);

      const evaluation = await evalChain.invoke({ question, response });
      expect(`${evaluation.toLowerCase()} - ${response}`).toContain("yes");
    });
  });
});

Summary

In this lesson, you implemented your first chain using LCEL. You will use this chain to convert raw data from Neo4j into a natural language response.

In the next lesson, you will learn how to validate the response generated by the LLM.

Build a Neo4j-backed Chatbot with TypeScript

Getting Started

Chains

Conversation History

Retrieval Chain

Cypher Retrieval Tool

Creating an Agent

Answer Generation Chain

Create a Prompt Template

Create the Runnable Sequence

Working Solution

Using the Chain

Testing your changes

Summary