Chat Models

In the previous lesson, you learned how to communicate with the OpenAI LLM using Langchain.

The communication was simple, you provided a prompt and got a response.

In this lesson, you will learn how to use a chat model to have a conversation with the LLM.

Chat Models vs Language Models

Until now, you have been using a language model to communicate with the LLM. A language model predicts the next word in a sequence of words. Chat models are designed to have conversations - they accept a list of messages and return a conversational response.

Chat models typically support different types of messages:

System - System messages instruct the LLM on how to act on human messages
Human - Human messages are messages sent from the user
AI - Responses from the AI are called AI Responses

Create a Chat Model

You are going to create an application that uses a chat model.

The application will:

Use a system message to provide instructions
Use a human message to ask a question
Receive an AI response to the question

Create a new Python program, import the required LangChain modules and instantiate the chat model.

OpenAI API Key

Remember to update the API key with your own.

python

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage

chat_llm = ChatOpenAI(
    openai_api_key="sk-..."
)

Create a system message to provide instructions to the chat model using the SystemMessage class.

python

instructions = SystemMessage(content="""
You are a surfer dude, having a conversation about the surf conditions on the beach.
Respond using surfer slang.
""")

Create a HumanMessage object to ask a question.

python

question = HumanMessage(content="What is the weather like?")

You can now call the chat model passing a list of messages. In this example, pass the system message with the instructions and the human message with the question.

python

response = chat_llm.invoke([
    instructions,
    question
])

print(response.content)

Reveal the complete code

python

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage  

chat_llm = ChatOpenAI(
    openai_api_key="sk-..."
)

instructions = SystemMessage(content="""
You are a surfer dude, having a conversation about the surf conditions on the beach.
Respond using surfer slang.
""")

question = HumanMessage(content="What is the weather like?")

response = chat_llm.invoke([
    instructions,
    question
])

print(response.content)

The response is an AIMessage object.

python

AIMessage(content="Dude, the weather is totally gnarly! It's sunny with some epic offshore winds. Perfect conditions for shredding some sick waves!", additional_kwargs={}, example=False)

Wrapping in a Chain

You can create a reusable chat model chain using what you have learned about prompts and chains.

Rather than passing a list of messages to the chat model, you can create a prompt that gives context to the conversation and then pass the question to the chat model.

Review this program and identify the following:

The prompt provides the instructions to the LLM.
The chain is created using the chat model and the prompt.
The question is passed to the chat model as a parameter of the invoke method.

python

from langchain_openai import ChatOpenAI
from langchain.prompts.prompt import PromptTemplate
from langchain.chains import LLMChain

chat_llm = ChatOpenAI(openai_api_key="sk-...")

prompt = PromptTemplate(
    template="""You are a surfer dude, having a conversation about the surf conditions on the beach.
Respond using surfer slang.

Question: {question}
""",
    input_variables=["question"],
)

chat_chain = LLMChain(llm=chat_llm, prompt=prompt)

response = chat_chain.invoke({"question": "What is the weather like?"})

print(response)

Creating a chain is the first step to creating a more sophisticated chat model. You can use chains to combine different elements into one call and support more complex features.

Giving context

Previously, you learned about grounding and how it can provide context to the LLM and avoid Hallucination.

Currently the chat model is not grounded; it is unaware of surf conditions on the beach. It responds based on the question and the LLMs training data (which could be months or years out of date).

You can ground the chat model by providing information about the surf conditions on the beach.

Review this example where the chat model can access current beach conditions (current_weather) as a variable (context) in the prompt.

python

from langchain_openai import ChatOpenAI
from langchain.prompts.prompt import PromptTemplate
from langchain.chains import LLMChain

chat_llm = ChatOpenAI(openai_api_key="sk-...")

prompt = PromptTemplate(
    template="""You are a surfer dude, having a conversation about the surf conditions on the beach.
Respond using surfer slang.

Context: {context}
Question: {question}
""",
    input_variables=["context", "question"],
)

chat_chain = LLMChain(llm=chat_llm, prompt=prompt)

current_weather = """
    {
        "surf": [
            {"beach": "Fistral", "conditions": "6ft waves and offshore winds"},
            {"beach": "Polzeath", "conditions": "Flat and calm"},
            {"beach": "Watergate Bay", "conditions": "3ft waves and onshore winds"}
        ]
    }"""

response = chat_chain.invoke(
    {
        "context": current_weather,
        "question": "What is the weather like on Watergate Bay?",
    }
)

print(response)

Run the program and predict what the response will be.

Click to reveal the response

Below is a typical response. The LLM has used the context passed in the prompt to provide a more accurate response.

{
    'context': {
        "surf": [
            {"beach": "Fistral", "conditions": "6ft waves and offshore winds"},
            {"beach": "Polzeath", "conditions": "Flat and calm"},
            {"beach": "Watergate Bay", "conditions": "3ft waves and onshore winds"}
        ]},
    'question': 'What is the weather like on Watergate Bay?',
    'text': "Dude, the surf at Watergate Bay is pumping! We got some sick 3ft waves rolling in, but unfortunately, we got some onshore winds messing with the lineup. But hey, it's all good, still plenty of stoke to be had out there!"
}

Investigate what happens when you change the context by adding additional beach conditions.

Providing context is one aspect of Retrieval Augmented Generation (RAG). In this program, you manually gave the model context; however, you could have retrieved real-time information from an API or database.

Check Your Understanding

Message types

Select the message types chat models typically support:

✓ Human
✓ AI
❏ Callback
❏ Prompt
✓ System
❏ Grounding

Hint

Chat models typically support three types of message.

Solution

The correct answers are:

Human
AI
System

Lesson Summary

In this lesson, you learned that chat models are designed for conversations and how to add additional context.

In the next lesson, you will learn how to give your chat model a memory so it can retain information between questions.

Neo4j & LLM Fundamentals

Neo4j and Generative AI

Vectors & Semantic Search

Introduction to Langchain

Using LLMs for Query Generation

Chat Models

Chat Models vs Language Models

Create a Chat Model

Wrapping in a Chain

Giving context

Check Your Understanding

Message types

Lesson Summary

Chatbot