Skip to main content

Chapter 15.1 - Chatting

Until now, our model has been trained in an Instruction → Response format.

### Instruction:
Who created Python?

### Response:
Guido van Rossum.

While this is enough for single-turn tasks, it cannot naturally handle follow-up questions like:

User:
Who created Python?

Assistant:
Guido van Rossum.

User:
When?

Assistant:
1991.

Modern LLMs are therefore fine-tuned on multi-turn conversations, allowing them to remember previous messages inside the context window.

Phase 1 : Use Multi Turn Dataset​

Instead of an instruction dataset, we will use a chat dataset.

We will use UltraChat 200K

https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k

Each training sample looks like:

{
  "messages": [
    {"role":"user","content":"Hello"},
    {"role":"assistant","content":"Hi!"},
    {"role":"user","content":"Who created Python?"},
    {"role":"assistant","content":"Guido van Rossum."}
  ]
}

Notice that instead of storing

  • instruction
  • input
  • output

we now store a complete conversation. Each conversation may contain

  • 1 user message
  • multiple user messages
  • multiple assistant replies

This allows the model to learn how conversations naturally flow.

Phase 2 : Change Dataset Loader​

Now we want our q_instructionDataSet.py file to convert from SPIIRE pattern to conversation pattern

{
  "messages": [
    {"role":"user","content":"Hello"},
    {"role":"assistant","content":"Hi!"},
    {"role":"user","content":"Who created Python?"},
    {"role":"assistant","content":"Guido van Rossum."}
  ]
}

becomes

User:
Hello

Assistant:
Hi!

User:
Who created Python?

Assistant:
Guido van Rossum.

<|endoftext|>

Phase 3 : Response-only Loss Masking​

In our current instruction tuning implementation, the loss is calculated for every token.

Instruction  → loss
Response     → loss

However, modern chat models are usually trained differently. Only assistant responses contribute to the training loss.

User         → ignored
Assistant    → trained
User         → ignored
Assistant    → trained

This is achieved by replacing every user token label with -100.

Example:

User:
Who created Python?

labels

-100 -100 -100 -100 ...

while

Assistant:
Guido van Rossum.

keeps its original labels. This prevents the model from wasting capacity learning to predict user messages and instead focuses all learning on generating assistant responses.

Phase 4 : Chat Prompt Builder​

Previously we built prompts like

### Instruction:
Convert 45 kilometers to meters.

### Response:

Now we build prompts from the complete conversation history.

Example

User:
Hello

Assistant:
Hi!

User:
Who created Python?

Assistant:

The model now predicts only the next assistant reply.

Phase 5 : Conversation History​

Unlike instruction tuning, chat models must remember previous messages.

We therefore maintain a history list.

history = [
    ("user", "Hello"),
    ("assistant", "Hi!"),
    ("user", "Who created Python?")
]

Before every generation, the entire history is converted into text and tokenized.

After generation finishes, the assistant response is appended back into the history.

history.append(("assistant", response))

This enables multi-turn conversations without modifying the transformer itself.

Phase 6 : ChatSession Class​

Instead of calling the generation function directly every time,

generate(...)

we introduce a wrapper class.

chat = ChatSession(model)

chat.ask("Hello")

chat.ask("Who created Python?")

chat.ask("When?")

Internally it performs

Conversation History
        │
        ▼
Prompt Builder
        │
        ▼
Tokenizer
        │
        ▼
GPT-2
        │
        ▼
Streaming Output
        │
        ▼
Update History

This creates an interface similar to modern chat applications.

Phase 7 : Context Window Management​

A conversation continuously grows.

User
Assistant
User
Assistant
User
Assistant
...

Eventually the total number of tokens exceeds the model's context length.

Possible solutions include

  • removing the oldest messages
  • keeping only the most recent conversation
  • summarizing older conversations into a shorter form

Modern LLMs typically use one of these strategies.

Phase 8 : System Prompt​

Modern chat models support a third role besides User and Assistant.

System:
You are a helpful AI assistant.

User:
Hello

Assistant:
Hi!

The system message defines the assistant's personality, behavior, safety rules, or speaking style without requiring retraining.

Summary​

At the end of this chapter, our GPT-2 pipeline changes from

Instruction
      │
      ▼
Response

to

Conversation
      │
      ▼
Prompt Builder
      │
      ▼
Tokenizer
      │
      ▼
GPT-2
      │
      ▼
Assistant Response
      │
      ▼
Update Conversation History

The transformer architecture remains exactly the same.

Only the training data, prompt formatting, and inference pipeline change to support natural multi-turn conversations.