Chat Templates

From messages to a sequence

An application might hold three records: system guidance, a user question and an assistant answer. The model sees tokens. A template chooses how to join the records and which special tokens mark each boundary.

For a schematic, not real model syntax, imagine:

START · USER · question · END-TURN · ASSISTANT · answer · END-TURN

Different models use different delimiters and may handle system messages differently. Use the checkpoint's actual template rather than borrowing markers from a familiar model. The Transformers chat-template guide shows why this matters.

Training and generation meet here

During SFT, the serialized sequence includes the demonstration answer. The loss mask selects which tokens are targets. During generation, a template may append the assistant prefix so the model knows where to continue. Continuing an unfinished assistant message is a different operation from starting a new one.

Avoid adding special beginning/end tokens twice when formatting and tokenizing separately. Test a rendered example before scaling up a dataset: inspect role markers, end tokens and the loss mask together.

Structure is not enforcement

A system-role marker tells the model which learned convention to apply. It does not mathematically prevent a later instruction from changing behavior. Prompt injection and application security need a separate treatment in the systems and safety chapters.

Check yourself: if training ends every assistant response with a turn terminator but inference starts with an unexpected delimiter, what changed? The weights did not change; the input format moved away from the one those weights learned.