Stop sequence
A literal piece of text that tells the generation engine to stop as soon as the model produces it.
See it
What it is
A stop sequence is a string that the generation engine watches for in the model's output. When that string appears, generation ends instead of continuing until the token cap or the model's own end token. It is a useful boundary for formats with a clear closing marker, or for stopping a completion before it starts inventing the next speaker in a transcript. The habit dates to the text-completion era, when a chat was one long transcript in a single prompt and passing `\nHuman:` as a stop sequence was the only thing keeping the model from writing the user's next line too.
Reach for one when your protocol already has an unambiguous delimiter such as `</answer>` or a document sentinel. Pick a string that cannot plausibly occur inside valid content, and test it through the exact SDK and streaming path you ship. Engines differ on whether the matched text is included in the returned output.
Gotcha: this is literal string detection, not understanding. If the model mentions the sentinel as ordinary content, the answer ends early; if it misspells or reformats the sentinel, generation continues. It is also not a substitute for structured output or a parser when the boundary actually depends on syntax.
Ask AI for it
Add an explicit stop sequence to this vLLM generation path. Wrap the requested answer in `<answer>` and `</answer>`, then configure `SamplingParams(stop=["</answer>"], include_stop_str_in_output=False)`. Strip only the opening tag after generation, preserve all content before the stop match, and add tests for a normal completion, a streamed match split across chunks, a response that reaches `max_tokens` without the marker, and user content that contains the literal closing tag.