# Streaming and cancellation

Streaming lets your application display output as it arrives. Set `stream: true`; the response uses Server-Sent Events instead of one JSON document.

## Request a stream

~~~bash
curl --no-buffer --fail-with-body https://api.simplellm.eu/v1/chat/completions \
  -H "Authorization: Bearer $SIMPLELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3.8-flash-next",
    "messages": [{"role": "user", "content": "Explain how an HTTP request is handled."}],
    "max_tokens": 2048,
    "stream": true,
    "stream_options": {"include_usage": true}
  }'
~~~

The usage option requests a final usage report where supported. It cannot guarantee a report after an interrupted connection.

## Understand the events

This shortened example illustrates event framing rather than a complete provider-specific response:

~~~text
data: {"choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"content":"An HTTP request"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"choices":[],"usage":{"prompt_tokens":20,"completion_tokens":40,"total_tokens":60}}

data: [DONE]
~~~

- Append text from `choices[].delta.content`; do not replace the entire response with each delta.
- An event can contain a role, reasoning, tool-call arguments, or usage instead of visible text.
- Usage-only events may have an empty `choices` array.
- A TCP/HTTP read can contain part of an event or several events; it is not a token boundary.
- If writing an SSE parser, buffer incomplete lines, support CRLF, and terminate an event at a blank line; join multiple data lines before JSON parsing.

Use an existing SDK's stream reader when possible. Do not count events or network chunks as exact tokens.

## Python stream with explicit cleanup

Install the `openai` package and set `SIMPLELLM_API_KEY` and `SIMPLELLM_MODEL` as described in the [Quickstart](/docs/quickstart.html).

~~~python
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.simplellm.eu/v1",
    api_key=os.environ["SIMPLELLM_API_KEY"],
    max_retries=0,
    timeout=600.0,
)
stream = client.chat.completions.create(
    model=os.environ["SIMPLELLM_MODEL"],
    messages=[{"role": "user", "content": "Explain request streaming."}],
    max_tokens=2048,
    stream=True,
    stream_options={"include_usage": True},
)
try:
    for chunk in stream:
        if chunk.usage is not None:
            print("\nUsage:", chunk.usage)
        for choice in chunk.choices:
            if choice.delta.content:
                print(choice.delta.content, end="", flush=True)
finally:
    stream.close()
~~~

The client timeout here is an application choice, not a SimpleLLM platform timeout guarantee. Interrupting the program exits the loop and closes the stream in the cleanup block.

## Cancel the upstream request

When the user presses Stop, close or abort the HTTP request; merely hiding the output in the UI does not cancel generation. If a browser talks to your backend, propagate the browser's disconnect to your backend's upstream request as well.

In the JavaScript SDK, abort the stream's controller when the application cancels:

~~~javascript
// Inside the stream loop or your application's Stop handler:
stream.controller.abort();
~~~

The complete JavaScript client setup is in [client examples](/docs/sdks.html).

> An interrupted stream with zero recorded output tokens is not charged. Already delivered output can still incur a charge, and missing final usage can require an estimate. See [partial-usage billing](/docs/billing.html).

## Interrupted streams

Treat connection closure without the expected completion signals as an incomplete result. Preserve delivered text with an incomplete/cancelled state so the user can distinguish it from a successful answer.

An HTTP 200 only confirms that streaming began. After headers have been sent, a later failure cannot reliably become a new HTTP error status; inspect stream events and connection errors too.

Do not silently replay a request after partial output. Retrying creates a new generation that can duplicate content, tool actions, and charges.