Skip to content
TanStackAI
Chat & Streaming

Streaming

chat() returns each token while the model writes the reply. useChat renders those tokens. Call stop() to cancel the run.

Quick setup

1. Send the stream from the server

Call chat(). Then wrap the result with toServerSentEventsResponse:

ts
// api/chat.ts
import {
  chat,
  chatParamsFromRequest,
  toServerSentEventsResponse,
} from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";

export async function POST(request: Request) {
  const { messages, threadId, runId } = await chatParamsFromRequest(request);

  const stream = chat({
    adapter: openaiText("gpt-6.1-sol"),
    messages,
    threadId,
    runId,
  });

  return toServerSentEventsResponse(stream);
}

2. Render the stream with useChat

tsx
import { useState } from "react";
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

export function Chat() {
  const [input, setInput] = useState("");
  const { messages, sendMessage, isLoading, stop } = useChat({
    connection: fetchServerSentEvents("/api/chat"),
    threadId: "support-chat",
  });

  return (
    <>
      {messages.map((message) => (
        <div key={message.id}>
          {message.parts.map((part, index) =>
            part.type === "text" ? <p key={index}>{part.content}</p> : null,
          )}
        </div>
      ))}
      <form
        onSubmit={(event) => {
          event.preventDefault();
          if (input.trim() === "") {
            return;
          }
          sendMessage(input);
          setInput("");
        }}
      >
        <input
          value={input}
          onChange={(event) => setInput(event.target.value)}
        />
        {isLoading ? (
          <button type="button" onClick={stop}>
            Stop
          </button>
        ) : (
          <button type="submit">Send</button>
        )}
      </form>
    </>
  );
}

Send a message. The text grows in the UI while the tokens arrive.

The same pattern works in every UI framework. See Quick Start for your framework.

How it works

  • chatParamsFromRequest reads the AG-UI body that useChat sends.
  • chat() returns a stream of events. toServerSentEventsResponse writes them to the response as Server-Sent Events.
  • useChat reads the events and updates messages after each chunk. isLoading is true while the run is in progress.

If your network blocks SSE, pick another transport on Transports.

Cancel a run

Call stop() on the client. The client aborts the fetch.

To stop the model on the server too, give the same AbortController to chat() and to toServerSentEventsResponse:

ts
import {
  chat,
  chatParamsFromRequest,
  toServerSentEventsResponse,
} from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";

export async function POST(request: Request) {
  const { messages, threadId, runId } = await chatParamsFromRequest(request);
  const abortController = new AbortController();

  const stream = chat({
    adapter: openaiText("gpt-6.1-sol"),
    messages,
    threadId,
    runId,
    abortController,
  });

  return toServerSentEventsResponse(stream, { abortController });
}

After stop():

  • An AbortError is expected.
  • Pending client-tool work for that turn does not continue.
  • A later addToolResult() for that turn has no effect.

Where to go next

Advanced

Stream without HTTP

You can iterate chat() yourself. Branch on chunk.type, then read chunk.delta:

ts
import { chat } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";

const stream = chat({
  adapter: openaiText("gpt-6.1-sol"),
  messages: [{ role: "user", content: "Hello!" }],
  threadId: "thread-1",
});

for await (const chunk of stream) {
  if (chunk.type === "TEXT_MESSAGE_CONTENT") {
    process.stdout.write(chunk.delta);
  }
}

Client callbacks

useChat takes two callbacks for the stream:

  • onChunk: runs for each event.
  • onFinish: runs with the completed message.

A bad request body

If the body is not valid, chatParamsFromRequest throws a Response with status 400. If your framework does not send a thrown Response as the HTTP reply, catch it and return it.

Backpressure

The response stream pauses chat() when its output queue is full. It reads more chunks when the client reads the queued data. toHttpResponse with NDJSON does the same.

A response with durability does not pause. Its run keeps writing to the log.

Main-thread work on the client

The shared ChatClient processes the chunks that are ready in order. It does not add a task between each chunk. After a fixed amount of work, it yields to keep the main thread responsive.

A stream that ends too early

  • The connection drops in the middle of a line. The client throws StreamTruncatedError and moves to error. See Transports: Errors.
  • An OpenAI or OpenRouter Responses stream ends without response.completed. The adapter emits RUN_ERROR with code incomplete-stream. This applies to chat and to structured output. The text that arrived stays available. The run does not call onFinish.
  • A Chat Completions stream ends without a finish. See below.

Chat Completions adapters built on @tanstack/openai-base also make sure that the stream finished. A started stream needs one of these:

  • a non-null finish_reason
  • a final chunk with only usage (choices: [] with usage)

If neither arrives, the adapter closes the open lifecycles and emits RUN_ERROR with code incomplete-stream. The partial text stays available. The run calls onError, skips pending server tools, and does not call onFinish.

The usage-only rule keeps providers compatible. Usage on an earlier chunk, or usage next to a choice, does not count. The OpenAI SDK hides [DONE], so this check cannot see that marker. A stream with only [DONE] after the content still needs a finish reason or a usage-only last chunk.

Edit on GitHub
Stream Events