React Streaming Response

Copy Page

Streaming is not decoration — it is the difference between an interface that feels dead and one that feels alive while a model works. This handles the full lifecycle: a thinking beat, a live caret trailing the text, and the throughput metrics an engineer actually watches.

Installation

Props

Metrics, because this is for engineers

Under the response sit the three numbers a team building on an LLM checks constantly: TTFT (time to first token — the latency users feel most), tokens, and tokens/sec (throughput). They're computed live from the playhead, in tabular-nums, so they never jitter the layout. A consumer wiring this to a real backend gets these for free by feeding real chunk timings into the same state.

Simulated here, real in production

The demo drives a requestAnimationFrame loop over a fixed string, so the whole lifecycle is visible in a static export with no model call and nothing to mislead. In production you delete the loop: keep the revealed slice and the phase machine, and push real stream chunks into idx/revealed as they arrive. Stop sets a cancel flag the loop checks on its next frame, freezing the response at exactly what had arrived rather than snapping to the full text — the honest behaviour, since those are the tokens the user actually received. On completion the partial plain-text render is replaced by the full markdown pass, so code blocks and lists resolve once the content is whole.