How to stream LLM responses in React Native with Server-Sent Events

On Myaigi, a coaching app with seven AI agents behind it, one answer can involve several model calls in a row. Total latency is the sum of all of them. We stream every answer to the React Native client, because users judge speed by whether text is moving, not by the total.

We picked Server-Sent Events over WebSockets for a simple reason: the traffic goes one way. The server talks; the app listens. SSE is plain HTTP, works with normal API servers and load balancers, and needs no extra protocol.

Why the app shouldn’t call the model directly

Anything shipped in an app bundle can be extracted. Put an OpenAI or Anthropic key in your React Native code and assume someone will find it. Your server holds the key, checks who’s asking, applies rate limits, and streams the result back.

Step 1: the server

A minimal Express endpoint. The model call is wrapped in an async generator so you can swap providers without touching the HTTP code.

import express from 'express';
import OpenAI from 'openai';

const app = express();
app.use(express.json());
const openai = new OpenAI(); // reads OPENAI_API_KEY

async function* streamModel(message, signal) {
  const stream = await openai.chat.completions.create(
    { model: process.env.MODEL, messages: [{ role: 'user', content: message }], stream: true },
    { signal },
  );
  for await (const chunk of stream) {
    const text = chunk.choices[0]?.delta?.content;
    if (text) yield text;
  }
}

app.post('/chat', async (req, res) => {
  res.set({
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive',
    'X-Accel-Buffering': 'no', // tell nginx not to buffer
  });
  res.flushHeaders();

  const controller = new AbortController();
  res.on('close', () => controller.abort()); // user left: stop paying for tokens

  try {
    for await (const text of streamModel(req.body.message, controller.signal)) {
      res.write(`data: ${JSON.stringify({ text })}\n\n`);
    }
    res.write('event: done\ndata: {}\n\n');
  } catch (err) {
    if (!controller.signal.aborted) {
      res.write(`event: error\ndata: ${JSON.stringify({ message: 'stream failed' })}\n\n`);
    }
  } finally {
    res.end();
  }
});

Each event is a data: line followed by a blank line. That blank line is the whole protocol; forget it and the client never sees a complete event.

Step 2: the React Native client

React Native’s built-in fetch doesn’t give you a readable stream. Expo’s expo/fetch does (SDK 52 and later), which means you can read the body chunk by chunk:

import { fetch } from 'expo/fetch';

export async function streamChat(
  message: string,
  onText: (text: string) => void,
  signal: AbortSignal,
) {
  const res = await fetch(`${API_URL}/chat`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json', Accept: 'text/event-stream' },
    body: JSON.stringify({ message }),
    signal,
  });
  if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = '';

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    buffer += decoder.decode(value, { stream: true });
    const events = buffer.split('\n\n');
    buffer = events.pop() ?? ''; // keep the half-received event for next time
    for (const event of events) {
      if (event.startsWith('event: done')) return;
      const data = event.split('\n').find((line) => line.startsWith('data: '));
      if (data) onText(JSON.parse(data.slice(6)).text);
    }
  }
}

On a bare React Native project without Expo’s runtime, use an EventSource implementation such as react-native-sse, or add polyfills for streaming fetch and TextDecoder.

Step 3: render without dropping frames

Models can send dozens of tokens a second. Calling setState for each one re-renders the message list dozens of times a second, and on a mid-range Android phone you’ll feel it. Collect text in a ref and flush it on a timer instead:

function useStreamedAnswer(message: string) {
  const [text, setText] = useState('');
  const pending = useRef('');

  useEffect(() => {
    const controller = new AbortController();
    const timer = setInterval(() => {
      if (!pending.current) return;
      const chunk = pending.current;
      pending.current = '';
      setText((t) => t + chunk);
    }, 50);

    streamChat(message, (t) => { pending.current += t; }, controller.signal)
      .catch(() => { /* show a retry button */ });

    return () => { controller.abort(); clearInterval(timer); };
  }, [message]);

  return text;
}

Aborting on unmount matters twice: the app stops downloading, and because the server listens for close, it stops the model call too. You stop paying for an answer nobody will read.

Bugs that will bite you

• A proxy buffers the whole response. Everything arrives at once at the end. Nginx buffers by default (hence X-Accel-Buffering: no), and Express compression middleware does too unless you exclude the route or flush after each write.
• Using req.on('close') to detect disconnects. In current Node.js it can fire as soon as the request body has been read, which aborts every stream immediately. Listen on the response: res.on('close').
• Parsing chunks as if they were events. A network chunk can end halfway through an event. Keep the leftover in a buffer, as in the client above.
• Idle connections get cut. Load balancers close quiet connections (AWS’s Application Load Balancer defaults to 60 seconds). If a model thinks for a while before answering, send a comment line like : ping every 15 seconds.
• localhost on the Android emulator is the emulator itself. Use 10.0.2.2 to reach your machine.

Why streaming works so well

The first token arrives after the model reads your prompt (prefill). Everything after that comes one token per step (decode). Streaming turns the long, sequential part into something the user watches rather than waits for. I go into why that split exists in LLM inference explained for app developers.