Guides

Streaming

Set stream: true to receive the completion as Server-Sent Events (SSE) — tokens arrive as they are generated, so you can render output live and measure time-to-first-token. The wire format is OpenAI-compatible, byte-for-byte.

With an SDK

The OpenAI SDKs handle SSE for you — iterate the stream and read choices[0].delta.content off each chunk. This is exactly the OpenAI streaming API:

node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.unoblox.ai/v1",
  apiKey: process.env.UNOBLOX_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "<model-id>",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.unoblox.ai/v1",
    api_key=os.environ["UNOBLOX_API_KEY"],
)

stream = client.chat.completions.create(
    model="<model-id>",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:  # terminal usage frame
        print(f"\n[usage] {chunk.usage.total_tokens} tokens")

The wire format

A streamed response is served with Content-Type: text/event-stream and Cache-Control: no-cache. The gateway proxies the provider's SSE bytes through unchanged, so each chunk is a standard chat.completion.chunk. You will see three kinds of line:

Line	Meaning
data: {…}	A completion chunk. The token is at choices[0].delta.content; the last content chunk carries finish_reason.
: UNOBLOX PROCESSING	A keepalive comment (any line starting with :). Ignore it — it keeps the connection warm during a slow first token.
data: [DONE]	The terminal marker. Stop reading when you see it.

raw SSE stream

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","model":"<model-id>","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"One"},"finish_reason":null}]}

: UNOBLOX PROCESSING

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-abc","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":11,"completion_tokens":8,"total_tokens":19}}

data: [DONE]

Token usage on the final frame

unoblox asks every upstream to emit a trailing usage frame (it sets stream_options.include_usage = true for you). When the provider supports it, the last data frame before [DONE] has an empty choices array and a usage object with the prompt, completion and total token counts. Not every provider emits it, so treat it as best-effort.

Cost is not in the stream.

The usage frame carries token counts, not ₹. Rupee cost is settled after the request — read it from the generation lookup or your usage rollups. See Pricing & credits.

Parsing SSE yourself

If you are not using an SDK, read the response body as a stream, split on the blank-line frame delimiter, and skip comment lines. Capture the x-generation-id response header — it is present on every response and identifies the request for later lookup.

manual SSE parsing

const res = await fetch("https://api.unoblox.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer " + process.env.UNOBLOX_API_KEY,
  },
  body: JSON.stringify({
    model: "<model-id>",
    messages: [{ role: "user", content: "Count to five." }],
    stream: true,
  }),
});

if (!res.ok) {
  // Pre-stream failures (auth, rate limit, balance) are normal HTTP errors.
  throw new Error(JSON.stringify(await res.json()));
}
const generationId = res.headers.get("x-generation-id");

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

outer: while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  // SSE frames are separated by a blank line.
  let sep;
  while ((sep = buffer.indexOf("\n\n")) !== -1) {
    const frame = buffer.slice(0, sep);
    buffer = buffer.slice(sep + 2);
    for (const line of frame.split("\n")) {
      if (line.startsWith(":")) continue;           // keepalive comment — ignore
      if (!line.startsWith("data:")) continue;
      const data = line.slice(5).trim();
      if (data === "[DONE]") break outer;            // terminal marker
      const chunk = JSON.parse(data);
      if (chunk.error) throw new Error(chunk.error.message); // in-stream error
      process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
    }
  }
}

Errors during a stream

Once the stream has started, the HTTP status is already 200, so a mid-stream upstream failure cannot change it. Instead the gateway emits a terminal error event — a data: {"error":{…}} frame carrying the standard error envelope, followed by data: [DONE]. Always check each parsed chunk for an error field.

Failures that happen before the stream opens — authentication, rate limits, an empty balance, an invalid body — arrive as an ordinary HTTP error with a JSON body, exactly like a non-streaming request. See Errors.

Long replies & timeouts

Thinking (reasoning) models can spend minutes on a hard prompt: at a typical decode speed of 30 tokens per second, a 4,000-token reply takes over two minutes, and most of that can be reasoning before the first word of the answer. With stream: false the server has nothing to send until the whole reply is finished, so the connection looks idle the entire time.

Use stream: true for anything that may run long.

A streamed request delivers the reasoning and the answer as they are generated (reasoning arrives as delta.reasoning_content on models that expose it), so the connection is never idle and your app sees progress from the first second.

If you must use a non-streaming call, raise your client's total timeout to at least 600 to 900 seconds. Many apps, frameworks and proxies set 60 to 120 seconds (the official OpenAI SDKs default to 10 minutes), and an app that disconnects before the reply is ready cannot receive it. Disable automatic retries for these calls, since a retry starts a second request.

python, non-streaming with a long timeout

import os
from openai import OpenAI

# Non-streaming: nothing is sent until the whole reply is finished, so give the
# client a total timeout longer than the slowest reply you expect.
client = OpenAI(
    base_url="https://api.unoblox.ai/v1",
    api_key=os.environ["UNOBLOX_API_KEY"],
    timeout=900,        # seconds
    max_retries=0,      # a retry would start a second, separately billed request
)

resp = client.chat.completions.create(
    model="<model-id>",
    messages=[{"role": "user", "content": "Write a detailed plan."}],
)
print(resp.choices[0].message.content)

node.js, non-streaming with a long timeout

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.unoblox.ai/v1",
  apiKey: process.env.UNOBLOX_API_KEY,
  timeout: 900_000, // milliseconds
  maxRetries: 0,    // a retry would start a second, separately billed request
});

const resp = await client.chat.completions.create({
  model: "<model-id>",
  messages: [{ role: "user", content: "Write a detailed plan." }],
});
console.log(resp.choices[0].message.content);

curl, non-streaming with a long timeout

# --max-time is curl's TOTAL limit. 900 s here; the default is no limit,
# but proxies and wrappers around curl often impose their own.
curl --max-time 900 https://api.unoblox.ai/v1/chat/completions \
  -H "Authorization: Bearer $UNOBLOX_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "messages": [{ "role": "user", "content": "Write a detailed plan." }]
  }'

Other tips: lower max_tokens to bound the worst case, and ask for less reasoning where the model supports it. Requests that your app abandons are marked "client aborted" in the request details of your usage, with how long the call ran before your app disconnected.

Streaming the other skins

The Anthropic Messages skin (/v1/messages) and the OpenAI Responses skin (/v1/responses) stream their own native event shapes — message_start / content_block_delta for Anthropic, response.output_text.delta for Responses. The transport (SSE, keepalives, terminal error events) is the same. Event lists are in the API reference.

PreviousRouting NextBring your own key