Streaming

Long text takes as long to synthesize as it takes. Streaming lets playback start on the first sentence — typically 1–2 seconds on the standard plans, or a ceiling agreed per contract on Enterprise — instead of after the last one.

Engine — and what it costs

Streaming runs on v4. It always did — v3 delivered its chunks in a burst at the end, so a stream request naming it was refused rather than served under a time-to-first-audio promise it could not keep — and since 2026-09-24 v4 is the only engine on the cloud API at all, so there is nothing to choose: omit engine, or send v4. Naming v3 on this route now gets the same 400 as on every other: Engine "v3" was retired on 2026-09-24. Use engine "v4" and a voice from GET /v1/voices?engine=v4 — the two catalogues share no ids.

The price is the v4 price. A stream is billed per submitted character at v4's multiplier — today 3×, times the platform-wide globalMultiplier — exactly like a POST /v1/tts job for the same text. The old comparison, a stream at v4's rate against a job on v3's 1.5×, no longer means anything, because the 1.5× rate is gone from every route; if you sized a budget against it, resize it. GET /v1/engines returns the live multipliers, no key required — see Engines.

This applies to cloned voices as well. Clones have enrolled on v4 since 2026-08-28, so they stream like any other v4 voice; an older v3-enrolled clone has to be re-enrolled first. See Cloned voices.

There are two streaming endpoints, and which you want depends on who is reading the bytes:

POST /v1/audio/speech with stream_format — plain audio or SSE, and what an OpenAI client already knows how to read. Start here.
POST /v1/tts/stream — VieNeu's native framed stream. Use it when you want each chunk delivered as a separately decodable unit, or when you want a positive signal that the stream finished.

The native stream

curl -N -X POST https://api.vieneu.io/api/v1/tts/stream \
  -H "Authorization: Bearer $VIENEU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "text": "…", "voiceId": "Ngọc Lan" }' \
  --output stream.bin

The response body is a sequence of length-prefixed frames:

[4-byte big-endian uint32 = N][N bytes of audio]   … repeated …
[4-byte big-endian uint32 = 0]                     ← end of stream

Read a length, read that many bytes, repeat. By default each payload is a self-contained WAV, so you can hand a frame straight to a decoder without waiting for the rest.

Response headers tell you what you actually got:

Header	Meaning
`X-Sample-Rate`	Sample rate of the audio, in Hz
`X-Output-Format`	The encoding: `wav`, `mp3`, `opus`, `pcm` or `ulaw`
`X-Stream-Format`	`len32-wav-chunks`, or `len32-frames` for other encodings

The zero-length frame is the point

A stream that finishes sends a final frame with length 0. A stream cut short by a failure mid-generation simply stops, with no such frame.

Its absence means the stream was cut short: discard the audio. You are not billed for a stream that never sent it — the refund is automatic.

But the marker alone is not success. A stream can arrive complete and still carry no speech — a generation that produced nothing sends its heartbeats (see below) and then a clean end-of-stream. We do not bill those either, so a caller who treats the marker as sufficient will book a success, save a silent file, and end the month reconciling against an invoice that never charged for it.

The condition we bill on, and the one you should use, is both halves:

the end-of-stream marker arrived, AND at least one frame carried samples.

Heartbeat frames

A stream that goes quiet for a while sends a heartbeat: a valid WAV file containing zero samples, 44 bytes on the wire. Its only job is to keep the connection from being closed for inactivity by whatever sits between us — a proxy, a corporate gateway, a mobile carrier's NAT.

Heartbeats appear only on WAV streams (X-Stream-Format: len32-wav-chunks), at most one every X-Stream-Heartbeat seconds, and only while the synthesizer has produced nothing new. A stream that flows normally never sends one.

You must skip them. The trap is that a heartbeat is a well-formed WAV, so a decoder will not reject it — decodeAudioData returns a zero-length buffer rather than throwing, and writing the frame to a file leaves a stray 44-byte header in the middle of your audio. Checking that a frame is non-empty is not enough; ask whether it carries samples:

def has_samples(frame: bytes) -> bool:
    """False for a heartbeat: a valid WAV whose `data` chunk is empty."""
    if not frame:
        return False                     # nothing there at all
    if len(frame) < 12 or frame[:4] != b"RIFF" or frame[8:12] != b"WAVE":
        return True                      # not a WAV we can read — assume audio
    off = 12
    while off + 8 <= len(frame):
        chunk_id = frame[off:off + 4]
        size = int.from_bytes(frame[off + 4:off + 8], "little")
        if chunk_id == b"data":
            return size > 0
        off += 8 + size + (size % 2)      # RIFF chunks are word-aligned
    return True

Walk the chunks rather than assuming the data chunk starts at byte 36 — a 44-byte header is the common case, not a rule.

When you cannot read the header, treat the frame as audio. Guessing wrong in that direction costs you one odd frame; guessing wrong in the other throws away speech your listener was waiting for.

Raw codec streams (pcm, ulaw, mp3, opus — X-Stream-Format: len32-frames) never carry heartbeats, because injecting a fake WAV into a codec bitstream would be injecting garbage. If you stream those, every frame is audio.

Formats

Pass outputFormat (and optionally sampleRate) to change the payload encoding:

{ "text": "…", "voiceId": "Ngọc Lan", "outputFormat": "ulaw" }

Format	Notes
`wav`	Default. Each frame is a self-contained file, 48 kHz.
`pcm`	Raw signed 16-bit little-endian, no header — read the rate from `X-Sample-Rate`.
`ulaw`	Raw G.711 mu-law, always 8 kHz. What a phone line wants.
`mp3`, `opus`	Available, but see the warning below.

Valid sampleRate values are 8000, 16000, 22050, 24000, 44100 and 48000. The framing never changes, whatever the encoding — so the zero-length terminator means the same thing in all of them.

mp3 and opus frames do not join cleanly

Every frame is encoded as a standalone file, so decode each frame separately — do not concatenate them.

Concatenated mp3 gains about 24 ms of silence at each seam (the encoder delay, re-applied per frame) plus an audible click. Concatenated opus is a chain of complete Ogg streams: ffmpeg reads it, most browsers stop at the first frame.

If you want one continuous mp3 or opus body rather than frames, request the whole file without streaming — the synchronous path encodes it in one pass and has none of these artifacts. /v1/audio/speech's stream_format therefore accepts only pcm and ulaw.

How a stream ends

Five outcomes, and they are not all failures. Your integration should tell them apart, because three of them mean "try again" and two do not.

What you see	What it means	Billed?
Frames, then a zero-length frame, at least one frame carrying samples	Success	yes
Frames, then a zero-length frame, but every frame was a heartbeat	Synthesis produced nothing	no — refunded
The body just stops, no zero-length frame	Cut short mid-generation	no — refunded
`503` with `"code": "STREAM_BUSY"`	Every node is healthy but streaming capacity is used up	no — nothing was charged
`502`	Every node for this engine failed	no — refunded

STREAM_BUSY carries "fallback": "generate", and that is a real instruction: the queued POST /v1/tts path has separate capacity and will accept the work right now. Retrying the stream immediately usually will not.

A stream that reaches us but produces no audio for 120 seconds is abandoned server-side and ends without the zero-length frame — so it arrives as the "cut short" row above. That ceiling exists so a wedged node cannot hold your connection open indefinitely.

If your client disconnects

You are charged. Closing the connection part-way — a user pressing Stop, a timeout on your side, a crashed worker of your own — bills the request. You received audio; we generated it.

This is deliberate, and it is the one case where "no zero-length frame" does not mean a refund. If you retry after aborting, budget for both attempts.

Limits

Limit	Value	On exceeding
Concurrent streams	4	`429`, `code: STREAM_CONCURRENCY`
Synthesis requests per minute	300	`429`, honour `Retry-After`
Text length	50 000 characters	`400`

Concurrency is counted against the token grant behind your key, not the key itself, so several keys issued on one account normally share the four rather than each getting four of their own. The slot is taken before the balance is touched: a stream refused here costs nothing.

The stream route has a much tighter connection cap

POST /v1/tts/stream is matched by its own nginx location — the one that turns response buffering off, without which the whole point of streaming is lost. That location does not inherit /api/v1's generous per-address connection allowance of 300; it falls back to the server-wide one of 20 concurrent connections per source IP, and to a per-address request rate of 500/min rather than 3000.

A stream holds its connection open for the whole synthesis, so those 20 are held, not cycled. Twenty concurrent streams from one egress address is a realistic number for a busy integration, and the twenty-first is refused by nginx with a bare 503 — an HTML error page, no Retry-After, no X-RateLimit-*, and none of the JSON a STREAM_BUSY carries. That absence is how you tell the two 503s apart: ours has a body and a "fallback" you can act on, the edge's has neither. It is also counted across everyone behind that address, not just you. See Rate limits.

Cloned voices

Pass a clone_… id as voiceId and it streams like any other voice — the same frames, the same terminator, the same headers:

{ "text": "…", "voiceId": "clone_9f2c1e04-…" }

The ids come from GET /v1/voices with your API key (they are listed alongside the catalogue, tagged "kind": "cloned"). You create the voice itself in the web Studio at vieneu.io/#/clone — the API no longer enrols voices — and it is usable here the moment it is saved. Both your own clones and any an administrator has published to the catalogue work.

It costs the same as a preset. Streaming is billed per submitted character × the engine's multiplier, and cloning adds no multiplier of its own. Enrolment is charged in the Studio, not here, so a stream with a clone_… voice costs exactly what the same text costs with a catalogue voice.

Clones enrolled before 2026-08-28

Every clone created since 2026-08-28 enrols on v4, and those stream exactly as described above. A clone enrolled earlier was cut for v3, and v3 was retired from the cloud API on 2026-09-24: such a voice is no longer listed by GET /v1/voices and cannot be rendered through the API on any route — the only engine that could read its reference clip is the one that now answers 400. Re-enrol it in the Studio; the new clone_… id streams like any other v4 voice. Audio you generated with the old voice before the retirement is unaffected.

Two failures are worth distinguishing, and both arrive as 400 before anything is billed:

Message	What happened
`Cloned voice "…" was not found among your voices.`	Wrong id, or a clone belonging to another account
`Cloned voice "…" is no longer available.`	The voice exists but its reference clip is gone — usually deleted mid-request

Reference decoder — Python

import struct, requests

def stream_frames(text, voice, api_key, output_format="wav"):
    """Yield each audio frame. Raises if the stream was truncated."""
    resp = requests.post(
        "https://api.vieneu.io/api/v1/tts/stream",
        headers={"Authorization": f"Bearer {api_key}"},
        json={"text": text, "voiceId": voice, "outputFormat": output_format},
        stream=True,
    )
    resp.raise_for_status()
    print("sample rate:", resp.headers.get("X-Sample-Rate"))

    buf, complete, any_audio = bytearray(), False, False
    for chunk in resp.iter_content(chunk_size=8192):
        buf.extend(chunk)
        # A frame may span chunks, and several may arrive in one.
        while len(buf) >= 4:
            (length,) = struct.unpack(">I", buf[:4])
            if length == 0:                 # end-of-stream marker
                complete = True
                del buf[:4]
                continue
            if len(buf) < 4 + length:       # frame not all here yet
                break
            frame = bytes(buf[4:4 + length])
            del buf[:4 + length]
            if has_samples(frame):          # skip heartbeats — see above
                any_audio = True
                yield frame

    if not complete:
        raise RuntimeError("stream truncated — discard this audio")
    if not any_audio:
        # Kết thúc sạch nhưng không một mẫu nào: chúng tôi cũng không tính tiền
        # ca này. Coi nó là thành công là ghi sổ lệch với hoá đơn.
        raise RuntimeError("stream carried no audio — not billed, do not save")

Reference decoder — JavaScript

/** False for a heartbeat: a valid WAV whose `data` chunk is empty. */
function hasSamples(frame) {
  if (frame.length === 0) return false;    // nothing there at all
  if (frame.length < 12) return true;      // too short to read — assume audio
  const dv = new DataView(frame.buffer, frame.byteOffset, frame.byteLength);
  const tag = (o) => String.fromCharCode(...frame.subarray(o, o + 4));
  if (tag(0) !== 'RIFF' || tag(8) !== 'WAVE') return true; // not WAV — assume audio
  let off = 12;
  while (off + 8 <= frame.length) {
    const size = dv.getUint32(off + 4, true);
    if (tag(off) === 'data') return size > 0;
    off += 8 + size + (size % 2);          // RIFF chunks are word-aligned
  }
  return true;
}

async function* streamFrames(text, voice, apiKey, outputFormat = 'wav') {
  const resp = await fetch('https://api.vieneu.io/api/v1/tts/stream', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ text, voiceId: voice, outputFormat }),
  });
  if (!resp.ok) throw new Error(`HTTP ${resp.status}`);

  const reader = resp.body.getReader();
  let buf = new Uint8Array(0);
  let complete = false;
  let anyAudio = false;

  for (;;) {
    const { done, value } = await reader.read();
    if (done) break;

    const next = new Uint8Array(buf.length + value.length);
    next.set(buf);
    next.set(value, buf.length);
    buf = next;

    for (;;) {
      if (buf.length < 4) break;
      const length = new DataView(buf.buffer, buf.byteOffset, 4).getUint32(0);
      if (length === 0) {          // end-of-stream marker
        complete = true;
        buf = buf.subarray(4);
        continue;
      }
      if (buf.length < 4 + length) break;
      const frame = buf.slice(4, 4 + length);
      buf = buf.subarray(4 + length);
      if (hasSamples(frame)) { anyAudio = true; yield frame; }  // skip heartbeats
    }
  }

  if (!complete) throw new Error('stream truncated — discard this audio');
  // Kết thúc sạch nhưng không một mẫu nào: chúng tôi cũng không tính tiền ca
  // này. Coi nó là thành công là ghi sổ lệch với hoá đơn.
  if (!anyAudio) throw new Error('stream carried no audio — not billed, do not save');
}

With the default WAV framing, each yielded frame is a complete file — in a browser you can feed them straight to decodeAudioData and queue the results. The hasSamples guard above is what makes that safe: without it a heartbeat decodes to a zero-length buffer and quietly joins the queue.

OpenAI-style streaming

If you are driving this from an OpenAI client, skip the framing entirely:

with client.audio.speech.with_streaming_response.create(
    model="tts-1", voice="Ngọc Lan", input="…", response_format="pcm",
    extra_body={"stream_format": "audio"},
) as resp:
    resp.stream_to_file("speech.pcm")   # raw s16le, 24 kHz

stream_format: "audio" gives you the audio bytes as chunked transfer encoding — append them and you have the whole thing. stream_format: "sse" gives Server-Sent Events: speech.audio.delta events carrying base64 audio, ending in exactly one speech.audio.done or speech.audio.error.

As with the native stream, speech.audio.done is the proof the audio is whole; if it never arrives, discard what you have — you were not billed.

stream_format accepts pcm and ulaw only, for the reason in the warning above: the other formats cannot be concatenated into a playable result. For a complete mp3 or opus file, make an ordinary non-streaming request.

Latency, honestly

On the standard plans — Starter through Hội viên, everything that shares the pooled lane — first audio lands in roughly 1–2 seconds end to end. That is fast enough for read-aloud, dubbing, IVR prompts and assistants that tolerate a beat before speaking. It is not the 200–300 ms class that hard real-time conversational agents expect, and no amount of client tuning moves it: the wait is the queue plus the opener chunk, not your connection.

Enterprise is the exception, by contract rather than by luck. That tier runs on a dedicated GPU node with a lane nobody else shares, so the latency ceiling becomes something we agree on — down to ≤ 250 ms — instead of whatever the pool happens to be doing. It is a ceiling written into the deal and sized to your traffic, not a number you can read off the shared endpoint and expect to hold. If you are building a real-time agent, talk to us before you design around the 1–2 second figure above.

The native stream​

The zero-length frame is the point​

Heartbeat frames​

Formats​

How a stream ends​

If your client disconnects​

Limits​

Cloned voices​

Reference decoder — Python​

Reference decoder — JavaScript​

OpenAI-style streaming​

Latency, honestly​