# Streaming Source: https://docs.vieneu.io/docs/cloud-api/streaming Long text takes as long to synthesize as it takes. Streaming lets playback start on the first sentence — typically **1–2 seconds** on the standard plans, or a ceiling agreed per contract on Enterprise — instead of after the last one. :::info Engine — and what it costs Streaming runs on **`v4`**. It always did — `v3` delivered its chunks in a burst at the end, so a stream request naming it was refused rather than served under a time-to-first-audio promise it could not keep — and since **2026-09-24** `v4` is the only engine on the cloud API at all, so there is nothing to choose: omit `engine`, or send `v4`. Naming `v3` on this route now gets the same `400` as on every other: `Engine "v3" was retired on 2026-09-24. Use engine "v4" and a voice from GET /v1/voices?engine=v4 — the two catalogues share no ids.` **The price is the `v4` price.** A stream is billed per submitted character at `v4`'s multiplier — today **3×**, times the platform-wide `globalMultiplier` — exactly like a `POST /v1/tts` job for the same text. The old comparison, a stream at `v4`'s rate against a job on `v3`'s 1.5×, no longer means anything, because the 1.5× rate is gone from every route; if you sized a budget against it, resize it. `GET /v1/engines` returns the live multipliers, no key required — see [Engines](./overview.md#engines). This applies to cloned voices as well. Clones have enrolled on `v4` since 2026-08-28, so they stream like any other `v4` voice; an older `v3`-enrolled clone has to be re-enrolled first. See [Cloned voices](#cloned-voices). ::: There are two streaming endpoints, and which you want depends on who is reading the bytes: - **`POST /v1/audio/speech` with `stream_format`** — plain audio or SSE, and what an OpenAI client already knows how to read. Start here. - **`POST /v1/tts/stream`** — VieNeu's native framed stream. Use it when you want each chunk delivered as a separately decodable unit, or when you want a positive signal that the stream finished. ## The native stream ```bash curl -N -X POST https://api.vieneu.io/api/v1/tts/stream \ -H "Authorization: Bearer $VIENEU_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "text": "…", "voiceId": "Ngọc Lan" }' \ --output stream.bin ``` The response body is a sequence of length-prefixed frames: ``` [4-byte big-endian uint32 = N][N bytes of audio] … repeated … [4-byte big-endian uint32 = 0] ← end of stream ``` Read a length, read that many bytes, repeat. By default each payload is a **self-contained WAV**, so you can hand a frame straight to a decoder without waiting for the rest. Response headers tell you what you actually got: | Header | Meaning | |---|---| | `X-Sample-Rate` | Sample rate of the audio, in Hz | | `X-Output-Format` | The encoding: `wav`, `mp3`, `opus`, `pcm` or `ulaw` | | `X-Stream-Format` | `len32-wav-chunks`, or `len32-frames` for other encodings | ### The zero-length frame is the point A stream that finishes sends a final frame with length `0`. A stream cut short by a failure mid-generation simply **stops**, with no such frame. Its absence means the stream was cut short: discard the audio. You are not billed for a stream that never sent it — the refund is automatic. **But the marker alone is not success.** A stream can arrive complete and still carry no speech — a generation that produced nothing sends its heartbeats (see below) and then a clean end-of-stream. We do not bill those either, so a caller who treats the marker as sufficient will book a success, save a silent file, and end the month reconciling against an invoice that never charged for it. The condition we bill on, and the one you should use, is both halves: > **the end-of-stream marker arrived, AND at least one frame carried samples.** ### Heartbeat frames A stream that goes quiet for a while sends a **heartbeat**: a valid WAV file containing **zero samples**, 44 bytes on the wire. Its only job is to keep the connection from being closed for inactivity by whatever sits between us — a proxy, a corporate gateway, a mobile carrier's NAT. Heartbeats appear **only on WAV streams** (`X-Stream-Format: len32-wav-chunks`), at most one every `X-Stream-Heartbeat` seconds, and only while the synthesizer has produced nothing new. A stream that flows normally never sends one. **You must skip them.** The trap is that a heartbeat is a *well-formed* WAV, so a decoder will not reject it — `decodeAudioData` returns a zero-length buffer rather than throwing, and writing the frame to a file leaves a stray 44-byte header in the middle of your audio. Checking that a frame is non-empty is not enough; ask whether it carries samples: ```python def has_samples(frame: bytes) -> bool: """False for a heartbeat: a valid WAV whose `data` chunk is empty.""" if not frame: return False # nothing there at all if len(frame) < 12 or frame[:4] != b"RIFF" or frame[8:12] != b"WAVE": return True # not a WAV we can read — assume audio off = 12 while off + 8 <= len(frame): chunk_id = frame[off:off + 4] size = int.from_bytes(frame[off + 4:off + 8], "little") if chunk_id == b"data": return size > 0 off += 8 + size + (size % 2) # RIFF chunks are word-aligned return True ``` Walk the chunks rather than assuming the `data` chunk starts at byte 36 — a 44-byte header is the common case, not a rule. When you cannot read the header, treat the frame as audio. Guessing wrong in that direction costs you one odd frame; guessing wrong in the other throws away speech your listener was waiting for. Raw codec streams (`pcm`, `ulaw`, `mp3`, `opus` — `X-Stream-Format: len32-frames`) never carry heartbeats, because injecting a fake WAV into a codec bitstream would be injecting garbage. If you stream those, every frame is audio. ### Formats Pass `outputFormat` (and optionally `sampleRate`) to change the payload encoding: ```json { "text": "…", "voiceId": "Ngọc Lan", "outputFormat": "ulaw" } ``` | Format | Notes | |---|---| | `wav` | Default. Each frame is a self-contained file, 48 kHz. | | `pcm` | Raw signed 16-bit little-endian, **no header** — read the rate from `X-Sample-Rate`. | | `ulaw` | Raw G.711 mu-law, always 8 kHz. What a phone line wants. | | `mp3`, `opus` | Available, but see the warning below. | Valid `sampleRate` values are 8000, 16000, 22050, 24000, 44100 and 48000. The framing never changes, whatever the encoding — so the zero-length terminator means the same thing in all of them. :::warning mp3 and opus frames do not join cleanly Every frame is encoded as a standalone file, so **decode each frame separately** — do not concatenate them. Concatenated mp3 gains about 24 ms of silence at each seam (the encoder delay, re-applied per frame) plus an audible click. Concatenated opus is a chain of complete Ogg streams: ffmpeg reads it, most browsers stop at the first frame. If you want one continuous mp3 or opus body rather than frames, request the whole file without streaming — the synchronous path encodes it in one pass and has none of these artifacts. `/v1/audio/speech`'s `stream_format` therefore accepts only `pcm` and `ulaw`. ::: ## How a stream ends Five outcomes, and they are not all failures. Your integration should tell them apart, because three of them mean "try again" and two do not. | What you see | What it means | Billed? | |---|---|---| | Frames, then a zero-length frame, at least one frame carrying samples | Success | **yes** | | Frames, then a zero-length frame, but every frame was a heartbeat | Synthesis produced nothing | no — refunded | | The body just stops, no zero-length frame | Cut short mid-generation | no — refunded | | `503` with `"code": "STREAM_BUSY"` | Every node is healthy but streaming capacity is used up | no — nothing was charged | | `502` | Every node for this engine failed | no — refunded | `STREAM_BUSY` carries `"fallback": "generate"`, and that is a real instruction: the queued `POST /v1/tts` path has separate capacity and will accept the work right now. Retrying the stream immediately usually will not. A stream that reaches us but produces no audio for **120 seconds** is abandoned server-side and ends without the zero-length frame — so it arrives as the "cut short" row above. That ceiling exists so a wedged node cannot hold your connection open indefinitely. ### If your client disconnects **You are charged.** Closing the connection part-way — a user pressing Stop, a timeout on your side, a crashed worker of your own — bills the request. You received audio; we generated it. This is deliberate, and it is the one case where "no zero-length frame" does not mean a refund. If you retry after aborting, budget for both attempts. ### Limits | Limit | Value | On exceeding | |---|---|---| | Concurrent streams | 4 | `429`, `code: STREAM_CONCURRENCY` | | Synthesis requests per minute | 300 | `429`, honour `Retry-After` | | Text length | 50 000 characters | `400` | Concurrency is counted against the **token grant** behind your key, not the key itself, so several keys issued on one account normally share the four rather than each getting four of their own. The slot is taken before the balance is touched: a stream refused here costs nothing. :::caution The stream route has a much tighter connection cap `POST /v1/tts/stream` is matched by its own nginx location — the one that turns response buffering off, without which the whole point of streaming is lost. That location does not inherit `/api/v1`'s generous per-address connection allowance of 300; it falls back to the server-wide one of **20 concurrent connections per source IP**, and to a per-address request rate of 500/min rather than 3000. A stream holds its connection open for the whole synthesis, so those 20 are held, not cycled. Twenty concurrent streams from one egress address is a realistic number for a busy integration, and the twenty-first is refused by nginx with a bare `503` — an HTML error page, no `Retry-After`, no `X-RateLimit-*`, and none of the JSON a `STREAM_BUSY` carries. That absence is how you tell the two `503`s apart: ours has a body and a `"fallback"` you can act on, the edge's has neither. It is also counted across everyone behind that address, not just you. See [Rate limits](./overview.md#rate-limits-and-request-ids). ::: ## Cloned voices Pass a `clone_…` id as `voiceId` and it streams like any other voice — the same frames, the same terminator, the same headers: ```json { "text": "…", "voiceId": "clone_9f2c1e04-…" } ``` The ids come from `GET /v1/voices` with your API key (they are listed alongside the catalogue, tagged `"kind": "cloned"`). You create the voice itself in the web Studio at [vieneu.io/#/clone](https://vieneu.io/#/clone) — the API no longer enrols voices — and it is usable here the moment it is saved. Both your own clones and any an administrator has published to the catalogue work. **It costs the same as a preset.** Streaming is billed per submitted character × the engine's multiplier, and cloning adds no multiplier of its own. Enrolment is charged in the Studio, not here, so a stream with a `clone_…` voice costs exactly what the same text costs with a catalogue voice. :::caution Clones enrolled before 2026-08-28 Every clone created since 2026-08-28 enrols on `v4`, and those stream exactly as described above. A clone enrolled earlier was cut for `v3`, and `v3` was retired from the cloud API on 2026-09-24: such a voice is no longer listed by `GET /v1/voices` and cannot be rendered through the API on any route — the only engine that could read its reference clip is the one that now answers `400`. Re-enrol it in the [Studio](https://vieneu.io/#/clone); the new `clone_…` id streams like any other `v4` voice. Audio you generated with the old voice before the retirement is unaffected. ::: Two failures are worth distinguishing, and both arrive as `400` **before anything is billed**: | Message | What happened | |---|---| | `Cloned voice "…" was not found among your voices.` | Wrong id, or a clone belonging to another account | | `Cloned voice "…" is no longer available.` | The voice exists but its reference clip is gone — usually deleted mid-request | ## Reference decoder — Python ```python import struct, requests def stream_frames(text, voice, api_key, output_format="wav"): """Yield each audio frame. Raises if the stream was truncated.""" resp = requests.post( "https://api.vieneu.io/api/v1/tts/stream", headers={"Authorization": f"Bearer {api_key}"}, json={"text": text, "voiceId": voice, "outputFormat": output_format}, stream=True, ) resp.raise_for_status() print("sample rate:", resp.headers.get("X-Sample-Rate")) buf, complete, any_audio = bytearray(), False, False for chunk in resp.iter_content(chunk_size=8192): buf.extend(chunk) # A frame may span chunks, and several may arrive in one. while len(buf) >= 4: (length,) = struct.unpack(">I", buf[:4]) if length == 0: # end-of-stream marker complete = True del buf[:4] continue if len(buf) < 4 + length: # frame not all here yet break frame = bytes(buf[4:4 + length]) del buf[:4 + length] if has_samples(frame): # skip heartbeats — see above any_audio = True yield frame if not complete: raise RuntimeError("stream truncated — discard this audio") if not any_audio: # Kết thúc sạch nhưng không một mẫu nào: chúng tôi cũng không tính tiền # ca này. Coi nó là thành công là ghi sổ lệch với hoá đơn. raise RuntimeError("stream carried no audio — not billed, do not save") ``` ## Reference decoder — JavaScript ```js /** False for a heartbeat: a valid WAV whose `data` chunk is empty. */ function hasSamples(frame) { if (frame.length === 0) return false; // nothing there at all if (frame.length < 12) return true; // too short to read — assume audio const dv = new DataView(frame.buffer, frame.byteOffset, frame.byteLength); const tag = (o) => String.fromCharCode(...frame.subarray(o, o + 4)); if (tag(0) !== 'RIFF' || tag(8) !== 'WAVE') return true; // not WAV — assume audio let off = 12; while (off + 8 <= frame.length) { const size = dv.getUint32(off + 4, true); if (tag(off) === 'data') return size > 0; off += 8 + size + (size % 2); // RIFF chunks are word-aligned } return true; } async function* streamFrames(text, voice, apiKey, outputFormat = 'wav') { const resp = await fetch('https://api.vieneu.io/api/v1/tts/stream', { method: 'POST', headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json', }, body: JSON.stringify({ text, voiceId: voice, outputFormat }), }); if (!resp.ok) throw new Error(`HTTP ${resp.status}`); const reader = resp.body.getReader(); let buf = new Uint8Array(0); let complete = false; let anyAudio = false; for (;;) { const { done, value } = await reader.read(); if (done) break; const next = new Uint8Array(buf.length + value.length); next.set(buf); next.set(value, buf.length); buf = next; for (;;) { if (buf.length < 4) break; const length = new DataView(buf.buffer, buf.byteOffset, 4).getUint32(0); if (length === 0) { // end-of-stream marker complete = true; buf = buf.subarray(4); continue; } if (buf.length < 4 + length) break; const frame = buf.slice(4, 4 + length); buf = buf.subarray(4 + length); if (hasSamples(frame)) { anyAudio = true; yield frame; } // skip heartbeats } } if (!complete) throw new Error('stream truncated — discard this audio'); // Kết thúc sạch nhưng không một mẫu nào: chúng tôi cũng không tính tiền ca // này. Coi nó là thành công là ghi sổ lệch với hoá đơn. if (!anyAudio) throw new Error('stream carried no audio — not billed, do not save'); } ``` With the default WAV framing, each yielded frame is a complete file — in a browser you can feed them straight to `decodeAudioData` and queue the results. The `hasSamples` guard above is what makes that safe: without it a heartbeat decodes to a zero-length buffer and quietly joins the queue. ## OpenAI-style streaming If you are driving this from an OpenAI client, skip the framing entirely: ```python with client.audio.speech.with_streaming_response.create( model="tts-1", voice="Ngọc Lan", input="…", response_format="pcm", extra_body={"stream_format": "audio"}, ) as resp: resp.stream_to_file("speech.pcm") # raw s16le, 24 kHz ``` `stream_format: "audio"` gives you the audio bytes as chunked transfer encoding — append them and you have the whole thing. `stream_format: "sse"` gives Server-Sent Events: `speech.audio.delta` events carrying base64 audio, ending in exactly one `speech.audio.done` or `speech.audio.error`. As with the native stream, **`speech.audio.done` is the proof the audio is whole**; if it never arrives, discard what you have — you were not billed. `stream_format` accepts `pcm` and `ulaw` only, for the reason in the warning above: the other formats cannot be concatenated into a playable result. For a complete mp3 or opus file, make an ordinary non-streaming request. ## Latency, honestly **On the standard plans** — Starter through Hội viên, everything that shares the pooled lane — first audio lands in roughly **1–2 seconds** end to end. That is fast enough for read-aloud, dubbing, IVR prompts and assistants that tolerate a beat before speaking. It is **not** the 200–300 ms class that hard real-time conversational agents expect, and no amount of client tuning moves it: the wait is the queue plus the opener chunk, not your connection. **Enterprise is the exception, by contract rather than by luck.** That tier runs on a dedicated GPU node with a lane nobody else shares, so the latency ceiling becomes something we agree on — down to **≤ 250 ms** — instead of whatever the pool happens to be doing. It is a ceiling written into the deal and sized to your traffic, not a number you can read off the shared endpoint and expect to hold. If you are building a real-time agent, talk to us before you design around the 1–2 second figure above.