Streaming

Long text takes as long to synthesize as it takes. Streaming lets playback start on the first sentence — typically 1–2 seconds — instead of after the last one.

Engine — and what it costs

Streaming runs on v4 only — it is the default engine, so most callers need do nothing. A stream request naming v3 is rejected: that engine delivers its chunks in a burst at the end, so serving it here would mean making a time-to-first-audio promise it cannot keep.

That has a price consequence. Every engine is billed at its own multiplier, and because streaming is pinned to v4, a stream is charged at v4's rate — twice what the same text costs on v3 today (3× vs 1.5×). If you sized your budget with POST /v1/tts on v3, streaming the same workload costs double. GET /v1/engines returns the live multipliers, no key required — see Engines.

This applies to cloned voices as well. Clones have enrolled on v4 since 2026-08-28, so they stream like any other v4 voice. See Cloned voices.

There are two streaming endpoints, and which you want depends on who is reading the bytes:

POST /v1/audio/speech with stream_format — plain audio or SSE, and what an OpenAI client already knows how to read. Start here.
POST /v1/tts/stream — VieNeu's native framed stream. Use it when you want each chunk delivered as a separately decodable unit, or when you want a positive signal that the stream finished.

The native stream

curl -N -X POST https://api.vieneu.io/api/v1/tts/stream \
  -H "Authorization: Bearer $VIENEU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "text": "…", "voiceId": "Ngọc Lan" }' \
  --output stream.bin

The response body is a sequence of length-prefixed frames:

[4-byte big-endian uint32 = N][N bytes of audio]   … repeated …
[4-byte big-endian uint32 = 0]                     ← end of stream

Read a length, read that many bytes, repeat. By default each payload is a self-contained WAV, so you can hand a frame straight to a decoder without waiting for the rest.

Response headers tell you what you actually got:

Header	Meaning
`X-Sample-Rate`	Sample rate of the audio, in Hz
`X-Output-Format`	The encoding: `wav`, `mp3`, `opus`, `pcm` or `ulaw`
`X-Stream-Format`	`len32-wav-chunks`, or `len32-frames` for other encodings

The zero-length frame is the point

A stream that finishes sends a final frame with length 0. A stream cut short by a failure mid-generation simply stops, with no such frame.

Its absence means the stream was cut short: discard the audio. You are not billed for a stream that never sent it — the refund is automatic.

But the marker alone is not success. A stream can arrive complete and still carry no speech — a generation that produced nothing sends its heartbeats (see below) and then a clean end-of-stream. We do not bill those either, so a caller who treats the marker as sufficient will book a success, save a silent file, and end the month reconciling against an invoice that never charged for it.

The condition we bill on, and the one you should use, is both halves:

the end-of-stream marker arrived, AND at least one frame carried samples.

Heartbeat frames

A stream that goes quiet for a while sends a heartbeat: a valid WAV file containing zero samples, 44 bytes on the wire. Its only job is to keep the connection from being closed for inactivity by whatever sits between us — a proxy, a corporate gateway, a mobile carrier's NAT.

Heartbeats appear only on WAV streams (X-Stream-Format: len32-wav-chunks), at most one every X-Stream-Heartbeat seconds, and only while the synthesizer has produced nothing new. A stream that flows normally never sends one.

You must skip them. The trap is that a heartbeat is a well-formed WAV, so a decoder will not reject it — decodeAudioData returns a zero-length buffer rather than throwing, and writing the frame to a file leaves a stray 44-byte header in the middle of your audio. Checking that a frame is non-empty is not enough; ask whether it carries samples:

def has_samples(frame: bytes) -> bool:
    """False for a heartbeat: a valid WAV whose `data` chunk is empty."""
    if not frame:
        return False                     # nothing there at all
    if len(frame) < 12 or frame[:4] != b"RIFF" or frame[8:12] != b"WAVE":
        return True                      # not a WAV we can read — assume audio
    off = 12
    while off + 8 <= len(frame):
        chunk_id = frame[off:off + 4]
        size = int.from_bytes(frame[off + 4:off + 8], "little")
        if chunk_id == b"data":
            return size > 0
        off += 8 + size + (size % 2)      # RIFF chunks are word-aligned
    return True

Walk the chunks rather than assuming the data chunk starts at byte 36 — a 44-byte header is the common case, not a rule.

When you cannot read the header, treat the frame as audio. Guessing wrong in that direction costs you one odd frame; guessing wrong in the other throws away speech your listener was waiting for.

Raw codec streams (pcm, ulaw, mp3, opus — X-Stream-Format: len32-frames) never carry heartbeats, because injecting a fake WAV into a codec bitstream would be injecting garbage. If you stream those, every frame is audio.

Formats

Pass outputFormat (and optionally sampleRate) to change the payload encoding:

{ "text": "…", "voiceId": "Ngọc Lan", "outputFormat": "ulaw" }

Format	Notes
`wav`	Default. Each frame is a self-contained file, 48 kHz.
`pcm`	Raw signed 16-bit little-endian, no header — read the rate from `X-Sample-Rate`.
`ulaw`	Raw G.711 mu-law, always 8 kHz. What a phone line wants.
`mp3`, `opus`	Available, but see the warning below.

Valid sampleRate values are 8000, 16000, 22050, 24000, 44100 and 48000. The framing never changes, whatever the encoding — so the zero-length terminator means the same thing in all of them.

mp3 and opus frames do not join cleanly

Every frame is encoded as a standalone file, so decode each frame separately — do not concatenate them.

Concatenated mp3 gains about 24 ms of silence at each seam (the encoder delay, re-applied per frame) plus an audible click. Concatenated opus is a chain of complete Ogg streams: ffmpeg reads it, most browsers stop at the first frame.

If you want one continuous mp3 or opus body rather than frames, request the whole file without streaming — the synchronous path encodes it in one pass and has none of these artifacts. /v1/audio/speech's stream_format therefore accepts only pcm and ulaw.

How a stream ends

Five outcomes, and they are not all failures. Your integration should tell them apart, because three of them mean "try again" and two do not.

What you see	What it means	Billed?
Frames, then a zero-length frame, at least one frame carrying samples	Success	yes
Frames, then a zero-length frame, but every frame was a heartbeat	Synthesis produced nothing	no — refunded
The body just stops, no zero-length frame	Cut short mid-generation	no — refunded
`503` with `"code": "STREAM_BUSY"`	Every node is healthy but streaming capacity is used up	no — nothing was charged
`502`	Every node for this engine failed	no — refunded

STREAM_BUSY carries "fallback": "generate", and that is a real instruction: the queued POST /v1/tts path has separate capacity and will accept the work right now. Retrying the stream immediately usually will not.

A stream that reaches us but produces no audio for 120 seconds is abandoned server-side and ends without the zero-length frame — so it arrives as the "cut short" row above. That ceiling exists so a wedged node cannot hold your connection open indefinitely.

If your client disconnects

You are charged. Closing the connection part-way — a user pressing Stop, a timeout on your side, a crashed worker of your own — bills the request. You received audio; we generated it.

This is deliberate, and it is the one case where "no zero-length frame" does not mean a refund. If you retry after aborting, budget for both attempts.

Limits

Limit	Value	On exceeding
Concurrent streams per API key	4	`429`
Synthesis requests per minute	300	`429`, honour `Retry-After`
Text length	50 000 characters	`400`

Concurrency is counted per key, not per token grant — two keys on the same account each get their own allowance.

Cloned voices

Pass a clone_… id as voiceId and it streams like any other voice — the same frames, the same terminator, the same headers:

{ "text": "…", "voiceId": "clone_9f2c1e04-…" }

The ids come from GET /v1/voices with your API key (they are listed alongside the catalogue, tagged "kind": "cloned"), and you create them with POST /v1/voices. Both your own clones and any an administrator has published to the catalogue work.

It costs the same as a preset. Streaming is billed per submitted character × the engine's multiplier, and cloning adds no multiplier of its own. The only clone-specific charge in the API is the one-off enrolment fee on POST /v1/voices.

Clones enrolled before 2026-08-28

The engine you name is the engine that renders, and a clone is not currently checked against the engine it was enrolled on. Every clone created since 2026-08-28 enrols on v4, so for those the two always agree and there is nothing to think about. But if you still own an older v3-enrolled clone, naming v4 (or omitting engine, since v4 is the default) will render it on v4 from a reference clip cut for v3 — you will get audio that does not sound like your voice, billed at v4's multiplier. Re-enrol such a voice with POST /v1/voices rather than streaming it. We intend to reject the mismatch outright in a future release.

Two failures are worth distinguishing, and both arrive as 400 before anything is billed:

Message	What happened
`Cloned voice "…" was not found among your voices.`	Wrong id, or a clone belonging to another account
`Cloned voice "…" is no longer available.`	The voice exists but its reference clip is gone — usually deleted mid-request

Reference decoder — Python

import struct, requests

def stream_frames(text, voice, api_key, output_format="wav"):
    """Yield each audio frame. Raises if the stream was truncated."""
    resp = requests.post(
        "https://api.vieneu.io/api/v1/tts/stream",
        headers={"Authorization": f"Bearer {api_key}"},
        json={"text": text, "voiceId": voice, "outputFormat": output_format},
        stream=True,
    )
    resp.raise_for_status()
    print("sample rate:", resp.headers.get("X-Sample-Rate"))

    buf, complete, any_audio = bytearray(), False, False
    for chunk in resp.iter_content(chunk_size=8192):
        buf.extend(chunk)
        # A frame may span chunks, and several may arrive in one.
        while len(buf) >= 4:
            (length,) = struct.unpack(">I", buf[:4])
            if length == 0:                 # end-of-stream marker
                complete = True
                del buf[:4]
                continue
            if len(buf) < 4 + length:       # frame not all here yet
                break
            frame = bytes(buf[4:4 + length])
            del buf[:4 + length]
            if has_samples(frame):          # skip heartbeats — see above
                any_audio = True
                yield frame

    if not complete:
        raise RuntimeError("stream truncated — discard this audio")
    if not any_audio:
        # Kết thúc sạch nhưng không một mẫu nào: chúng tôi cũng không tính tiền
        # ca này. Coi nó là thành công là ghi sổ lệch với hoá đơn.
        raise RuntimeError("stream carried no audio — not billed, do not save")

Reference decoder — JavaScript

/** False for a heartbeat: a valid WAV whose `data` chunk is empty. */
function hasSamples(frame) {
  if (frame.length === 0) return false;    // nothing there at all
  if (frame.length < 12) return true;      // too short to read — assume audio
  const dv = new DataView(frame.buffer, frame.byteOffset, frame.byteLength);
  const tag = (o) => String.fromCharCode(...frame.subarray(o, o + 4));
  if (tag(0) !== 'RIFF' || tag(8) !== 'WAVE') return true; // not WAV — assume audio
  let off = 12;
  while (off + 8 <= frame.length) {
    const size = dv.getUint32(off + 4, true);
    if (tag(off) === 'data') return size > 0;
    off += 8 + size + (size % 2);          // RIFF chunks are word-aligned
  }
  return true;
}

async function* streamFrames(text, voice, apiKey, outputFormat = 'wav') {
  const resp = await fetch('https://api.vieneu.io/api/v1/tts/stream', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${apiKey}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ text, voiceId: voice, outputFormat }),
  });
  if (!resp.ok) throw new Error(`HTTP ${resp.status}`);

  const reader = resp.body.getReader();
  let buf = new Uint8Array(0);
  let complete = false;
  let anyAudio = false;

  for (;;) {
    const { done, value } = await reader.read();
    if (done) break;

    const next = new Uint8Array(buf.length + value.length);
    next.set(buf);
    next.set(value, buf.length);
    buf = next;

    for (;;) {
      if (buf.length < 4) break;
      const length = new DataView(buf.buffer, buf.byteOffset, 4).getUint32(0);
      if (length === 0) {          // end-of-stream marker
        complete = true;
        buf = buf.subarray(4);
        continue;
      }
      if (buf.length < 4 + length) break;
      const frame = buf.slice(4, 4 + length);
      buf = buf.subarray(4 + length);
      if (hasSamples(frame)) { anyAudio = true; yield frame; }  // skip heartbeats
    }
  }

  if (!complete) throw new Error('stream truncated — discard this audio');
  // Kết thúc sạch nhưng không một mẫu nào: chúng tôi cũng không tính tiền ca
  // này. Coi nó là thành công là ghi sổ lệch với hoá đơn.
  if (!anyAudio) throw new Error('stream carried no audio — not billed, do not save');
}

With the default WAV framing, each yielded frame is a complete file — in a browser you can feed them straight to decodeAudioData and queue the results. The hasSamples guard above is what makes that safe: without it a heartbeat decodes to a zero-length buffer and quietly joins the queue.

OpenAI-style streaming

If you are driving this from an OpenAI client, skip the framing entirely:

with client.audio.speech.with_streaming_response.create(
    model="tts-1", voice="Ngọc Lan", input="…", response_format="pcm",
    extra_body={"stream_format": "audio"},
) as resp:
    resp.stream_to_file("speech.pcm")   # raw s16le, 24 kHz

stream_format: "audio" gives you the audio bytes as chunked transfer encoding — append them and you have the whole thing. stream_format: "sse" gives Server-Sent Events: speech.audio.delta events carrying base64 audio, ending in exactly one speech.audio.done or speech.audio.error.

As with the native stream, speech.audio.done is the proof the audio is whole; if it never arrives, discard what you have — you were not billed.

stream_format accepts pcm and ulaw only, for the reason in the warning above: the other formats cannot be concatenated into a playable result. For a complete mp3 or opus file, make an ordinary non-streaming request.

Latency, honestly

First audio lands in roughly 1–2 seconds end to end. That is fast enough for read-aloud, dubbing, IVR prompts and assistants that tolerate a beat before speaking. It is not yet in the 200–300 ms class that hard real-time conversational agents expect. If you are building one of those, talk to us about where it sits on the roadmap rather than designing around the current number.

The native stream​

The zero-length frame is the point​

Heartbeat frames​

Formats​

How a stream ends​

If your client disconnects​

Limits​

Cloned voices​

Reference decoder — Python​

Reference decoder — JavaScript​

OpenAI-style streaming​

Latency, honestly​