marafx2007/WarmBreeze-TTS-2-gguf

Breeze-TTS-2 GGUF

GGUF conversion of BreezeBlue/Breeze-TTS-2 for the native Rust implementation in Liyulingyue/rust-model-inference.

Breeze TTS 2 is a 3.5B bilingual (English / Chinese) text-to-speech model that supports three inference modes through a single Python CLI:

• Plain TTS — synthesize speech from a text prompt
• Voice Design — drive voice characteristics via a natural-language --instruction (e.g. "温柔地说。") and a CFG scale (--cfg-scale)
• Voice Clone — clone a speaker's timbre from a reference 24 kHz WAV (--ref-audio) plus its transcript (--ref-text)

The model is a multi-module TTS stack: a Qwen3 backbone (1.4B) + T5Gemma2 text encoder (1B) + depth decoder (0.4B) + Mimi codec (0.1B), totalling ~3.5B parameters. The audio tokenizer is based on Qwen3-TTS.

Files

Each row holds a precision variant of the same Breeze-TTS-2 main model. All variants share the same breeze-tts-2-mmproj-F32.gguf audio codec projection. Files are independent — pick one main GGUF plus the codec.

File	Size	Quantization	Use case
breeze-tts-2-BF16.gguf	6.6 GB	BF16 (default)	Reference / maximum fidelity
breeze-tts-2-F16.gguf	6.6 GB	F16	Reference precision, identical size to BF16
breeze-tts-2-F32.gguf	13 GB	F32	CPU numerical-stability debug
breeze-tts-2-Q8_0.gguf	3.6 GB	Q8_0	Recommended deployment (~3.6× faster, ~36% smaller)
breeze-tts-2-Q4_0.gguf	2.1 GB	Q4_0	Maximum compression (output quality degrades — see Limitations)
breeze-tts-2-mmproj-F32.gguf	651 MB	F32 codec (always required)	Mimi audio codec + speaker embeddings

The main model has 28 backbone layers + 26 text encoder layers + 12 depth decoder layers, a hidden size of 2048 (backbone) / 1152 (text encoder) / 1024 (depth decoder), and a vocabulary of 262158 tokens (text) + 2051 audio codes. The codec projector uses the custom breeze_audio architecture and expects 24 kHz mono PCM16 WAV input.

Compatibility

The validated runtime is rust-model-inference after merged PR #64 (Breeze TTS 2 native inference). Built against commit 2f224d2 and later (the path that landed rope + activation SIMD plus the per-precision Breeze routing).

The model is not compatible with stock llama.cpp. The breeze and breeze_audio architectures are specific to the Rust implementation above. The audio codec projector requires the breeze_audio mmproj path which has no llama.cpp equivalent.

Rust Usage

git clone https://github.com/Liyulingyue/rust-model-inference.git
cd rust-model-inference
cargo build --release --bin rust-model-inference
BIN=target/release/rust-model-inference
MODEL=/path/to/breeze-tts-2-Q8_0.gguf
CODEC=/path/to/breeze-tts-2-mmproj-F32.gguf

Plain TTS

"$BIN" --tts \
  --model "$MODEL" \
  --mmproj "$CODEC" \
  --prompt "你好，这是一个声音合成测试。" \
  --out plain.wav --seed 42

Voice Design (instruction + CFG)

--instruction is a natural-language voice description (e.g. "温柔地说。", "愤怒地说。"). CFG defaults to 3 with a non-empty instruction, 1 otherwise. Explicit --cfg-scale 1 disables CFG.

"$BIN" --tts \
  --model "$MODEL" \
  --mmproj "$CODEC" \
  --prompt "你好。" \
  --instruction "温柔地说。" \
  --cfg-scale 3 \
  --out instruction.wav --seed 42

Voice Clone (reference audio + transcript)

--ref-audio (24 kHz mono PCM16 WAV) and --ref-text (the reference transcript, required) must be supplied together. Multi-channel input is mixed down automatically; non-24 kHz sample rates are resampled.

"$BIN" --tts \
  --model "$MODEL" \
  --mmproj "$CODEC" \
  --prompt "再见。" \
  --ref-audio plain.wav --ref-text "你好，这是一个声音合成测试。" \
  --out clone.wav --seed 42

Sampling defaults

--temperature 0.9 --top-k 50 --top-p 1 --seed 42; --temperature 0 selects greedy decoding. --threads N controls the rayon worker pool (default = physical cores). With seed fixed, sampling is reproducible.

Conversion

The converter is torch-free — it reads sharded BF16 / F32 safetensors through mmap and writes GGUF directly:

python3 tools/breeze/convert_breeze.py \
  /path/to/Breeze-TTS-2 \
  --out-dir /path/to/output \
  --quant q8_0 \
  --codec-quant f32

--quant accepts bf16, f16, f32, q8_0, q4_0. --codec-quant accepts f32 or q8_0. The script preserves BF16 / F32 source bytes for tensors that the Rust loader pins to F32 / BF16 (norms, codebook initialisation flags, eoi embeddings, codec_model.* snapshots) — those never participate in the chosen precision mode.

Alignment

End-to-end timing per main-model precision on a single prompt ("天气真好", seed 42, 4 threads, --max-tokens 64):

Precision	Frames	Time	RSS	Notes
BF16	35	38s	6.5 GB	Reference (bit-exact with cargo test --release --test nemotron_h_parity)
F16	35	21s	6.5 GB	F16×F32 AVX2 SIMD matmul
F32	35	65s	22.2 GB	AVX2+FMA matmul (no F16×F16 fallback)
Q8_0	35	11s	4.3 GB	Recommended (3.4× faster, 45% less RAM than BF16)
Q4_0	128	162s	3.4 GB	Output degrades — 4-bit noise propagates through TTS frame count; not recommended for production

BF16 / F16 / F32 outputs are bit-exact identical (md5 c19502ff...) — the underlying matmul kernels differ only in reduction-order ULP drift that the sampler absorbs. Q8_0 drifts by 1 ULP (v3 baseline md5 b051f3c1...), tracked as the maintainable reference in docs/TODO.md (TODO-006).

The BF16 main model round-trips byte-for-byte through the converter (bytes.chunks_exact(2).flat_map(|c| bf16->f32) is a lossless decode), so a model output loaded from breeze-tts-2-BF16.gguf is bit-exact with the original PyTorch BF16 checkpoint after a single forward pass.

Checksums

bcb6e27cc31949754bfd1a48ad914c41e5204b6259720ca1c469f7cd4ed91313  breeze-tts-2-BF16.gguf
b1bff9eb99d52a2438404d4bb4f3e20a61a6bcff1dfeea41ae5eec51860e1531  breeze-tts-2-F16.gguf
1c0b4a736091f78dd43fc61846e481194482f20418d275e815b0adfc9514d39b  breeze-tts-2-F32.gguf
be226657da01b6089445fbc338d7aa46b1b70af56f159b60e4e4a5a589243440  breeze-tts-2-mmproj-F32.gguf
7e4c2f1a30b6d25746eda88ecfdbb0272f58195c98f14b0dcf524cd72d8c61f4  breeze-tts-2-Q4_0.gguf
ace2a8dbd1a2047ebfeb0f3f8a99fc426b13531cf5872e8c90e10a017e0df54a  breeze-tts-2-Q8_0.gguf

Limitations

• Q4_0 output quality degrades: 4-bit quantisation noise on the bulk attn / mlp weights propagates through the TTS frame counter, producing ~120 frames where BF16 produces ~35 (the model keeps generating audio tokens because the noisy hidden states don't fall below the stop threshold). Q4_K and per-tensor mixed-precision paths are tracked in docs/TODO.md TODO-004. Until they ship, use Q8_0 for any real deployment.
• CPU only: the validated Rust build runs CPU-only. Vulkan paths exist for other models but are not exercised against Breeze.
• No streaming API: the Rust CLI is one-shot per WAV. No WebSocket or microphone capture.
• Voice direction (instruction + reference together) is not routed: with both --instruction and --ref-audio set, the reference is consumed but CFG still follows the instruction; the reference-aware CFG dual path is not implemented. Use either instruction OR reference, not both.
• Vocal events ((laugh), [笑], etc.) are not parsed; the text encoder treats them as plain BPE tokens.
• Quantization may change output relative to the BF16 reference. The --parity-trace feature in the Rust CLI emits per-step JSONL traces that can be diffed against the upstream PyTorch reference for a given prompt; the BF16 main model output is bit-exact for the validated fixtures.

License and Attribution

The source model is provided by BreezeBlue under the BreezeBlue Research and Non-Commercial License. The model weights, checkpoints, adapters, derivative models, and self-hosted outputs are governed by that license. A BreezeBlue paid subscription does not grant commercial rights to the open-weight model or to self-hosted outputs — see the original license text.

The audio tokenizer is derived from Qwen3-TTS (Apache License 2.0). The Rust inference runtime in this repository is Apache License 2.0. This conversion redistributes GGUF weight files only; it does not modify the model weights themselves.