quantized duplicated precisions.json, which is authoritative for the
quantized/non-quantized partition. default expressed a per-repository
recommendation nothing consumed: the library resolves its own per-platform
default. Both are gone from the schema.
Adds library_name to the model card, and pipeline_tag where it was missing.
Brings the config and card licenses into agreement, and repairs the variant
declarations that did not match the files actually published here.
The int8 build quantized only the tied logit head, leaving the rest of the
decoder in fp32. Quantizing every decode-side linear takes the file from
154 MB to 130 MB with a transcript identical to the fp32 decoder.
Encode is untouched and its output is bit-identical to the fp16 variant:
the cross-attention key and value projections are excluded, since the
encoder hoists them and they run in the Vulkan partition.
Match the vulkan config to the canonical exporter spec
The published config dropped int8's default flag and declared the encode
input as -1. The canonical schema marks one default per quant-group, and the
other whisper backends declare the encode length as its 480000 maximum.
Exported with the conv1d-as-conv2d lowering, the slice-based STFT magnitude,
hoisted cross-attention K/V and a head-major KV layout. Requires the ExecuTorch
1.4.1 Vulkan runtime from the v0.10.0-libs-1.4.1 artifacts (gelu clamp, reduce
barrier and Adreno local-work-group fixes).
perf(coreml): run decode on the CPU, not the Neural Engine
decode is a single-token program whose cost on device is dispatch rather than arithmetic, so it was paying Neural Engine overhead on every call under ExecuTorch's ALL default. Pinning only decode to CPU_ONLY (encode stays on ALL) measures 6.7 ms/step against 20.0 on an iPhone 16.
On 14.5s of real speech the whole transcribe is 1.5-3.1x faster depending on size, with a byte-identical transcript: this removes overhead, it does not change numerics.
Exported from export-scripts MR !18, built on main pins (ExecuTorch 1.4.1). Replaces an ExecuTorch 1.2.0 build of the same change; transcripts verified byte-identical on device at all six sizes.
perf(coreml): run decode on the CPU, not the Neural Engine
decode is a single-token program whose cost on device is dispatch rather than arithmetic, so it was paying Neural Engine overhead on every call under ExecuTorch's ALL default. Pinning only decode to CPU_ONLY (encode stays on ALL) measures 6.7 ms/step against 20.0 on an iPhone 16.
On 14.5s of real speech the whole transcribe is 1.5-3.1x faster depending on size, with a byte-identical transcript: this removes overhead, it does not change numerics.
Exported from export-scripts MR !18.
