Lightweight text-to-speech (TTS) systems are typically distributed so that a device holds one shared network capable of producing any voice in a catalogue, even though many deployments ultimately use a single voice at a time. We examine Distribution-Time Voice Specialization (DTVS): resolving the voice axis at distribution time rather than inside the network at inference time, replacing one N-voice deployment with N independently deployable single-voice specialists. DTVS is a deployment-time analogue of Mixture-of-Experts [6], though it moves in the opposite direction: it specializes before deployment and removes runtime routing entirely, so the router is the user's own choice, external to the model, and needs no gate network, while each specialist is a complete standalone artifact. The device-side cost of DTVS is the cost of one specialist, a quantity independent of N, so a single distilled voice is sufficient to characterise it. Distilling one voice from Kokoro-82M [1] into a smaller VITS/HiFi-GAN [3][4] student yields a 68.5 MB artifact that, against the same teacher on the same Arm CPU of an NVIDIA DGX Spark GB10 [8], attains 8.6× lower real-time factor (RTF 0.0389 vs 0.3345), 11.4× lower per-sentence latency (82.9 ms vs 947.5 ms) and 7.5× less resident memory (356 MB vs 2661 MB), with no GPU. We are careful about what this does and does not establish: our student changes two things at once — it drops multi-voice conditioning and uses a smaller architecture — so the reduction is attributable to the combination, not to specialization alone (§3.2). We state the costs plainly: DTVS converts a per-device memory cost into an aggregate catalogue cost, Kokoro on the GB10 GPU retains 2.74× lower RTF once loaded, and we report no speech-quality evaluation. We also report the Arm-specific runtime work that made CPU-only serving viable, and two negative results. All figures are produced by scripts and re-checked by a checker that fails on drift.
Keywords — distribution-time voice specialization, on-device inference, text-to-speech, knowledge distillation, mixture-of-experts, Arm CPU, asymmetric multiprocessing, ONNX Runtime, edge AI, voice banking, reproducible benchmarking
A person who is losing the ability to speak — to amyotrophic lateral sclerosis, to head and neck cancer, to a stroke — may bank a recording of their voice while they still can. What they typically receive in return is a synthetic voice that lives inside somebody else's service: it needs a network, an account, a subscription, and a company that is still trading in ten years. The engineering question underneath the human one is simple. Can a person's voice be made small enough and fast enough to live on hardware they already own, for as long as they keep the file?
Contemporary lightweight TTS makes this look close. Kokoro-82M [1] is an 82 M-parameter open-weight model, Apache-2.0 licensed and derived from StyleTTS 2 [2], that produces quality competitive with far larger systems. It is distributed as a shared model checkpoint together with separate per-voice style packs, and the network accepts a per-voice reference/style tensor at inference. The distinction matters for our argument and makes it cleaner: the per-voice data is small, but selecting one voice still requires the whole shared model. The released checkpoint is 326 MB, and on an Arm CPU it takes close to a second to speak a sentence, making GPU inference attractive for latency-sensitive use.
So the capacity to serve any voice is paid for continuously and used intermittently. A device installs the shared network, keeps it resident, and then uses it for exactly one voice. This paper asks what happens if that axis is collapsed before the model ever reaches the device.
Contributions. (i) We state the Distribution-Time Voice Specialization hypothesis, give the cost model that makes its device-side claim measurable with a single voice, and identify the confound that prevents us claiming specialization alone as the cause (§3). (ii) We report a measured single-voice specialist against its own teacher on identical hardware (§5, §6). (iii) We describe the Arm-specific runtime work — topology-derived thread selection, verified against a profile — without which the CPU-only result does not hold (§7). (iv) We publish two negative results and the cost side of the trade (§8, §9), and a claim checker that fails when prose and measurement disagree (§10).
Stated conservatively, the contribution is not that removing voices causes an 8.6× RTF improvement. It is that a single-voice specialist deployment was built and measured, and then made practical on asymmetric Arm CPUs through hardware-aware ONNX Runtime tuning.
Let MN be a TTS model supporting voices v1…vN. Serving voice vi requires the whole of MN to be installed and resident, because the voice identity is an input to the network rather than a property of the artifact.
DTVS replaces MN with a set of specialists Si = distil(MN, vi), each a complete model that can produce exactly one voice. Selection becomes a function of the installation, not of the forward pass: the user chooses once, the device holds one artifact.
Write Cdev for what a single device must install and hold resident, and Ccat for what the publisher must store and distribute in aggregate. Then
The claim DTVS makes is about Cdev, and Cdev(DTVS) = |S| contains no term in N. It is therefore measured completely by building one specialist and characterising it. Training twenty specialists would produce twenty samples of a quantity whose dependence on N is nil; it would raise our compute bill and leave the hypothesis exactly as well or as badly supported.
What a single voice cannot establish is uniformity — whether every voice in a catalogue distils equally well. We do not claim it, and we mark it as an open question in §11.
We therefore do not claim that removing the voice axis causes the reduction. The defensible claim is narrower and is the one we make throughout: a distribution-time single-voice specialization can yield a substantially smaller and faster deployment artifact, and we measured one that does.
The ablation that would establish causality is well defined and we did not run it: train the same student architecture twice, once multi-voice and once single-voice, and compare. That isolates the voice axis with architecture held constant. Until somebody runs it, the honest reading of §6 is that specialization plus a smaller architecture, together, produced the artifact — with the practical consequence unchanged, because a deployment gets both or neither.
The pipeline is eight scripts. A teacher speaks a fixed corpus in the target voice; the resulting pairs train a smaller student from scratch; the student is exported to ONNX and served on the CPU. Only the teacher step differs between distilling a catalogue voice and cloning a person's own voice from a single reference recording.
Grapheme-to-phoneme conversion is delegated to eSpeak NG [11], which is invoked as a separate executable; VoiceYog does not link eSpeak NG code into its runtime. eSpeak NG is GPL-3.0-or-later, and we describe only that engineering arrangement — we draw no conclusion about its licensing consequences, which depend on both the mechanism and the semantics of the communication and are outside the scope of this paper. The prompt corpus is built offline from a phonetically-motivated word bank and the CMU ARCTIC prompts [7].
The primary machine is an NVIDIA DGX Spark [8] built on the GB10 Grace Blackwell superchip: a 20-core Arm CPU comprising ten Cortex-X925 cores at 3.90 GHz and ten Cortex-A725 cores at 2.81 GHz, with a Blackwell GPU sharing 128 GB of unified LPDDR5X memory. A second target, an Apple M1 Max with 8 performance and 2 efficiency cores, is used to test that the runtime logic generalises across asymmetric Arm parts.
The benchmark of record uses 20 held-out sentences, five warm-up sentences, and a separate process per engine, with the GPU verified idle at start. Peak RSS is the process maximum. Real-time factor (RTF) is synthesis time divided by audio duration; lower is better.
| Engine | RTF | Latency | Peak RSS | On disk |
|---|---|---|---|---|
| Ours — Arm CPU only | 0.03888 | 82.9 ms | 356 MB | 68.5 MB |
| Ours — Arm CPU → GPU | 0.01957 | 42.2 ms | 1898 MB | 68.5 MB |
| Kokoro-82M — GPU | 0.01418 | 40.1 ms | 3464 MB | 326 MB |
| Kokoro-82M — Arm CPU | 0.33447 | 947.5 ms | 2661 MB | 326 MB |
Like for like. Rows one and four are the same task on the same silicon, so the comparison carries no hidden hardware term. The two speed ratios differ and we report both rather than picking one: 8.6× lower RTF (0.03888 vs 0.33447) and 11.4× lower mean per-sentence latency (82.9 ms vs 947.5 ms). They diverge because the two engines synthesised different total audio durations over the same 20 sentences, and RTF normalises by audio produced while latency does not. RTF is the more conservative figure and is the one we quote when a single number is needed; wherever this paper says "8.6×" it means RTF. Memory is 7.5× less and the file is 4.8× smaller.
Against the accelerator. Compared with the teacher on the GB10 GPU, the specialist holds 9.7× less memory and reaches first audio 5.8× sooner from cold (0.94 s vs 5.42 s, medians of three runs). That gap is consistent with avoiding CUDA context initialisation and GPU-side model setup and allocation, but we did not profile initialisation, so we attribute it to those operations only as a plausible explanation rather than a measured decomposition. We specifically do not describe it as copying weights into discrete video memory: the GB10 exposes 128 GB of coherent unified memory [8], so the traditional host-to-device transfer is not the mechanism here.
Once both are warm, Kokoro on the GPU retains 2.74× lower RTF (0.01418 vs 0.03888), equivalently about 2.1× lower mean sentence latency (40.1 ms vs 82.9 ms). Both are measured and both are published; the case here rests on memory, cold start and the absence of an accelerator, not on beating a GPU at steady-state throughput.
A 68.5 MB graph does not by itself produce 82.9 ms. Before tuning anything we profiled the shipped workload with Arm Performix (code_hotspots, 41,593 samples): 94.0% of Arm CPU time is inside ONNX Runtime's kernels, 3.1% in OpenBLAS, 2.0% in libc, and Python interpreter overhead is under 1%. The tuning knob is therefore attached to nearly all of the workload rather than to a wrapper around it.
Both targets are asymmetric. ONNX Runtime parallelises an operator across intra-op threads and joins them, and that join is a barrier: the operator finishes when its slowest thread finishes. A thread placed on a lower-capacity core can therefore become the straggler for every other thread, on every operator, for the whole graph — so on these parts, adding cores can subtract performance. We deliberately do not quantify the per-core gap from clock frequency: the Cortex-X925 and Cortex-A725 are different microarchitectures, and a ratio of advertised clocks is not a ratio of workload throughput. The evidence for the effect is the sweep in Table 2, which measures it end to end rather than inferring it.
The runtime therefore reads the core layout at startup — from MIDR_EL1 and cpu_capacity on Linux, hw.perflevel0 on macOS — and derives a thread count instead of accepting one. On our DGX Spark test system Linux enumerated the performance cores as 5–9,15–19, interleaved across clusters rather than contiguous, so a range chosen by inspection would place half the threads on efficiency cores. Logical CPU numbering is a property of what the OS reported on this machine, not an architectural guarantee, which is precisely why the runtime queries it instead of hard-coding it.
| Machine | Cores | Chosen | vs all cores | vs ORT default |
|---|---|---|---|---|
| DGX Spark GB10 | 20 | 9 | 1.39× | 1.65× |
| Apple M1 Max | 10 | 8 | 2.21× | 1.20× |
On both machines the topology's prediction was the measured optimum. The same binary and the same graph retune themselves per part with no recompilation and no configuration.
INT8 quantization produced an artifact that does not run. Dynamic quantization made the file 3.3× smaller and rewrote 183 convolutions into ConvInteger, for which ONNX Runtime's CPU provider had no aarch64 kernel in this configuration. The same failure occurred at the same node on both targets. A smaller file is not an optimization if the artifact cannot be loaded.
Splitting the graph across CPU and GPU lost. A cooperative path that runs the encoder prefix on the Arm CPU and the decoder on the GPU reaches RTF 0.0196 — 0.72× Kokoro on the GPU, i.e. slower — while using 1.8× less memory. The stage instrumentation totals 24.8 ms: 14.2 ms of Arm CPU prefix, a 0.12 ms handoff, and 10.4 ms of GPU decoder. Percentages in this paragraph are shares of that 24.8 ms instrumented total, not of the 42.2 ms end-to-end latency in Table 1, which additionally contains untimed setup — so the prefix is 57.4% and the handoff 0.49% on that basis. The prefix runs strictly before the GPU stage, so each processor idles while the other works. The handoff is not the problem, and the result is reported rather than dropped.
DTVS is not free, and the cost is exactly the one the model in §3 predicts. Table 3 works it through for a catalogue of twenty voices, using our measured single-voice artifact and the released teacher.
| Monolith | DTVS | |
|---|---|---|
| Shipped to one device | 326 MB | 68.5 MB |
| Resident during inference | 2661 MB | 356 MB |
| Accelerator for practical latency | GPU | none |
| Aggregate catalogue | 326 MB | 1,370 MB |
| Voices available per device | N | 1 |
DTVS trades an aggregate storage cost, paid once by a publisher where storage is cheap, against a per-device memory cost, paid continuously by every user on hardware where memory is the binding constraint. It is the wrong architecture for a product whose value is switching voices freely, and the right one for a person who needs one voice to keep working on a machine they own. The single-voice measurement is what makes that trade quantitative rather than rhetorical.
A second consequence of DTVS is structural rather than numerical, and we raise it as motivation and future work rather than as a result. Regulation is moving toward persistent provenance for synthetic audio: California's AI Transparency Act [16], as amended by AB 853 [17] and operative 2 August 2026, requires covered providers of generative-AI systems to attach a latent, tamper-resistant disclosure to AI-generated audio, to offer a manifest disclosure, and to publish a free detection tool.
Existing provenance schemes answer "was this generated, and by which system?" They answer "which voice?" less well. In a shared N-voice network the voice is a runtime argument, so the artifact hash identifies the system rather than the persona, and recovering the voice means trusting a field the runtime reported about itself. Under DTVS that distinction collapses: one voice is one artifact, so the model checksum already present in the record is a stable per-voice identifier, because there is exactly one voice the artifact can produce. Voice identity becomes a property of the artifact rather than a parameter of the call. That alignment costs nothing extra — it falls out of the deployment structure — which is why we think it is worth stating even though we did not build for it.
What we implement is modest and we describe it precisely. Each clip is accompanied by a signed record naming the model, its checksum, the runtime and the platform, verifiable offline by a recipient holding neither the model nor the private key, and tamper-evident: our tests confirm that altering a single bit of the audio makes verification fail.
The obvious next step follows from the gap rather than from the result: pairing DTVS with an acoustic watermark would supply the half we lack, with the artifact providing the identity and the watermark providing survivability through redistribution. We have not built or evaluated that, and we claim nothing about it here.
This section discusses regulation as engineering motivation only. It is a research reading of a moving legal landscape, may contain inaccuracies, and is not legal advice; anyone with an actual compliance obligation should consult counsel and the statute itself.
Every number above is written to JSON by the script that measured it, and a checker [13] compares the prose against those files and exits non-zero on disagreement — 43 claims at the time of writing. It requires Python 3 and nothing else: no model, no virtual environment, no network. The thread sweep re-runs in about a minute on the reader's own machine and will report where their hardware disagrees with ours.
Every generated clip additionally carries a signed provenance record naming the model, its checksum, the runtime and the platform, verifiable offline by a recipient holding neither the model nor the private key; the design follows the shape of C2PA Content Credentials [12] without claiming conformance to that specification.
One voice, not a catalogue. We measured a single specialist. §3.1 argues why that measures the quantity DTVS depends on; it does not show that every voice distils equally well, and voices with unusual prosody or limited data may not.
The reduction is not isolated to specialization. As set out in §3.2, our student drops multi-voice conditioning and changes architecture simultaneously. The ablation that separates them — the same architecture trained multi-voice and single-voice — remains unrun, and until it is, no causal claim about the voice axis alone is supported by this work.
Two machines. The topology result holds on a GB10 and an M1 Max. Other asymmetric Arm parts are expected to behave similarly and were not tested.
Steady-state throughput. A GPU running the teacher remains faster per sentence once loaded. DTVS addresses footprint and cold start.
No speech-quality evaluation. This is the most important gap and it limits how the speed result should be read: without a quality measurement, a reader cannot tell whether the 68.5 MB student retained enough fidelity for the comparison to be meaningful. Closing it needs, at minimum, ASR-based intelligibility (Whisper WER/CER) and a speaker-similarity score against the teacher, and preferably a human AB preference test. We report none of these and claim nothing about perceptual quality.
No attestation, and no compliance claim. The provenance discussion in §9.1 is motivation, not a contribution. Our record is detached rather than embedded in the audio, so it is not a watermark and cannot survive a recipient who discards it; we neither implement nor evaluate acoustic watermarking, speaker attestation, or any mechanism that would let a voice be identified from the waveform alone.
Unmeasured next step. Our pinned ONNX Runtime 1.20.1 contains no KleidiAI [10] symbols, while a newer build on the same machine contains eleven kai_run_matmul_* symbols. That build reports version 1.28.0 with git-commit-id 45de2a8b06 (cp312-manylinux_2_28_aarch64); it is a pre-release wheel and was not, at the time of writing, in the stable PyPI channel [15], so we identify it by commit rather than by version number alone. Those kernels target the operators that dominate our profile, which makes upgrading a measurable next step and not a claimed result.
A shared multi-voice network asks every device to carry the ability to serve any voice so that it can serve one. Distribution-Time Voice Specialization moves that choice out of the forward pass and into distribution, and the cost model says the device-side saving does not depend on how many voices the catalogue holds — which is why one specialist is enough to characterise it. The specialist we built is 68.5 MB, attains 8.6× lower RTF and 11.4× lower per-sentence latency in 7.5× less memory than its own teacher on the same Arm CPU, and needs no accelerator. We cannot say how much of that came from dropping the voice axis and how much from the smaller architecture, and we report no quality evaluation; what we can say is that the artifact exists, was measured, and was made practical on asymmetric Arm CPUs by reading the silicon rather than guessing at it. For the person who banked one voice and wants it to still work in ten years, on hardware they own, offline, that is the difference between a service and a file.