Voice in the Browser

How the voices are made

Each voice is cloned from a short set of real recordings of one person. The model on this page, Kokoro, is the smallest of several we train per voice. It's the least accurate, but it's small enough to run entirely in your browser, on your own device: nothing is sent to a server. The larger models sound closer to the real person, but they need a graphics card (GPU) and much bigger files.

The recipe: a big teacher trains small students

1. Real recordings. A few minutes to half an hour of the person speaking, with checked transcripts, cut into short segments. A few segments are held back from all training, so the models can be tested on sentences they never heard.
2. The teacher. Dots.TTS, a large open model, is fine-tuned on those recordings (minutes on a rented GPU) until it sounds like the person. We listen to several checkpoints and keep the best, sometimes a mix of them.
3. Making more speech. The teacher reads thousands of lines in the cloned voice: everyday sentences, stage-play dialogue and story narration, taken only from sources that are free for any use (public-domain books and plays, Tatoeba's CC0 sentences, and lines written for the project).
4. Quality filter. A speech recognizer (Whisper) transcribes every line; takes with misread words, odd speed or clipping are dropped, and everything is levelled to one volume.
5. The students. Qwen3-TTS (0.6B and 1.7B) and Kokoro are trained on the teacher's hours of speech plus the real recordings. They learn the voice from hours of examples instead of minutes.

Listen: the real voice next to each model

Each row is one sentence. The first column is the original recording we started from (where one exists); the others are the same sentence spoken by each model. Press play on any of them.

By the numbers

What we measured while making the “dad” voice. Speed is given as seconds of work per second of speech (the “RTF” shown under the player): below 1 is faster than real time, 0.5 means a minute of speech takes 30 seconds. The mini PC is a Beelink SER5 (AMD Ryzen, integrated graphics); the rented GPU is an NVIDIA L40S (48 GB) at about $0.73 an hour.

How fast each model speaks

Example: narrating a whole book

The Adventures of Tom Sawyer is about 70,000 words, roughly 8 hours of narration.

Training, step by step (one rental session)

Data and file sizes

Side quest: a smaller teacher (LoRA)

The fine-tuned Dots teacher is a 9.6 GB file, and every training session on a rented GPU needs it. Most of those gigabytes are the public base model, which the GPU can download for free; only the change made by fine-tuning is ours. So we tried shipping just that change in compressed form, as a LoRA (low-rank adapter).

We didn't train a LoRA; we extracted one from the finished fine-tune. For every weight we subtracted the base model from the fine-tuned one. The 475 large matrices were compressed with an SVD, keeping only their strongest r directions (the “rank”); the 391 small weights were kept in full. Base model + adapter rebuilds an approximation of the teacher.

Same sentence, same random seed, so the only difference is the model:

What we did instead: fine-tuning Dots takes only about 5 minutes on the rented GPU, and with the same settings and random seed it lands on the same voice. So each session downloads the public base model and simply retrains the teacher; all we upload is a ~150 MB bundle of scripts, text and the real recordings. The LoRA route is still in the pipeline as an option, and a LoRA trained from the start (rather than extracted afterwards) might do better.

Running the bigger models: the voice CLI

The larger models run on a home PC through a small command-line tool, voice. Every voice is a folder (voices/<name>/) holding its checkpoints for each engine and a few reference clips; adding a voice means adding a folder. The tool lists what's there, speaks text with any model and narrates whole books.

$ voice list
dad
├─ qwen-tts  (24 kHz, GPU via Vulkan; size = one runnable GGUF; checkpoints are BF16, 2.4/4.3 GB)
│                                    rating  params runs as     size     direct  hybrid  note
│  ├─  0.6B-student-epoch4           ★★★☆☆   0.6B   Q8_0 GGUF   969 MB   ready   ready   good; volume swings
│  └─* 1.7B-student-lev-epoch4       ★★★★☆   1.7B   Q8_0 GGUF   2.0 GB   ready   ready   chosen 1.7B: consistent
├─ dots  (48 kHz, CPU only here, slow; the teacher)
│                                    rating  params runs as     size     direct  hybrid  note
│  ├─  retrain500-step300            ★★★★★   2.2B   FP32        9.6 GB   ready   ready   #2: best breathing/pauses
│  └─  retrain500-step500            ★★★★★   2.2B   FP32        9.6 GB   ready   ready   #1 teacher: best overall
├─ kokoro  (24 kHz, CPU, fast; the web page uses ONNX exports: FP16 164 MB / FP32 325 MB)
│                                    rating  params runs as     size     direct  hybrid  note
│  └─  b4-epoch0                     ★★★★☆   82M    FP32        328 MB   ready   -       sounds most like him
└─ refs: 10s, 2min

$ voice rate "1.7B lev" 5 "my pick"                         # ratings live next to each model (notes.json)
$ voice say -m "1.7B lev" "Good evening, and welcome back to the show."
out/voice/20261024-171200-dad-1.7B-student-lev-epoch4.wav  (3.4 s audio, RTF 1.0)

$ voice say -m "dots retrain500 step500" --seed 2 "..."     # best quality, slow on a CPU
$ voice say --ref 10s "..."                                 # hybrid: clone from a reference clip
$ voice book my-novel.txt --mp3 --tempo 0.9                 # chapters -> MP3, resumable
$ voice serve                                               # this page, on the home network

Reading the list. params is the model size; runs as is the format it runs in on the PC (Qwen is converted to 8-bit GGUF files for its graphics chip; Dots and Kokoro run at full FP32 precision on the CPU); size is what one runnable copy takes on disk; rating and note are our own listening verdicts, stored next to each model so the best ones are easy to find later.

Direct vs hybrid. A fine-tuned Qwen model can speak the voice two ways. Direct: the voice is built into the model as a named speaker, so it needs nothing else; this is the normal, fastest way. Hybrid: the same fine-tuned model is combined with Qwen's speaker encoder, a part that listens to a short reference recording and clones whatever voice it hears in it. Then you pass a clip (e.g. 10 seconds of the person) with each request, and the model imitates that clip on top of what it learned in training: useful for steering style (a calmer or livelier take), at the cost of processing the clip first. Dots can clone from a reference too; Kokoro can't (its voice is a fixed "voicepack" made from the real recordings).

• Qwen models run on the PC's graphics chip through Vulkan (qwentts.cpp): the 0.6B model is faster than real time on a mini PC with integrated graphics.
• Kokoro runs on the CPU (and in the browser, as on this page). Dots runs on the CPU here only for testing: it needs a big GPU to be practical.
• Every command prints the exact underlying command it runs, so results are easy to reproduce.

Notes: making it work on iPhone

The same page that ran fine on Android, Windows and Linux needed a string of fixes for iPhone and iPad, where every browser (Safari, Chrome, Firefox) runs on Apple's WebKit engine. In the order we hit them:

We weren't the only ones: other projects using Kokoro in the browser report the same Safari stream issue, thread hangs and iPhone tab crashes; the plain WebAssembly build was the workaround that made Kokoro run on iPhone.

References

The open projects, models and data this was built on.