How I Gave My Kiosk a Voice Clone Trained on 213 Lines of Reach Cortana

What this is: the implementation guide for the voice arc of my family voice kiosk. I replaced the kiosk’s generic TTS voice with a voice clone of Reach Cortana, trained on 213 clean voice lines extracted from my own legally-owned copy of Halo: The Master Chief Collection. The shipped path has four parts: extracting and vetting the corpus from the game’s audio banks, an F5-TTS false start that taught the data lessons, an OmniVoice zero-shot spike that proved the direction, and a LoRA finetune merged into a standalone model that now answers every question the kiosk gets. This guide is the companion to the kiosk guide: I Built My Family a Voice Assistant That Never Touches the Cloud.

What is deliberately out of scope: RVC timbre training. It was tried first, it sounded right on single samples and wrong on whole answers, and the OmniVoice switch replaced it as the voice. A trained RVC model still sits in the kiosk’s fallback path as a timbre layer on Kokoro, but training one is no longer required to recreate this system: skip it and the fallback is plain Kokoro. This guide covers the path that shipped.

Environment used here: Windows 11, two RTX 5060 Ti 16 GB cards, Python 3.11 and 3.12 venvs, CUDA 12.8. Training ran on one 5060 Ti.

Ground truth: every number below was on 2026-09-07 on this setup. Nothing is from a spec sheet. The kiosk server source and auth module are published with the kiosk guide’s manifest; the lab log, the training data, and the model stay on the box (section 8).

Boundary first: the source audio is my own MCC install, the training corpus never left the machine, and the finished voice is a local model that serves my LAN only. The OmniVoice code is Apache-2.0; the model weights are CC-BY-NC (trained on Emilia), which private non-commercial family use complies with. If you cannot say the same about your source material and your use, this pipeline is not for you.

Follow it top to bottom, or hand it to an AI agent as the spec (section 9).

0. Goal and acceptance

The goal: the family kiosk stops sounding like a generic TTS voice and starts sounding like her. Not an impression layered on flat prosody, a delivery that holds up word for word across whole answers.

Acceptance criteria, all met on 2026-09-07:

Time: one day (2026-09-07). Cost: zero new hardware; compute is a spare GPU afternoon.

1. The corpus: from a Steam folder to 213 clean lines

This is the part that looks magical and is not. The whole build stands or falls on source audio, and the process of getting clean, verified, female, on-voice audio is a pipeline with its own traps. Here is the actual one.

1.1 Find the audio and learn what it is

The game install was at the usual Steam location, and Reach’s dialogue is not in loose files: it is in an FMOD bank.

...\Steam\steamapps\common\Halo The Master Chief Collection\
    haloreach\fmod\pc\english.fsb        (380 MB, all English voice)
    haloreach\fmod\pc\english.fsb.info   (8.8 MB binary name table)

The .fsb magic identifies the format: FSB5, FMOD’s bank format, with a header of magic, version, sample count, header size, and data size. One number in that header is the whole key to the project: 31,433 samples. The .info file next to it has exactly 31,433 names. Index i in the name table is sample i in the bank, so every one of the 31,433 streams has a real path name like data\sound\dialog\combat\cortana\default\... instead of being an anonymous stream in a 380 MB blob. Parse the name table once and you can search the dialogue like a filesystem.

1.2 Decode the bank

The decoder is vgmstream (r2117 build), the standard tool for game audio:

vgmstream-cli -o out.wav -s N file.fsb

-s N selects sample N (1-based); without it you get the whole bank as one file. With a small driver script that walks the name table and maps each sample index, the result for the Cortana subset was 588 of 588 samples decoded cleanly: 15.0 minutes of 48 kHz mono, archived under {index}_{slug}.wav names in a raw-decode folder. The .aif.wav double-extension bit us later (section 3); ignore it for now.

1.3 Find who is speaking: the path map and its three lies

Every decoded stream carries its dialogue path. Paths are speaker maps, and this is where the first contamination happened. Three traps, each one real:

• dialog\firefight\cla_cov_* is Covenant dialogue, not Cortana.
• dialog\firefight\cla_unsc_* is the UNSC firefight ANNOUNCER: a male voice at 95 to 148 Hz. It sits in the firefight announcer slot where Cortana also speaks in the fiction, it sounds announcer-adjacent, and samples like “Mission Aborted” sit at 95 Hz. This batch was in the first training corpus as roughly 20 percent of the data before I caught it by ear and the pitch gate confirmed it.
• dialog\combat\cortana\default\11_involuntary is her grunts. Grunts are not speech; exclude them.

The real Cortana corpus lives in paths that literally say so: dialog\combat\cortana\default\* (02_combat kill quips, 03_postcombat praise, including “Don’t make a girl a promise…” and “If I had fingers, they’d be crossed”), plus firefight_preview_dialog\cortana and dialog\levels\m80 (campaign). The final whitelist is those three path families and nothing else.

Then every one of the 588 decoded samples was transcribed with whisper on CPU (about 2 minutes for the set) into a JSON file of file, duration, and text, so the corpus could be searched by what she actually says. That is how the calm reference lines were found later: not by listening to hours of audio, but by searching transcripts for conversational content.

1.4 The gates that keep a corpus honest

After the male-announcer catch, every file in the corpus had to pass four gates before it counted:

1. Provenance, by hash. SHA-256 of the file must match its own decode from the game bank. No other source exists. Zero violations.
2. Path whitelist. The dialogue path must be inside the three Cortana families. Zero violations.
3. Pitch. Median F0 at or above 170 Hz (librosa pyin), the female gate. 192 of the 213 files sit at or above 180 Hz.
4. Formant and voicing, for the low register. 21 files sit at 149 to 182 Hz, mostly deadpan delivery, and pitch alone cannot clear them. The tiebreaker is a formant profile (F2/F3 matching the female references, not the male 96 to 148 Hz announcer references) plus a floor-120 voicing-retention test: female speech keeps 50 to 81 percent voiced frames at that floor, male speech collapses to 16 to 24 percent.

One measurement is a trap by itself: median F2 is not a sex test, because it tracks vowel content. It flagged eight false positives on its own, including “low on ammo” with its /oʊ/ vowel. The verdict after all four gates: 0 male voices in 213 files, ~5.3 minutes of audio.

1.5 Why the corpus is small on purpose

5.3 minutes is small by commercial TTS standards and large enough to matter, because every second passed four gates. The earlier attempts were bigger and dirtier (the announcer batch alone added 20 percent of audio that had to be thrown away). A bigger pile of dirty audio trains worse than a small pile of verified audio, and a finetune this small is cheap enough to redo, so the gate cost is one-time and the corpus stays honest.

2. The engine search: F5-TTS first, OmniVoice second

The first zero-shot engine was not OmniVoice. It was F5-TTS, and it failed in instructive ways before OmniVoice made the cut.

2.1 F5-TTS: the false start that taught the data lessons

F5-TTS ran in its own venv on the same box, and it cost a morning of Windows-specific pain that is worth one paragraph: torchcodec needs a matching FFmpeg shared build (torchcodec==0.11.1+cpu plus FFmpeg 8.1.1 shared DLLs, because the current gyan.dev builds ship avcodec-63, which torchcodec does not support), and scripts must call os.add_dll_directory before importing f5_tts. The output of infer() is a numpy array, not a file. And zero-shot needs ref_text; auto-ASR returns None, so reference lines are transcribed with whisper first.

The samples told the real story, and my ear was the QC:

The cutscene lesson is the durable one: mixed cutscene audio, with a music bed under the voice, breathy emotional delivery, and compression, makes a zero-shot clone drift to a generic voice. Clean dry game audio clones well; mixed cutscene audio does not. And one reference file from this era carried a non-game preamble, “good morning here is my voice reading a sample for you to compare”, caught by whisper transcription later. File names lie; transcriptions do not. That preamble explains a lot of the weak F5 cloning in hindsight.

2.2 OmniVoice zero-shot: the spike that shipped

OmniVoice (k2-fsa/OmniVoice) is a 0.6B model on a Qwen3-0.6B backbone: code Apache-2.0, weights CC-BY-NC (trained on Emilia), which private non-commercial family use complies with. The zero-shot API is two calls:

model.generate(text, ref_audio, ref_text)          # -> numpy int16 @ 24 kHz
create_voice_clone_prompt(ref_audio, ref_text)     # -> encode once, save/load .pt

The second call is the kiosk-shaped one: the reference is encoded once into a prompt file at build time and loaded at boot, so no answer ever pays reference work. The reference was three calm lines built from the corpus by the selection script (9.7 s total), transcribed and whisper verified.

Measured on the spike, one 5060 Ti, fp16:

The personality line from that spike, pre-finetune:

My verdict on the first sample was “night and day improvement” over everything before it, and OmniVoice became the primary voice immediately. Zero-shot was good enough to ship. The finetune in section 3 exists because zero-shot still leans on the reference at generation time, and because a finetuned model is the more robust daily driver: no reference audio to manage, one merged file, identical output every time.

3. The finetune: 213 lines into a standalone model

The corpus was finetuned into the model with a LoRA adapter. The data format is one JSONL per line of audio, the schema straight from the finetune examples, and the first two lines of the training file on the box read:

{"id": "0001_29755_morefoe_2", "audio_path": "...ft_audio\\0001_29755_morefoe_2.wav", "text": "Warning, more covenant on approach.", "language_id": "en"}
{"id": "0002_29756_morefoe_3", "audio_path": "...ft_audio\\0002_29756_morefoe_3.wav", "text": "Blast! There's more of them!", "language_id": "en"}

The pipeline, in order:

1. Build the JSONLs from the transcribed corpus (197 train / 16 dev lines), with the audio paths re-pointed into the finetune audio folder. A small helper fixes the legacy double-extension ids: webdataset splits tar member names at the FIRST dot, so .aif.wav ids silently break the reader. Single-extension ids only.
2. Tokenize to shards: audio runs through the higgs-audio-v2 tokenizer into WebDataset shards (33 train, 16 dev). On Windows this step needs --loader_workers 0: 24 workers exhausts the paging file (WinError 1455). And webdataset operations need file:// prefixed paths, because a C:/ path is parsed as a URL scheme; plain paths stay plain for open() and existence checks.
3. Train: the shipped configuration, all measured:

4. Merge: the adapter merges into a standalone model, 3.1 GB total, no PEFT dependency at runtime. Merge after train, test the merged file, ship the merged file: the sidecar is one model file and one inference path, and the voice cannot break because an adapter failed to load.

Note (install the right OmniVoice): PyPI omnivoice 0.2.1 has no LoRA support. The LoRA config is silently ignored and the run escalates to a full finetune that OOMs. Install the GitHub master build with --force-reinstall --no-deps or the LoRA path does not exist.

Note (the finetune invalidates the zero-shot prompt): the finetune retrains the audio embeddings, so the zero-shot cortana_prompt.pt from section 2 is incompatible with the finetuned model. The kiosk sidecar rebuilds the clone prompt at boot from the finetuned model instead. If you skip that step, the sidecar fails health checks with a prompt mismatch that looks like a model-load bug.

250 steps for 5.3 minutes of audio was proportionate: enough passes for the model to internalize the delivery, few enough that the run finished before the Windows crash gremlins in section 6 could eat a checkpoint. Longer is a knob; measured cost is about 2 minutes per 250-step run.

4. Quality control: transcription and pitch, not ears alone

A voice clone is easy to fool yourself about. The QC battery the finetuned model had to pass:

Two QC layers, one objective (transcription, pitch) and one subjective (the ear test). The objective layer catches model collapse and mispronunciation; the ear test catches the uncanny valley that numbers cannot. Sample deliverables from both the zero-shot and finetuned models were saved as audio files for exactly this: side-by-side listening decided the swaps.

The finished thing, 37 seconds: the tablet kiosk answering, recorded off the screen. This is the ear test that numbers cannot express.

5. Into the kiosk: one sidecar, one port, automatic fallback

The clone does not live inside the kiosk server. It runs as a separate process, kiosk_omni_sidecar.py, that the server spawns and calls over HTTP, so a crash in the voice model can never take the kiosk down, and the clone owns a whole GPU so a long answer cannot starve speech to text or the brain. The integration contract, quoted from the published server source:

The server passes its own PID to the sidecar at spawn ([python, sidecar.py, str(os.getpid())]): the sidecar monitors its parent and exits when the kiosk goes down, so a tear-down leaves no orphan holding GPU0 VRAM.

Measured after the finetune went live: GPU0 at 5.4 of 16 GB with the merged model (up from 2.7 GB for zero-shot), sentence pieces 1.5 to 7.9 s of audio, 1.2 to 1.5 s generation per piece, e2e PASS.

The piece-splitting rule that fixed cut-out speech. OmniVoice generation costs about one second fixed per call regardless of piece size. Comma-level fragments, about one second of audio each, could not keep the gapless player fed, and the symptom was intermittent speech cutting out. The server splits on sentence boundaries instead, into pieces of 3 to 8 s of audio, keeping the client queue two or more pieces ahead; a 200 character ceiling falls back to comma granularity so a never-ending sentence still streams. Speech rate is 1.35, matching the Kokoro cadence the family already liked, so the clone and the fallback voice do not sound like different speeds to the same question.

6. Avoid My Mistakes

1. PyPI omnivoice has no LoRA support. The config is silently ignored and the run escalates into a full finetune that OOMs. Install GitHub master with --force-reinstall --no-deps.
2. Tokenization workers on Windows exhaust paging. --loader_workers 0. Twenty-four workers earned a WinError 1455.
3. webdataset on Windows rejects drive letters. C:/ paths are parsed as a URL scheme by gopen. Use file:// prefixes for webdataset operations and plain paths for open() and existence checks.
4. Tar member names split at the first dot. Corpus ids must be single-extension; legacy .aif.wav names broke the reader silently.
5. The trainer OOMs on length-grouped padding at batch 64. Use max_batch_size 4 to 8, num_workers 0, prefetch_factor None. expandable_segments is a Windows no-op, do not rely on it.
6. Windows crashes at random epoch resets and saves (0xC0000005), and a mid-save crash leaves an empty checkpoint directory that cannot resume. Run short (250 steps) with a single end-save. Relaunch is cheap.
7. Speaker maps lie twice. cla_cov_* is Covenant, and cla_unsc_*, the firefight announcer slot, is a MALE voice at 95 to 148 Hz that spent hours in the corpus as “Cortana”. The folder that says cortana is her; the folder that implies her is not. Gate on pitch after you map by path.
8. Path names lie, pitch does not, and transcripts catch what both miss. A 16-file male batch passed the name check and failed the pitch gate. A reference file carried a non-game preamble that whisper transcription caught after approval. Gate everything on measured content, verify after approval, and ban YouTube audio outright: mixed cutscene audio drifts a zero-shot clone to a generic voice.
9. An unmerged adapter is a runtime dependency you do not want. The kiosk loads one merged 3.1 GB model with no PEFT layer. Merge, test the merged file, ship the merged file.
10. Judge with your ears, then with numbers. The timbre-only approach passed single-sample listening and failed whole-answer prosody. The OmniVoice spike passed both. Transcription and pitch QC catch what listening misses, and listening catches what numbers cannot express.
11. A finetune that retrains embeddings invalidates the zero-shot prompt. Rebuild the clone prompt from the finetuned model or the sidecar dies on a mismatch that looks like a load bug.

7. Day-2 operations

• Verify alive: netstat -ano | findstr ":18083" LISTEN, then check the kiosk log for the clone ready line, then run the kiosk e2e from the kiosk guide’s manifest.
• Sidecar log: omni_sidecar.log next to the sidecar script.
• The kiosk server boots the sidecar at startup and the sidecar exits with its parent, so bring-up and tear-down are just the kiosk’s own restore and stop files (kiosk guide, section 11).
• Want more of her: the knob is steps. Measured cost: about 2 minutes per 250-step run on one 5060 Ti, plus a merge and a prompt rebuild.
• Samples: the zero-shot and finetuned deliverables are kept as audio files for side-by-side QC on every future voice change.

8. What stays on the box

The lab log, the finetune JSONLs, and the trained model stay on the box: this is a private project and nothing about the corpus or the clone is published anywhere. The measured configuration in sections 3 and 5 is complete enough to recreate the run, and the kiosk’s own code (server, auth, tests, clients) is published with the kiosk guide’s manifest so the integration contract is executable. What you still need from the box if you want a command-level rebuild: the sidecar scripts (kiosk_omni_sidecar.py, kiosk_rvc_sidecar.py), the extraction and selection helpers the lab log names (parse_info, decode, montage, survey, select_build), and the tokenize and train shell scripts. The log on the box documents what each one does.

9. Hand it to an agent

This guide is structured so an AI agent can execute the mechanical phases: bank parsing, decoding, transcription, selection against the whitelist and the pitch gate, token sharding, training, merging, and the QC battery. Keep yourself in the approval path at four gates: source acquisition (your install, your license), the training run, the merge, and any change to the kiosk voice chain. Tell the agent the source boundary in the same sentence as the task: no scraped audio, no YouTube, nothing redistributed, model weights under CC-BY-NC mean non-commercial only. A voice trained on audio you do not own is a problem no flag can fix later.

Recap: the game ships dialogue in an FMOD bank with a name table that maps 31,433 streams to real dialogue paths; decode the Cortana subset, find her by the folder that says cortana, and gate every file on hash, path, pitch, and formant after a male announcer snuck into the corpus. Prove the engine with a zero-shot spike before training anything, bake the corpus in with a short LoRA run, merge to a standalone model, rebuild the prompt, and ship it as one sidecar on one port behind a fallback chain so the kiosk never goes silent. One day, zero dollars, and the family kiosk now answers in her voice. The kiosk itself, brain and all, is documented in the companion guide.