Kokoro can now clone any voice with just ~600 MB of extra VRAM!

Wait 5 sec.

While looking for audio models, I came across OpenVoice. It's getting pretty old, but surprisingly still ranks high on TTS leaderboards. What caught my attention was its design: essentially TTS + voice conversion. The ToneColorConverter can run independently of the TTS model. That got me thinking: Can we pair it with other TTS models? I tried it with Kokoro, and the results were pretty good! So I added a small recipe any-tts to audio.cpp for Kokoro + OpenVoice ToneColorConverter. Recipes were introduced in v0.9.2 as predefined pipelines that combine multiple audio models. The overheads: ~600 MB additional peak VRAM ~10% slowdown on CUDA ~0.5× the original speed on CPU, but still 2× real-time You can also try pairing any tiny TTS model with ToneColorConverter. The TTS model determines the speech content, pronunciation, and prosody, while the VC model transfers the target speaker's voice characteristics. The built-in voice you choose can also affect the final output quality. A simple way to give existing TTS models voice cloning capabilities without retraining them!   submitted by   /u/Acceptable-Cycle4645 [link]   [comments]