Voice cloning with 1 minute of audio
I recorded 1 minute of my voice reading a Chinese paragraph. GPT-SoVITS processed it in 3 steps: ASR (transcribe the audio to text), VITS training (learn the voice characteristics), and GPT generation (generate speech from text). The whole pipeline took 8 minutes on my RTX 3090. The cloned voice had my tone, pacing, and accent. It sounded about 85% like a real recording. The remaining 15% difference was in emotional expression: the clone sounds slightly flat compared to a real voice. For narration and instructional content, 85% is good enough. For emotional storytelling, it is not there yet.
Chinese quality is exceptional, English is catching up
Chinese speech generation is the primary focus and it shows. Tones are accurate 95% of the time, word boundaries are natural, and the output sounds like a native speaker. For English, the voice has a slight Chinese accent and mispronounces about 8% of words (mostly uncommon technical terms). GPT-SoVITS v2 improved English significantly: accent reduced by about 40% compared to v1. For Chinese-English mixed content (common in tech presentations), the quality is 85% acceptable. For pure English content, use ElevenLabs or OpenAI TTS.
Hardware requirements and inference speed
Voice cloning training: needs 8GB+ VRAM. My RTX 3090 processed 1 minute of audio in 8 minutes. A 6GB GPU would take 20-30 minutes. CPU-only: 2-3 hours per minute of training audio. Inference (generating speech): 2-3 seconds per sentence on GPU, 30 seconds per sentence on CPU. The web UI is built with Gradio and works on Windows and Linux. Installation requires Python 3.10, PyTorch with CUDA, and about 5GB of model downloads. The Chinese community has detailed tutorials on Bilibili, but English documentation is sparse.
What it does better than ElevenLabs for Chinese
GPT-SoVITS handles Chinese tones better than ElevenLabs. The four Mandarin tones (mā, má, mǎ, mà) are correctly generated 95% of the time vs about 85% for ElevenLabs Chinese voices. It also handles Chinese-English code-switching more naturally: '我们使用Python的asyncio库来handle并发请求' comes out with the correct English pronunciation of 'Python', 'asyncio', and 'handle' instead of reading them as Chinese characters. For Chinese content creators, this makes GPT-SoVITS the better choice even though ElevenLabs has a more polished UI.
Ethics and what to watch for
Voice cloning is a dual-use technology. GPT-SoVITS requires explicit consent for voice cloning and the authors have a usage policy against impersonation. In practice, the tool has no built-in consent verification: it clones whatever audio you give it. If you are using this for commercial content, get written permission from the voice owner. For my test, I cloned my own voice, which sidesteps the ethical concerns entirely. For any voice that is not yours, the legal landscape is still evolving.
