F5-TTS

F5-TTS is a fully non-autoregressive text-to-speech system built on flow matching with a Diffusion Transformer architecture, developed by researchers at Shanghai Jiao Tong University, the University of Cambridge, and Gile Auto Research Institute. It was released publicly in October 2024 alongside its arXiv paper (arXiv:2410.06885) and was accepted into the main proceedings of ACL 2025, one of the premier venues in natural language processing. The project has accumulated 14.4k GitHub stars, over 600,000 monthly Hugging Face downloads, and 89 community-contributed fine-tuned variants, making it one of the most adopted open-source TTS systems since its launch. The code is MIT-licensed; the pre-trained model weights carry a CC-BY-NC-4.0 license due to the Emilia training dataset.

The model can clone a voice from 3-15 seconds of reference audio without any fine-tuning, producing natural-sounding speech with the reference speaker's timbre and cadence. It ships two primary model variants: F5TTS_Base (the original October 2024 release) and F5TTS_v1_Base (March 2025, with improved quality and faster inference). Both variants run on NVIDIA, AMD, Intel, and Apple Silicon hardware. A dedicated MLX port by Lucas Newman produces samples in roughly 4 seconds on an M3 Max. For developers who want programmatic access without a hosted API, F5-TTS integrates directly into Python pipelines via its pip package, a Gradio web UI, or CLI. It pairs naturally with Whisper for transcription-to-speech workflows.

What F5-TTS produces in April 2026

The model's output is stereo or mono audio generated from a Vocos or BigVGAN vocoder, with a real-time factor of 0.15, meaning it generates audio approximately 6.7 times faster than real time on supported hardware. Inference on most sentence-length inputs runs in under 7 seconds. The Sway Sampling inference strategy, introduced in the original paper, significantly reduces the denoising steps needed without sacrificing quality, giving it a practical edge over earlier diffusion-based TTS systems that required 50-100 DDPM steps.

Primary language support covers English and Chinese (Mandarin) with seamless code-switching between them mid-utterance, a capability the paper specifically benchmarks. The architecture's simplicity has attracted community fine-tunes for Polish, Arabic, Turkish, Indonesian, and other languages, though these require 10-100 hours of in-language training data to reach zero-shot capability. The F5TTS_v1_Base model, released March 12, 2025, improved on the base model's prosody and intelligibility and is the recommended starting point for new deployments as of April 2026.

Hardware requirements are approachable by open-source standards. The model loads cleanly in approximately 3GB of GPU VRAM. Apple Silicon users can run the MLX-native port without the CUDA stack. CPU-only inference is possible but slow. The project ships Docker images and a Gradio interface for both inference and fine-tuning, making it deployable on a home workstation or a cloud GPU instance. For teams building voice pipelines, F5-TTS output can be chained with AudioRead for downstream audio processing.

Where F5-TTS sits versus OpenVoice V2 and Coqui XTTS-v2

The meaningful differences between these three models are architectural, not superficial.

OpenVoice V2 (MyShell AI, April 2024): OpenVoice uses a two-stage pipeline. First, a base TTS model (MeloTTS) generates natural speech in a target language. Then a separate lightweight tone color converter module, built on VITS-style speaker encoding and decoding, clones the reference voice's timbre over the top. This decoupling means synthesis and cloning are independent steps. The benefit is explicit, granular control over emotion, accent, rhythm, pauses, and intonation as separate dials. The limitation is that output quality is bounded by the base TTS model, and the two-stage approach introduces artifacts when timbre conversion is applied aggressively. OpenVoice V2 also carries a fully permissive MIT license, meaning it can be used in commercial products without the training-from-scratch requirement that F5-TTS imposes. OpenVoice V2 natively supports English, Spanish, French, Chinese, Japanese, and Korean out of the box.

Coqui TTS / XTTS-v2 (defunct company, model still active): XTTS-v2 uses a GPT-2-style autoregressive model for token prediction paired with a HiFi-GAN vocoder. Because it generates audio tokens sequentially, it supports streaming with approximately 200ms time to first chunk, making it suitable for real-time voice applications. F5-TTS's non-autoregressive flow matching approach generates all frames in parallel via denoising, which produces faster bulk synthesis but makes native streaming architecturally difficult. XTTS-v2's stronger advantage is language coverage: 17 languages including German, Italian, Portuguese, Russian, Polish, Turkish, Dutch, Czech, Arabic, Japanese, Hungarian, Korean, and Hindi, versus F5-TTS's primary English and Chinese. Coqui AI shut down in early 2024; the model is maintained by community fork "coqui-tts" on PyPI but faces growing compatibility issues with modern PyTorch versions. For teams needing voice cloning in any of those 17 languages today, XTTS-v2 remains the default open-source option despite the maintenance risk.

"The cloning it did with the 10-second ref is surprisingly good. I did not expect it to give such a successful result in such a simple way." - srkngl, HuggingFace Discussions (SWivid/F5-TTS #12), November 28, 2024

Against commercial alternatives like ElevenLabs or PlayHT, F5-TTS trades convenience and multilingual breadth for zero ongoing API cost and full data privacy. Every generation stays on your hardware. For developers building voice features into applications with high volume or sensitive audio data, that tradeoff is frequently worth the setup overhead.

The licensing and copyright reality

F5-TTS has a split licensing structure that catches many developers off guard. The code repository is MIT-licensed, which is fully permissive: you can fork it, modify it, and use it in commercial software without restriction. The pre-trained model weights, however, are CC-BY-NC-4.0 due to their training on the Emilia dataset, which operates under its own CC-BY-NC terms.

The practical implication: you cannot use the official F5TTS_Base or F5TTS_v1_Base model weights in a commercial product, and this restriction survives fine-tuning. The maintainer confirmed this directly in GitHub Discussion #997: "CC-BY-NC Emilia trained Base Model cannot be used commercially also after finetuning." To build a commercial voice product on the F5-TTS architecture, you must train a new base model from scratch using commercially-permissive data, without loading any Emilia-derived weights as a starting point. This requires at minimum 10-100 hours of in-language audio and meaningful GPU budget, which is non-trivial for independent developers.

For non-commercial projects (research, education, personal tools, hobbyist projects), this licensing structure poses no obstacle. The model is genuinely free to use for those purposes, and the permissive code license means you can deploy it however you like. But if you are building a SaaS product, a commercial voice API, or any monetized application, review this constraint carefully before integrating F5-TTS weights. This is the key advantage OpenVoice V2 holds with its MIT weights, and why that model sees more commercial adoption despite arguably lower raw output quality. Tools like ComfyUI have similarly navigated community pressure around restrictive model weights by maintaining clean separation between MIT-licensed framework code and model-specific terms.

Where F5-TTS reliably falls short

Several recurring issues appear across GitHub Issues, HuggingFace Discussions, and community channels:

Long-text speed acceleration: The most consistently reported bug. At approximately 200 words, speech speed is normal. At 300 words, it noticeably accelerates. At 500+ words, the output is too fast to understand. The root cause is a duration calculation formula in utils_infer.py that underestimates required time as text length grows. User hotdogarea documented the progressive compression in GitHub Issue #811 (February 2025): 42 characters at 0.051 sec/char, 146 characters at 0.014 sec/char, 374 characters at 0.006 sec/char. The issue was closed but the underlying behavior persists in long-document workflows.

Dependency drift causing gibberish output: Between January and March 2025, multiple users on the public HuggingFace demo (mrfakename/E2-F5-TTS) experienced audio that degraded into unintelligible garbled output after initially working correctly. User bigbrotherr described it as "gibberish and garbage" after 13 attempts. The root cause was dependency drift between the Space and the main repository; the fix required re-cloning in a fresh virtual environment. This is an expected reality of self-hosted models but worth planning for in production deployments.

"I'm using the same audio and text that worked before, but after not using it since 1/10/25, the output is now just gibberish." - gmirsky2, HuggingFace Spaces (mrfakename/E2-F5-TTS discussion #48), March 11, 2025

macOS installation complexity: Running F5-TTS on Apple Silicon requires careful setup, specifically avoiding conda-managed CUDA packages (which don't exist for M2/M3) and enabling the MPS fallback via environment variable. Users who install without understanding these constraints commonly see poor audio quality or crashes before resolving the configuration.

Non-streaming output: The flow matching architecture is inherently non-autoregressive: it generates all audio frames via a parallel denoising process. This means you cannot stream audio to a speaker as tokens arrive, the way XTTS-v2 can. The full clip must be generated before playback begins. For conversational voice applications or low-latency real-time use, this is a hard architectural limit.

Limited out-of-box language coverage: Despite the paper's framing as "multilingual," in practice the default model handles English and Chinese well. Other languages require fine-tuning with substantial in-language data. The GitHub discussions forum shows active community efforts for Arabic, Polish, Indonesian, and Turkish, but these are community-contributed, not officially maintained.

Who F5-TTS is for

Use it when: You need high-quality voice cloning for non-commercial English or Chinese content and want zero ongoing API costs. It excels for audiobook narration (generate chapters from a 15-second reference clip of your own voice), indie game dialogue (generate all NPC lines locally without per-character fees), and podcast production (maintain consistent voice across content without re-recording). Researchers working on TTS, voice conversion, or speech synthesis will find the MIT code license and clean DiT architecture easy to extend. If you are building on Apple Silicon and want a native MLX workflow without CUDA, the community port is production-ready. F5-TTS also integrates naturally with Whisper for closed-loop transcription-to-cloned-speech pipelines, and with Coqui TTS fine-tuning patterns for teams already familiar with open-source voice workflows.

Skip it when: Your project is commercial and you cannot train from scratch on CC-BY-compatible data. Skip it if you need robust coverage of more than two languages without significant fine-tuning investment. Skip it if your application requires real-time streaming audio (low-latency conversational assistants, live voice synthesis). Skip it if your team lacks Python environment management experience, as dependency issues are the most common failure mode. For commercial-grade multilingual TTS, ElevenLabs or PlayHT offer managed APIs with broader language coverage, hosted reliability, and clear commercial terms, at per-character costs that are worth evaluating against the operational overhead of self-hosting.