ESPnet released a TTS model on the VITS architecture for speech synthesis

The team of ESPnet introduced a new speech synthesis model trained on the LJSpeech dataset. The release is based on the VITS architecture, which combines generative adversarial networks and variational autoencoders to achieve high naturalness of sound.

The model demonstrates an evaluation of 226 points, obtained during testing for 1 hour of work. The tool is fully integrated into the ESPnet2 ecosystem, allowing users to deploy the system via the standard pip package manager or use the built-in espnet_tts commands to generate an audio stream.

The VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) architecture allows the model to directly convert input text into a wave form, bypassing intermediate stages of spectrogram generation. This reduces delays in processing and improves the quality of prosody in synthesized speech.

This release is targeted at developers using the ESPnet framework to create applied speech recognition and synthesis systems. The model is available for download in the repository on the Hugging Face platform, where configuration files are also presented for fine-tuning synthesis parameters for specific tasks.

The update expands the library of pre-trained solutions for working with voice, providing a ready-to-use tool for text voiceover tasks. Using VITS as part of ESPnet2 simplifies the implementation of high-quality TTS in local applications without the need to train a model from scratch.