ESPnet released a Chinese speech recognition model on the Conformer architecture

Team ESPnet introduced a new model for automatic speech recognition (ASR), optimized for processing the Chinese language. The system is built on the Conformer architecture and was trained on a large-scale WeNetSpeech dataset.

The model demonstrates WER (Word Error Rate) scores ranging from 13.4% to 68.0% depending on the type of test dataset. Training the system took 1 hour on a specialized computing cluster, which allowed achieving a score of 12 points on the project's internal metric.

The solution is focused on transcription and audio stream processing tasks in the Chinese language. Using the Conformer architecture provides a balance between local and global dependencies in the audio signal, which is critical for accurate recognition of tonal languages.

The model is available for integration within the ESPnet2 ecosystem, which provides tools for creating end-to-end speech recognition and synthesis systems. Developers can use pre-trained weights for fine-tuning on their own specific domains or for deploying ready-made transcription services.