Global: Meta Introduces Advanced AI Model for Speech and Text Translations

Meta has introduced a cutting-edge artificial intelligence (AI) model capable of conducting speech and text translations across nearly 100 languages.

Unveiled as SeamlessM4T, this new AI model, as Meta highlighted in a press release on Tuesday (August 22), offers a comprehensive solution for multimodal and multilingual translation tasks. It encompasses speech recognition, speech-to-text translation, speech-to-speech translation, text-to-text translation, and text-to-speech translation functionalities.

The distinct advantage of SeamlessM4T lies in its unified system approach, effectively enhancing efficiency and precision by mitigating errors and minimizing delays during the translation process, as stated in the release.

SeamlessM4T is now available under a research license, enabling researchers and developers to further build upon its capabilities. Furthermore, Meta is also releasing the metadata of SeamlessAlign, an open multimodal translation dataset encompassing approximately 270,000 hours of collected speech and text alignments. This dataset will serve as a valuable resource for future research and advancements within the field.

The development of SeamlessM4T builds upon Meta’s prior achievements in language translation technology. Last year, the company introduced No Language Left Behind (NLLB), a text-to-text machine translation model accommodating 200 languages. NLLB has been successfully integrated into Wikipedia as a designated translation provider.

Meta has also demonstrated the Universal Speech Translator, the first direct speech-to-speech translation system for Hokkien, a Chinese dialect without a widely standardized writing system. Earlier this year, Meta launched Massively Multilingual Speech, a technology offering speech recognition, language identification, and speech synthesis capabilities across an impressive array of over 1,100 languages.

SeamlessM4T incorporates insights and knowledge gleaned from these diverse projects, providing an exceptional multilingual and multimodal translation experience, the release emphasized.

Notably, Meta’s progress in harnessing AI for universal language translations was reported in February 2022, with a focus on enhancing spoken interactions with voice assistants. Demonstrating the capabilities of their AI, a voice assistant detected low salt supplies as a family prepared a meal and promptly ordered more.

Furthermore, this technology offers remarkable potential for bridging linguistic gaps in various applications, like generative AI-powered language translation services that facilitate seamless communication between customers and service providers, thereby expanding the accessibility of telecom services to diverse markets.