Meta Platforms has released an AI model capable of translating and transcribing speech in dozens of languages, a potential building-block for tools enabling real-time communication across language divides.
The company said in a blog post that its SeamlessM4T model could support translations between text and speech in nearly 100 languages, as well as full speech-to-speech translation for 35 languages, combining technology that was previously available only in separate models.
CEO Mark Zuckerberg has said he envisions such tools facilitating interactions between users from around the globe in the metaverse, the set of interconnected virtual worlds on which he is betting the company’s future.
Meta is making the model available to the public for non-commercial use, the blog post said.
The world’s biggest social media company has released a flurry of mostly free AI models this year, including a large language model called Llama that poses a serious challenge to proprietary models sold by OpenAI and Google.
Zuckerberg says an open AI ecosystem works to Meta’s advantage, as the company has more to gain by effectively crowd-sourcing the creation of consumer-facing tools for its social platforms than by charging for access to the models.
Nonetheless, Meta faces similar legal questions as the rest of the industry around the training data ingested to create its models.
In July, comedian Sarah Silverman and two other authors filed copyright infringement lawsuits against both Meta and OpenAI, accusing the companies of using their books as training data without permission.
For the SeamlessM4T model, Meta researchers said in a research paper that they gathered audio training data from 4 million hours of “raw audio originating from a publicly available repository of crawled web data,” without specifying which repository.
A Meta spokesperson did not respond to questions on the provenance of the audio data.
Text data came from datasets created last year that pulled content from Wikipedia and associated websites, the research paper said.