Hello Hollywood AI Reader,

Welcome to this week’s edition of Hollywood AI! If you’re looking to stay current on everything AI in Hollywood, you’ve come to the right place! Here’s what’s on tap today:

• ✍🏼 The Future of Localization: AI And Its Impact On Subtitling And Dubbing

• 🗣 VALL-E: Microsoft’s New Text-To-Speech Tool

• 🍿 Hollywood AI Kernels

✍🏼 The Future of Localization: AI And Its Impact on Subtitling and Dubbing

Subtitling and dubbing are the two primary methods used to translate films from one language to another. Subtitling involves adding text at the bottom of the screen that translates the original dialogue into another language. Dubbing involves replacing the original dialogue stream with another language. Both of these workflows require extensive manual labor. As a result, the use of AI in subtitling and dubbing has become increasingly popular in the industry.

According to a report by Grand View Research published in July 2021, the global language services market size was valued at USD 53.45 billion in 2020 and is expected to grow at a compound annual growth rate of 6.7% from 2021 to 2028. Localization services, including subtitling and dubbing, are a significant component of the language services market.

Hollywood produces a large number of films and TV shows that are distributed globally. The localization market in Hollywood is a significant contributor to the overall language services market. With the growth of streaming services and the increasing demand for content in multiple languages, it’s likely that the localization market in Hollywood will continue to grow for years to come.

AI-powered subtitling and dubbing software have made it easier to produce high-quality content in multiple languages. The use of AI has also made the production process faster and more efficient, reducing the cost of translating content. AI translations have come a long way in recent years, and their accuracy has significantly improved. However, whether or not AI translations are as accurate as human translations depends on several factors.

In general, AI translations are most accurate when translating content that is relatively simple, such as short phrases or sentences with straightforward grammar and vocabulary. When it comes to more complex content, such as a feature length film, AI translations are not nearly as accurate as human translations.

AI translations rely on statistical algorithms and machine learning models that are trained on vast amounts of data to identify patterns in language and make predictions about how to translate text. While these models are capable of producing accurate translations, they don’t always capture the nuances of language, cultural references, jokes, or idiomatic expressions that can impact the meaning of a text.

On the other hand, human translators have a deep understanding of language, culture, and context that allows them to produce translations that are not only accurate but also culturally appropriate and nuanced. Human translators are also able to make judgment calls when dealing with ambiguous or unclear text, which is something that AI translations struggle with.

Ultimately, the accuracy of AI translations compared to human translations depends on the specific use case and the quality of the AI translation system being used. For Hollywood studios, AI can help make several manual translation tasks easier and more efficient for human translators. Some examples :

1. Translation Memory: AI-powered translation memory tools can suggest translations for previously translated content, reducing the amount of time required for improving consistency.

2. Key Terms & Phrases: Management tools can help translators maintain consistency in their translations by suggesting terms based on previous translations.

3. Quality Control: AI-powered quality control tools can help human translators identify errors in their translations, such as grammar or spelling mistakes, inconsistent terminology, or mistranslations.

4. Transcription: Speech recognition technology can transcribe audio content into text, saving significant time on a crucial step of the translation process.

Despite the advantages of using AI in subtitling and dubbing, there are some challenges and limitations that need to be considered. One of the most significant challenges is the accuracy of the translation. While AI-powered software can be highly accurate, there are still nuances in language and context that is often not accurately translated. This is obviously important in Hollywood films where accuracy and subtle details are critical to the storyline.

Another challenge is the cost of using AI-powered software. While the software is more efficient than manual labor, it often requires a significant investment. This may be prohibitive for smaller production companies or independent filmmakers operating on tight budgets.

The use of AI in subtitling and dubbing also raises concerns about the loss of jobs. This isn’t something unique to the language services market. When we talk about AI disrupting industries, job losses is a concern on everyone’s mind. Rightfully so.

The truth is that AI translations are not accurate enough to be used in place of professional, human translations. The technology may get there one day, but we’re not there yet. AI is a very powerful tool that can assist Hollywood translators with several time-consuming and mundane tasks. It’s not yet ready to translate a Hollywood feature length film.

🗣 VALL-E: Microsoft’s New Text-To-Speech Tool

Researchers at Microsoft have recently released their newest AI project, VALL-E. VALL-E is a text-to-speech model that can simulate a person’s voice with only a three second audio sample and exhibits learning capabilities competent enough to create new speech, all while matching tone, speech patterns, and room acoustics. When given two prompts: a piece of short text and an audio sample, VALL-E transcribes the information into a format that can read by breaking it down into components. Then, by using this data it’s able to recreate the voice from the sample prompt. Researchers trained the model on 60 thousand hours of existing speech pulled from public domain audio books, which is hundreds of times larger the amount of data than existing models have trained on. VALL-E’s website showcases demos of how well the program works, providing some remarkable results with only one conclusion; VALL-E vastly outperforms the competition.

As the creation of new AI programs continues to rise so do the concerns for cyber security and deep fakes. There is specific concern with this new model, and others like it, being directly linked to a rise in cyber crimes, as voice spoofing could become more accurate. But Microsoft has already released a moral statement regarding the possible risks VALL-E could pose in the future and in turn have not made it available for public use. Researchers at Microsoft intended its use to be for “high quality text to speech applications”. But that may be a little ways away as researchers state it still has several issues.

Recently speech to speech technology has been used to recreate actors voice’s such as Mark Hamill’s in The Mandalorian and Val Kilmer’s in Top Gun: Maverick. Text to speech has not been as commonly used in the industry due to its history of sounding robotic, but with the creation of VALL-E, these concerns no longer pose a problem, opening up the industry to new possibilities.

🍿 Hollywood AI Kernels

AI-generated videos are weird. Google and Meta recently announced that they’re also working on text-to-video tools. Are these videos going to get better soon? → wired

What’s Jeffrey Katzenberg’s Take On AI? All the rage is about generative AI, but he’s excited about something called “sanctioned AI.” More here → yahoo

Iconic Hollywood roles reimagined with Bollywood actors. An Indian design company, Lazy Eight, used an AI program to reimagine top Bollywood actors as iconic Hollywood characters. Watch the video→ youtube

That’s a wrap for today’s newsletter. Thanks for reading Hollywood AI!

If you enjoyed today’s edition, we appreciate you spreading the word! New to Hollywood AI? We got you. Click the Subscribe button below and we’ll make sure you don’t miss out on any of the fun.