Tavus, a leading AI video research company backed by Sequoia, announced the release of Hummingbird-0 into research preview, a zero-shot lip sync model created from components of its flagship Phoenix-3 replica model. Now, with just one video and any voice track, developers can bring faces to life-instantly-without model training or manual tweaking. This step up in quality opens the door to high-quality user-generated content, foreign language dubbing for localization, and personalized videos created at scale, in minutes.
“Lip sync technology has been around for years, but until now, it’s never really been great — open source or otherwise,” said Effie Goenawan, Head of Product at Tavus. “With Hummingbird-0, we’re giving developers access to a state-of-the-art lip sync model that unlocks an entirely new level of creative potential. It actually emerged as a happy accident while we were developing our full-face replica rendering model, Phoenix-3, and it’s a testament to the brilliance and curiosity of our research team.”
Helping Content Creation Take Flight
The Hummingbird model is designed to modify the lip movements in a given video to match the content of a driving audio signal. The guiding principle is to preserve the original identity, expressions, and visual quality of the person in the video while synchronizing their lip movements with the new audio.
Also Read: EDO Launches Engaged Audience Planning, Empowering Brands, Agencies, Networks, and Streamers to Elevate Advanced Audiences and Outcomes-Based Planning for 2025 Upfronts Decisions
Notably, with Hummingbird-0, users can create content much faster because they don’t have to train a model. All that’s needed is a video of a person speaking–one already in existence or one created using a video generator like Veo or Kling. From making memes talk to instantly localizing thousands of B2B videos, Hummingbird-0 puts high-quality lip sync just an API call away.
“Text-to-video generation models have become enormously popular for content creation, but there is a problem in that the video is muted; there’s no voice,” said Hassaan Raza, CEO of Tavus. “We are adding that voice that can go on top of any video where there is a human. This serves as an enabler not just for more, different, or better content, but for new types of products and experiences altogether. Once developers get a taste of Hummingbird-0, they want to know more about what our entire family of models can do. Hummingbird-0 barely scratches the surface of our capabilities as we continue developing the human layer of AI.”
SOURCE: Businesswire