Xiaomi Open-Sources ControlFoley for Enhanced Video Sound Generation

News Flash 2026-05-29 19:04

Xiaomi has released ControlFoley, an open-source framework designed to enhance video sound effects generation. Unlike traditional AI dubbing models that infer sounds from visuals, ControlFoley offers creators precise control over audio style by generating sounds based on video content and accepting text descriptions or reference audio. This allows for transformations such as converting a knock into a "metal strike" while maintaining synchronization with video visuals. ControlFoley utilizes a spatiotemporal audiovisual encoder and a "time-timbre decoupling" strategy, achieving state-of-the-art performance on standard video dubbing benchmarks. It competes closely with commercial systems like Kling-Foley in metrics such as semantic alignment and synchronization, though it slightly underperforms in certain KL divergence metrics. The framework's technical report, code, and demo are now publicly accessible.

Share to:

This content is for informational purposes only and does not constitute investment advice.

Curated Series

SuperEx Popular Science Articles Column

SuperEx Popular Science Articles Column

This collection features informative articles about SuperEx, aiming to simplify complex cryptocurrency concepts for a wider audience. It covers the basics of trading, blockchain technology, and the features of the SuperEx platform. Through easy-to-understand content, it helps users navigate the world of digital assets with confidence and clarity.

Unstaked related news and market dynamics research

Unstaked related news and market dynamics research

Unstaked (UNSD) is a blockchain platform integrating AI agents for automated community engagement and social media interactions. Its native token supports governance, staking, and ecosystem features. This special feature explores Unstaked’s market updates, token dynamics, and platform development.

XRP News and Research

XRP News and Research

This series focuses on XRP, covering the latest news, market dynamics, and in-depth research. Featured analysis includes price trends, regulatory developments, and ecosystem growth, providing a clear overview of XRP's position and potential in the cryptocurrency market.

How do beginners trade options?How does option trading work?

How do beginners trade options?How does option trading work?

This special feature introduces the fundamentals of options trading for beginners, explaining how options work, their main types, and the mechanics behind trading them. It also explores key strategies, potential risks, and practical tips, helping readers build a clear foundation to approach the options market with confidence.

What are the risks of investing in cryptocurrency?

What are the risks of investing in cryptocurrency?

This special feature covers the risks of investing in cryptocurrency, explaining common challenges such as market volatility, security vulnerabilities, regulatory uncertainties, and potential scams. It also provides analysis of risk management strategies and mitigation techniques, helping readers gain a clear understanding of how to navigate the crypto market safely.