Title of paper:
Audio-Visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation
Authors:
Fa-Ting Hong, Zunnan Xu, Zixiang Zhou, Jun Zhou, Xiu Li, Qin Lin, Qinglin Lu, Dan Xu
Description:
This paper introduces ACTalker, an end-to-end video diffusion framework designed for natural talking head generation with both multi-signal and single-signal control capabilities.
The framework employs a parallel Mamba structure with multiple branches, each utilizing a separate driving signal to control specific facial regions.
A gate mechanism is applied across all branches, providing flexible control over video generation.
To ensure natural coordination of the controlled video both temporally and spatially, the Mamba structure enables driving signals to manipulate feature tokens across both dimensions in each branch.
Additionally, a mask-drop strategy is introduced, allowing each driving signal to independently control its corresponding facial region within the Mamba structure, preventing control conflicts.
Experimental results demonstrate that this method produces natural-looking facial videos driven by diverse signals, and that the Mamba layer seamlessly integrates multiple driving modalities without conflict.
Link of abstract paper:
https://arxiv.org/abs/2504.00000
Link of download paper:
https://arxiv.org/pdf/2504.00000.pdf
Code:
https://github.com/harlanhong/actalker
Datasets used in paper:
The paper does not specify the datasets used.
Hugging Face demo:
No Hugging Face demo available.
#ACTalker #TalkingHeadGeneration #VideoDiffusion #MultimodalControl #MambaStructure #DeepLearning #ComputerVision #AI #OpenSource

✨RELIC: Interactive Video World Model with Long-Horizon Memory

📝 Summary:
RELIC is a unified framework enabling real-time, memory-aware exploration of scenes with user control. It integrates long-horizon memory and spatial consistency using video-diffusion distillation, achieving 16 FPS generation with robust 3D coherence.

🔹 Publication Date: Published on Dec 3

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.04040
• PDF: https://arxiv.org/pdf/2512.04040
• Project Page: https://relic-worldmodel.github.io/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#WorldModels #VideoDiffusion #DeepLearning #RealTimeAI #ComputerVision

✨SpaceTimePilot: Generative Rendering of Dynamic Scenes Across Space and Time

📝 Summary:
SpaceTimePilot is a video diffusion model for dynamic scene rendering, offering independent control over spatial viewpoint and temporal motion. It achieves precise space-time disentanglement via a time-embedding, temporal-warping training, and a synthetic dataset.

🔹 Publication Date: Published on Dec 31, 2025

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.25075
• PDF: https://arxiv.org/pdf/2512.25075
• Project Page: https://zheninghuang.github.io/Space-Time-Pilot/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoDiffusion #GenerativeAI #DynamicScenes #ComputerGraphics #DeepLearning

✨SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer

📝 Summary:
SALAD improves video Diffusion Transformers by combining linear and sparse attention with an input-dependent gating mechanism. It achieves 90% sparsity and a 1.72x speedup while maintaining quality and requiring minimal finetuning data.

🔹 Publication Date: Published on Jan 23

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2601.16515
• PDF: https://arxiv.org/pdf/2601.16515

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoDiffusion #Transformers #Sparsity #EfficientAI #DeepLearning

✨Efficient Autoregressive Video Diffusion with Dummy Head

📝 Summary:
Autoregressive video diffusion models underutilize historical frames. Dummy Forcing improves efficiency through heterogeneous memory allocation and dynamic head programming. This method achieves up to 2.0x speedup with less than 0.5% quality drop, enabling faster video generation.

🔹 Publication Date: Published on Jan 28

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2601.20499
• PDF: https://arxiv.org/pdf/2601.20499
• Project Page: https://csguoh.github.io/project/DummyForcing/
• Github: https://github.com/csguoh/DummyForcing

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#VideoDiffusion #AutoregressiveModels #GenerativeAI #DeepLearning #AI