Wan-Animate-2: Pushing the Application Boundaries of Character Animation

Tongyi Lab,Alibaba Group
arXiv

Abstract

Character image animation remains a foundational yet challenging task in computer vision. Existing approaches can be broadly categorized into three paradigms: methods based on explicit motion representations suffer from extraction errors and identity drift; methods based on implicit motion features lose fine-grained dynamics through compression; and in-context learning approaches avoid intermediate representations but incur prohibitive computational costs. Furthermore, all current systems are designed for offline synthesis, unable to meet the real-time requirements of interactive applications such as digital avatars and live-streaming hosts. To address these limitations, we present Wan-Animate-2, an end-to-end character animation framework that directly consumes the driving video within a redesigned Diffusion Transformer. Our architecture achieves superior motion fidelity and identity preservation by eliminating intermediate motion extractors entirely. We further introduce text-driven viewpoint control that decouples the output camera perspective from the driving video—a capability rarely supported by prior character animation methods that rely on explicit motion representations. Beyond generation quality, we present Wan-Animate-2-Lite, an efficient variant that reduces inference latency to real-time thresholds through a three-stage training paradigm: teacher forcing pretraining with error buffer mechanism, and Self-Forcing distillation with chunk-wise backpropagation. This enables streaming character animation for interactive applications, opening new deployment scenarios that were previously infeasible. Qualitative evaluations and user studies demonstrate that Wan-Animate-2 achieves high-fidelity animation results across diverse characters and motion patterns. To foster further research and community development, we will release the Wan-Animate-2-Base model weights to the public.

Method

MY ALT TEXT

Overview of our framework. Our method conditions video generation on a reference image and a reference video through a dual-branch DiT architecture. To effectively fuse static appearance and dynamic motion cues, we introduce Time-Align RoPE for temporal alignment between denoising video tokens and reference tokens, and Sparse-Ref Attention to selectively attend to informative reference features.

Results

High Fidelity Character Animation

Wan-Animate-2 achieves superior visual fidelity with enhanced dynamic details, enabling the replication of intricate motions, nuanced facial expressions, and physically plausible character-scene interactions. It exhibits exceptional robustness and stability across diverse input combinations of varied aspect ratios and character morphologies.

Multi-Character Animation

Wan-Animate-2 enables controllable multi-person video synthesis, supporting versatile animation paradigms such as single-to-multiple and multiple-to-multiple motion driving.

Multi-Camera Animation

Wan-Animate-2 enables view-controllable synthesis, allowing for multi-angle observations of the same character animation via explicit camera pose conditioning. Since motion is not tied to rigid spatial anchors, the viewpoint can be effectively decoupled from the reference information, ensuring robust multi-view content consistency.

Real-Time Streaming Animation

Wan-Animate-2-Lite enables real-time streaming character re-enactment directly from live camera feeds, ensuring seamless synchronization of both motions and facial expressions with minimal latency.

Comparisons with SOTAs