The Hong Kong University of Science and Technology (Guangzhou) has developed an AI-driven 3D digital human technology known as DEGAS (Detailed Expressions on Full-Body Gaussian Avatars). This innovative system utilizes 3D Gaussian Splatting (3DGS) to create realistic full-body avatars capable of rich facial expressions and body movements.

Hybrid 3D Representation

DEGAS employs a hybrid 3D representation that integrates Gaussian splatting with a triangle mesh. This approach allows for the disentanglement of motion and appearance, with explicit mesh geometry capturing low-frequency motion and surface deformation, while Gaussian splatting models high-frequency geometry and detailed appearance. The result is a photorealistic human avatar that can render at over 300 frames per second (FPS) on modern GPUs and 30 FPS on mobile devices.

Real-Time Facial and Body Animation

By incorporating real-time facial and body animation techniques, DEGAS enables full-body 3D digital human avatars to exhibit lifelike expressions and movements. The system directly controls the rotation and translation of the Gaussians through the mesh, ensuring compatibility with various animation methods, including skeletal animation and blend shapes. This design facilitates seamless integration into interactive applications, providing users with an immersive experience.

LLM and Voice Clone

Our digital human achieves authentic characterization by utilizing customized large language models(LLM) and voice cloning models. By converting voice signals into motion signals, we add a new and vivid mode to the use of the traditional language models.

Applications and Impact

The development of DEGAS opens new avenues in fields such as virtual reality, gaming, and digital communication, where high-quality, real-time avatars enhance user engagement. The technology's efficiency and compatibility with existing animation techniques make it a versatile tool for creating dynamic digital humans.

For more information and access to the project's resources, please visit [DEGAS].