Research Outcomes

In collaboration with Professor Elke Reinhuber, we adopt the theme of "Urban Beautician" to transform micro-level urban restoration practices—such as wall paintings and other creative interventions—into interactive virtual narratives through volumetric video and AR technology.

On September 20, 2025, the Beijing Cultural and Arts Foundation's 2024 funded project "Immersive Peking Opera Cultural Dissemination: The Legend of the White Snake VR Interactive Narrative Exhibition" opened at Qianmen in Beijing.

Opera embodies deep cultural heritage and traditional lifestyles and deserves engaging promotion and preservation. However, traditional Chinese opera performances and experiences often fall short of meeting the expectations of modern audiences. This paper focuses on using the latest technologies to set up a bridge that not only preserves and promotes traditional cultural heritage but also aligns with modern viewing habits. Through volumetric video capture and 3D reconstruction, we captured high-fidelity performances of opera artists and recreated an ancient stage with interactive particle effects to build an experience in virtual reality (VR). Users can engage with the environment through their singing and gesture movement where the amplitude of Cantonese opera vocals and gestures will influence interactive point cloud particles, creating a dynamic atmosphere. This experience aims to trigger the curiosity of younger audiences in the appreciation of traditional performing arts.

In this exhibition, we created a guided tour video for Ms. Liping Shi using volumetric video technology. By integrating the digital avatar of Ms. Shi produced in the laboratory with a virtual Miao Embroidery studio, we comprehensively captured her movements, expressions, and voice, authentically recreating her process of explaining Miao Embroidery techniques.

We propose DREAMS Avatar Dataset, including multi-view captures of 6 subjects. Each subject performs two sequences. The first sequence is of standard body motions and required facial expressions including smile, laugh, angry, surprise, etc,. The second sequence is a freestyle.

We present DEGAS (Detailed Expressions on Full-Body Gaussian Avatars), the first 3D Gaussian Splatting (3DGS)-based modeling of full-body avatars with rich facial expressions.