Introduction

The laboratory's core equipment is a fully self-developed light-field reconstruction system. This system utilizes a spherical array comprising hundreds of ultra-high-precision industrial cameras and infrared emitters, forming a data acquisition matrix for 360-degree synchronized data capture and reconstruction of people within the array. Concurrently, we are capable of performing high-precision, ultra-realistic 3D reconstruction of both scenes and objects, facilitating the synergistic coexistence of "human-object-scene" within virtual environments. The laboratory is equipped with various VR/AR devices, mocap device and robotic systems, multiple high-end development workstations and servers, all dedicated to developing relevant applications within the field.

Light-Field Studio

Full-stack self-developed light field reconstruction system. This system consists of hundreds of ultra-high-precision industrial cameras and infrared emitters deployed in a spherical arrangement to form a data acquisition matrix. It performs synchronized data collection in a surround manner for individuals/objects capturing, obtaining color and depth information. Collected data is transfered to our local server with 10 Gigabit Ethernet, and then automatically reconstructed into three-dimensional models using algorithms deployed on the cloud. This process generates a dynamic model sequence composed of 22+ continuous three-dimensional models per second, known as volumetric videos.

Specification

Cameras

32x 4K color industrial cameras + 48x 2K IR industrial cameras

Audio capture system

Sensitivity 40dBFS/Pa

Synchronized capture system

Max throughput 10GB/s, delay <1ms

Capture system device

8x i9+RTX3060+64GB memory+2T SSD

Cluster storage

32T HDD

Network

10 Gigabit optical migration LAN system

GPU server

8x RTX3090 + 128G memory + 512G SSD

Processing stat

Single frame <15min, max faces > 10w. Dynamic 22 frames/sec, supporting H.264 format.