Introduction

The laboratory's core equipment is a full-stack self-developed light field reconstruction system, which consists of hundreds of ultra-high-precision industrial cameras and infrared emitters deployed in a spherical arrangement. It performs synchronized data collection in a surround manner. The collected data is then automatically reconstructed into 3D models using algorithms deployed on the cloud, and it generates a dynamic model sequence, known as volumetric videos. We also have devices such as the Apple Vision Pro and other VR/AR equipment, multiple high-performance development machines and servers, dedicated to development in relevant fields.

Light-Field Studio

Full-stack self-developed light field reconstruction system. This system consists of hundreds of ultra-high-precision industrial cameras and infrared emitters deployed in a spherical arrangement to form a data acquisition matrix. It performs synchronized data collection in a surround manner for individuals/objects capturing, obtaining color and depth information. Collected data is transfered to our local server with 10 Gigabit Ethernet, and then automatically reconstructed into three-dimensional models using algorithms deployed on the cloud. This process generates a dynamic model sequence composed of 22+ continuous three-dimensional models per second, known as volumetric videos.

Specification

Cameras

32x 4K color industrial cameras + 48x 2K IR industrial cameras

Audio capture system

Sensitivity 40dBFS/Pa

Synchronized capture system

Max throughput 10GB/s, delay <1ms

Capture system device

8x i9+RTX3060+64GB memory+2T SSD

Cluster storage

32T HDD

Network

10 Gigabit optical migration LAN system

GPU server

8x RTX3090 + 128G memory + 512G SSD

Processing stat

Single frame <15min, max faces > 10w. Dynamic 22 frames/sec, supporting H.264 format.