Opportunity
The rapid growth of Extended Reality (XR) and the Metaverse has increased demand for high-fidelity 3D representations, such as point clouds, for applications like virtual social interaction, e-commerce, and digital twins. However, point clouds—particularly those representing human or animal bodies—require immense storage and bandwidth due to their high resolution and large data volumes. For instance, a single high-resolution human point cloud can occupy 11 MB, while a 10-second volumetric video sequence may consume 3 GB. Traditional compression methods (e.g., MPEG’s V-PCC and G-PCC) and learning-based approaches often ignore the inherent geometric priors of structured objects (e.g., human bodies), leading to redundancy in encoding. This inefficiency motivates the development of a compression method that leverages geometric priors to reduce data volume while preserving detail.
Technology
This patent introduces a learning-based point cloud geometry compression framework that utilizes geometric priors to enhance efficiency. The method involves:
- Geometric Prior Representation: A deformable template mesh (e.g., SMPL model) is aligned with the source point cloud, driven by a compact set of parameters (pose, shape, etc.). These parameters are quantized into a bitstream.
- Residual Feature Compression: Features are extracted from both the source point cloud and the aligned point cloud (derived from the template mesh) using sparse tensors. The features of the aligned point cloud are warped onto the source’s coordinates, and residual features are computed via subtraction. These residuals are compressed using an entropy model.
- Decoding: The decoder reconstructs the point cloud by decoding the parameter bitstream to regenerate the aligned mesh, converting it to a point cloud, and adding decoded residual features to recover fine details.
Key innovations include:
- Leveraging parametric human models (e.g., SMPL) as geometric priors to reduce redundancy.
- Feature-level residual computation, which is more efficient than direct coordinate matching.
- A plug-and-play framework compatible with existing sparse convolution-based compression methods.
Advantages
- Higher Compression Efficiency: Geometric priors reduce bitrate by up to 92% compared to traditional methods (e.g., G-PCC) and 29% over learning-based baselines (e.g., PCGCv2).
- Preservation of Details: Residual feature encoding retains fine-grained geometry, achieving better visual quality at lower bitrates.
- Scalability: Effective for point clouds with varying geometry precision (10-bit to 11-bit coordinates).
- Flexibility: Compatible with different feature extraction and warping techniques.
Applications
- Metaverse and XR: Efficient streaming of high-resolution avatars and virtual environments.
- Volumetric Video: Compression for real-time 3D video communication.
- Autonomous Systems: LiDAR point cloud compression for robotics and autonomous vehicles.
- Medical Imaging: Storage/transmission of 3D scans with minimal loss.
