PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes (2019-11-25T00:00:00.000000Z)

TL;DR

PQ-NET is introduced, a deep neural network which represents and generates 3D shapes via sequential part assembly which encodes a sequence of part features into a latent vector of fixed size and reconstructs the 3D shape, one part at a time, resulting in a sequential assembly.

Abstract

We introduce PQ-NET, a deep neural network which represents and generates 3D shapes via sequential part assembly. The input to our network is a 3D shape segmented into parts, where each part is first encoded into a feature representation using a part autoencoder. The core component of PQ-NET is a sequence-to-sequence or Seq2Seq autoencoder which encodes a sequence of part features into a latent vector of fixed size, and the decoder reconstructs the 3D shape, one part at a time, resulting in a sequential assembly. The latent space formed by the Seq2Seq encoder encodes both part structure and fine part geometry. The decoder can be adapted to perform several generative tasks including shape autoencoding, interpolation, novel shape generation, and single-view 3D reconstruction, where the generated shapes are all composed of meaningful parts.

Authors

Baoquan Chen

12 papers

Rundi Wu

3 papers

Kai Xu

6 papers

TL;DR

Abstract

Authors

References60 items

Convolutional Networks

Probabilistic Reconstruction Networks for 3D Shape Inference from a Single Image

SDM-NET: Deep Generative Network for Structured Deformable Mesh

PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

Learning Part Generation and Assembly for Structure-aware Shape Synthesis

DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction

Learning Shape Templates With Structured Implicit Functions

DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation

Composite Shape Modeling via Latent Space Factorization

Occupancy Networks: Learning 3D Reconstruction in Function Space

Learning Implicit Fields for Generative Shape Modeling

PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding

Global-to-local generative model for 3D shapes

Learning to Generate the "Unseen" via Part Synthesis and Composition

Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

3D-PSRNet: Part Segmented 3D Point Cloud Reconstruction From a Single Image

Structure-aware Generative Network for 3D-Shape Modeling

Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers

Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images

Learning Free-Form Deformations for 3D Object Reconstruction

A Papier-Mache Approach to Learning 3D Surface Generation

Automatic differentiation in PyTorch

3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks

The shape variational autoencoder: A deep generative model of part‐segmented 3D objects

3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks

Wasserstein Generative Adversarial Networks

Learning Representations and Generative Models for 3D Point Clouds

Synthesizing 3D Shapes via Modeling Multi-view Depth Maps and Silhouettes with Deep Generative Networks

Improved Training of Wasserstein GANs

Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs

A Point Set Generation Network for 3D Object Reconstruction from a Single Image

Learning Shape Abstractions by Assembling Volumetric Primitives

OctNet: Learning Deep 3D Representations at High Resolutions

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Learning a Predictable and Generative Vector Representation for Objects

Deep Residual Learning for Image Recognition

ShapeNet: An Information-Rich 3D Model Repository

Deep Convolutional Inverse Graphics Network

Sequence to Sequence Learning with Neural Networks

Meta-representation of shape families

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Learning part-based templates from large collections of 3D shapes

A probabilistic model for component-based shape synthesis

Exploration of continuous variability in collections of 3D shapes

Symmetry Hierarchy of Man‐Made Objects

On Visual Similarity Based 3D Model Retrieval

The space of human body shapes: reconstruction and parameterization from range scans

A Morphable Model For The Synthesis Of 3D Faces

Bidirectional recurrent neural networks

‘Families’

Recognition-by-components: a theory of human image understanding.

Parts of recognition

Analysis and synthesis of 3D shape families via deep-learned generative models of surfaces

and Y

Does the brain do inverse graphics?

Modern phrase structure grammar

Octree - based Convolutional Neural Networks for 3 D Shape Analysis

Grass : Generative recursive autoencoders for shape structures

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names