Learning 3D Part Assembly from a Single Image
This repository contains a PyTorch implementation of the paper:
Learning 3D Part Assembly from A Single Image.
Yichen Li*, Kaichun Mo*, Lin Shao, Minhyuk Sung, Leonidas Guibas,
ECCV 2020
Introduction
Autonomous assembly is a crucial capability for robots in many applications. For this task, several problems such as obstacle avoidance, motion planning, and actuator control have been extensively studied in robotics. However, when it comes to task specification, the space of possibilities remains underexplored. Towards this end, we introduce a novel problem, single-image-guided 3D part assembly, along with a learningbased solution. We study this problem in the setting of furniture assembly from a given complete set of parts and a single image depicting the entire assembled object. Multiple challenges exist in this setting, including handling ambiguity among parts (e.g., slats in a chair back and leg stretchers) and 3D pose prediction for parts and part subassemblies, whether visible or occluded. We address these issues by proposing a two-module pipeline that leverages strong 2D-3D correspondences and assembly-oriented graph message-passing to infer part relationships. In experiments with a PartNet-based synthetic benchmark, we demonstrate the effectiveness of our framework as compared with three baseline approaches.
Dependencies
- Python 3.6
- CUDA 10.0.
- PyTorch. code tested with version 1.3.1
- Blender. for visualization of results 2.7.9
- (Optional) Tensorboard for visualization of the training process.
- For the project it has been used TensorboardX
pip install -r requirements.txt
- Chamfer Distance
cd exps/utils/cd
python setup.py install
Dataset
Data is available here: link.
wget http://download.cs.stanford.edu/orion/impartass/assembly_data.zip
Training
Training the segmentation stage first
cd exps/exp_segmentation
sh train.sh
modify your parameters including data_path, exp_name and etc. (see closed issues for details info)
Training the assembly stage
cd exps/exp_assemble
sh train.sh
Pre-trained models
Pretrained weights for the chair category is available at link.
wget http://download.cs.stanford.edu/orion/impartass/chair_weights.zip
Cite
Please cite our work if you find it useful:
@article{li2020impartass,
title={Learning 3D Part Assembly from a Single Image},
author={Li, Yichen and Mo, Kaichun and Shao, Lin and Sung, Minghyuk and Guibas, Leonidas},
journal={European conference on computer vision (ECCV 2020)},
year={2020}
}
148 Dec 1, 2022
213 Dec 17, 2022
833 Dec 28, 2022
47 Oct 11, 2022
50 Dec 27, 2022
67 Dec 15, 2022
9 Jun 17, 2022
35 Nov 10, 2022
253 Jan 1, 2023
42 Nov 17, 2022
3.2k Dec 25, 2022
7 Feb 10, 2022
46 Dec 14, 2022
50 Oct 19, 2022
32 Feb 21, 2022
18 Oct 6, 2022
26 Nov 23, 2022
22 Nov 30, 2022
117 Dec 11, 2022