Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images

ICCV 2021

Long-Nhat Ho
Anh Tran
Quynh Phung
Minh Hoai

[Paper]
[GitHub]
[Video]



Abstract

Recovering the 3D structure of an object from a single image is a challenging task due to its ill-posed nature. One approach is to utilize the plentiful photos of the same object category to learn a strong 3D shape prior for the object. This approach has successfully been demonstrated by a recent work of Wu et al. (2020), which obtained impressive 3D reconstruction networks with unsupervised learning. However, their algorithm is only applicable to symmetric objects. In this paper, we eliminate the symmetry requirement with a novel unsupervised algorithm that can learn a 3D reconstruction network from a multi-image dataset. Our algorithm is more general and covers the symmetry-required scenario as a special case. Besides, we employ a novel albedo loss that improves the reconstructed details and realisticity. Our method surpasses the previous work in both quality and robustness, as shown in experiments on datasets of various structures, including single-view, multi-view, image-collection, and video sets.

Method Overview

Overview of the LeMul system.
We train a decomposing network to optimize different loss components.
Note that we use diffuse shading images to visualize depth maps.

Results



Acknowledgements

This project was greatly inspired by Unsup3d (LeSym)