Courses

Q&A for Accelerating 3D Deep Learning with PyTorch3D

Author(s)/Presenter(s):
Justin Johnson, Facebook AI Research; University of Michigan, Ann Arbor, United States of America
Nikhila Ravi, Facebook AI Research, United States of America
Jeremy Reizenstein, Facebook AI Research, United States of America
David Novotny, Facebook AI Research, United States of America
Shubham Tulsiani, Facebook AI Research, United States of America

Abstract:
3D understanding plays an important role in advancing the ability of AI systems to better understand and operate in the real world — including navigating physical space in robotics, improving virtual reality experiences, and even recognizing occluded objects in 2D content. But research in 3D deep learning has been limited due to the lack of sufficient tools and resources to support the complexities of using neural networks with 3D data and the fact that many traditional graphic operators are not differentiable. We have built PyTorch3D, a highly modular and optimized library with unique capabilities designed to make 3D deep learning easier with PyTorch. PyTorch3D provides a set of frequently used 3D operators and loss functions for 3D data that are fast and differentiable, as well as a modular differentiable rendering API — enabling researchers to import these functions into current state-of-the-art deep learning systems right away. PyTorch3D was a catalyst in our work to build Mesh R-CNN (ICCV 2019), which achieved full 3D object reconstruction from images of complex interior spaces. In addition to Mesh R-CNN, PyTorch3D's efficient and differentiable renderer has unlocked a variety of exciting applications such as novel view synthesis from a single image in an end-to-end manner, as shown in the recent SynSin project (CVPR 2020). PyTorch3D has gained popularity in the open source community, reaching over 3.3k stars on GitHub. In recent months there have been several new research works in 3D deep learning powered by PyTorch3D at leading conferences. In this course, we will do a deep dive into PyTorch3D. We will start with hands-on tutorials providing a detailed overview of the library. Finally, we will give a short presentation on new research projects which have been enabled by PyTorch3D. Repo: https://github.com/facebookresearch/pytorch3d Page: https://pytorch3d.org/ Blog: https://ai.facebook.com/blog/-introducing-pytorch3d-an-open-source-library-for-3d-deep-learning/

Back