Pre-recorded Sessions: From 4 December 2020 | Live Sessions: 10 – 13 December 2020
4 – 13 December 2020
iOrthoPredictor: Model-guided Deep Prediction of Teeth Alignment
-
Ultimate Supporter
-
Ultimate Attendee
Date/Time:
04 – 13 December 2020
All presentations are available in the virtual platform on-demand.
Lecturer(s):
Lingchen Yang, State Key Laboratory of CAD & CG, Zhejiang University, China
Zefeng Shi, State Key Laboratory of CAD & CG, Zhejiang University, China
Yiqian Wu, State Key Laboratory of CAD & CG, Zhejiang University, China
Xiang Li, Zhejiang University, China
Kun Zhou, Zhejiang University, China
Hongbo Fu, City University of Hong Kong, Hong Kong
Youyi Zheng, State Key Laboratory of CAD & CG, Zhejiang University; Zhejiang University, China
Bio:
Description: In this paper, we present iOrthoPredictor, a novel system to visually predict the teeth alignment effect in a single photograph. Our system takes a frontal face image of a patient with visible malpositioned teeth along with a corresponding 3D teeth model as input, and generates a facial image with aligned teeth, mimicking the real orthodontic treatment effect. The key enabler of our method is an effective disentanglement of an explicit representation of the teeth geometry from the in-mouth appearance, where the accuracy of teeth geometry transformation is ensured by the 3D teeth model while the in-mouth appearance is modeled as a latent variable. The disentanglement enables us to achieve fine-scale geometry control over the alignment while retaining the original teeth appearance attributes and lighting conditions. The whole pipeline consists of three deep neural networks: a U-Net architecture to explicitly extract the 2D teeth silhouette maps representing the teeth geometry in the input photo, a novel multilayer perceptron (MLP) based network to predict the aligned 3D teeth model, and an encoder-decoder based generative model to synthesize the in-mouth appearance conditional on the original teeth appearance and the aligned teeth geometry. Extensive experimental results and a user study demonstrate that iOrthoPredictor is effective in generating high-quality visual prediction of teeth alignment effect, and applicable to industrial orthodontic treatments.
