Pre-recorded Sessions: From 4 December 2020 | Live Sessions: 10 – 13 December 2020
4 – 13 December 2020
Dynamic Facial Asset and Rig Generation from a Single Scan
-
Ultimate Supporter
-
Ultimate Attendee
Date/Time:
04 – 13 December 2020
All presentations are available in the virtual platform on-demand.
Lecturer(s):
Jiaman Li, University of Southern California, USC Institute for Creative Technologies, United States of America
Zhengfei Kuang, University of Southern California, USC Institute for Creative Technologies, United States of America
Yajie Zhao, USC Institute for Creative Technologies, United States of America
Mingming He, USC Institute for Creative Technologies, United States of America
Karl Bladin, USC Institute for Creative Technologies, United States of America
Hao Li, USC Institute for Creative Technologies; Pinscreen, Inc., United States of America
Bio:
Description: The creation of high-fidelity computer-generated (CG) characters used in film and gaming requires intensive manual labor and a comprehensive set of facial assets to be captured with complex hardware, resulting in high cost and long production cycles. In order to simplify and accelerate this digitization process, we propose a framework for the automatic generation of high-quality dynamic facial assets, including rigs which can be readily deployed for artists to polish. Our framework takes a single scan as input to generate a set of personalized blendshapes, dynamic and physically-based textures, as well as secondary facial components (e.g., teeth and eyeballs). Built upon a facial database consisting of pore-level details, with over $4,000$ scans of varying expressions and identities, we adopt a self-supervised neural network to learn personalized blendshapes from a set of template expressions. We also model the joint distribution between identities and expressions, enabling the inference of the full set of personalized blendshapes with dynamic appearances from a single neutral input scan. Our generated personalized face rig assets are seamlessly compatible with cutting-edge industry pipelines for facial animation and rendering. We demonstrate that our framework is robust and effective by inferring on a wide range of novel subjects, and illustrate compelling rendering results while animating faces with generated customized physically-based dynamic textures.
