Courses

Q&A for Virtual Hands in VR: Motion Capture, Synthesis, and Perception

Author(s)/Presenter(s):
Yuting Ye, Facebook Reality Labs, United States of America
Franziska Mueller, Max-Planck-Institut für Informatik, Germany
Victor Zordan, Clemson University, United States of America
Michael Neff, University of California Davis, United States of America

Abstract:
We use our hands every day: to grasp a cup of coffee, write text on a keyboard, or signal that we are about to say something important. We use our hands to interact with our environment and to help us communicate with each other without thinking about it. Wouldn't it be great to be able to do the same in virtual reality? However, accurate hand motions are not trivial to capture. In this course, we present the current state of the art when it comes to virtual hands. Starting with current examples for using hands in virtual reality (VR), we dive into the latest methods and technologies to motion capture and represent hands in VR. As hands can currently not be captured in every situation and as constraints stopping us from intersecting with objects are typically not available in VR, we present research on how to synthesize hand motions and simulate grasping motions. Finally, we provide an overview of our knowledge of how virtual hands are being perceived, resulting in practical tips on how to represent and handle virtual hands. Our goals are (a) to present a broad state of the art of the current usage of hands in VR, (b) to provide more in-depth knowledge about the functioning of current hand motion tracking and hand motion synthesis methods, (c) to give insights on our perception of hand motions in VR and how to use those insights when developing new applications, and finally (c) to identify gaps in knowledge that might be investigated next. While the focus of this course is on VR, many parts also apply to augmented reality, mixed reality, and character animation in general, and some content originates from these areas.

Back