Mimicking speaker's lip movement on a 3D head model using cosine function fitting

Abstract

Real-time mimicking of human facial movement on a 3D head model is a challenge which has attracted attention of many researchers. In this research work we propose a new method for enhancing the capturing of the shape of lips. We present an automatic lip movement tracking method which employs a cosine function to interpolate between extracted lip features in order to make the detection more accurate. In order to test the proposed method, mimicking lip movements of a speaker on a 3D head model is studied. Microsoft Kinect II is used in order to capture videos and both RGB and depth information are used to locate the mouth of a speaker followed by fitting a cosine function in order to track the changes of the features extracted from the lips.