Beneficiary: Necati Cihan Camgöz, Master’s Student at Boğaziçi University
Host: Assist. Prof. Dr. Vitomir Struc, University of Ljubljana, Faculty of Electrical Engineering
Period: 27/04/2015 to 01/05/2015
Purpose of the STSM
First and far most important goal of this short term scientific mission to Ljubljana University was to establish a fruitful collaboration between two institutes while gaining further knowledge on my research field from my esteemed host. Our goals further included to share knowledge on facial landmark localization methods, analyze experimental results which were conducted on Bosphorus 3D face dataset, discuss possible improvements over the previously developed Supervised Descent Method (SDM) based approaches, and examine the effects of having better facial landmark localization on 3D face recognition.
Description of the work carried out during the STSM
By working with Assist. Prof. Dr. Vitomir Struc and his post-doctoral scholar colleague Dr. Janez Krizaj, who are experts on 3D facial landmark localization and had their own approach based on SDM, we were able assess the performance of our methods and discuss the possible ways to improve 3D facial landmark localization. Since the host was already working on the Bosphorus 3D face dataset there was no need for an introduction. We can categorize the work carried out during the STSM in to the following topics:
Creating the Experimental Setup:
Taking state-of-the art methods on 3D Facial Landmark Localization into consideration, we have created our experimental setup on Bosphorus 3D face dataset; so that we would be able to compare our results with them. In our setup we divided the dataset in to two approximately equal, user exclusive parts and did two fold cross-validation for our experiments.
Comparing 3D Facial Landmark Localization Methods and Analyzing the Experimental Results:
Since both parties had their own 3D facial landmark localization approach based on SDM, we experimented with both of the approaches. Our experiments showed us that the approach proposed by the host had higher 3D facial landmark localization accuracy. Furthermore, by comparing with the state-of-the-art methods we have realized that this approach had state-of-the-art performance while working with 10 and 22 most commonly used landmarks on Bosphorus 3D face dataset.
To examine the effects of the features which have been used to describe 3D facial landmarks we have experimented with various sizes of Histogram of Oriented Gradients (HOG) and Scale Invariant Feature Transform (SIFT). These experiments showed us that using HOG features generally resulted in higher accuracy of 3D facial landmark localization. Also finding the optimal size for the features affected the performance drastically. We also experimented with changing the size of the features in each step of the SDM, which gave the approach a coarse-to-fine alignment effect and increased the performance.
Discussing ways to improve the current 3D Facial Landmark Localization Method:
By examining the experimental results we came to the conclusion that there are two ways we can improve the current approach. First is by coming up with a better structure for SDM, i.e. a cascaded approach, which will allow the landmark localization to be more precise as it iterates through its stages. The second idea is to use or create better features for describing 3D facial landmarks, which will exploit the 3D information of the data.
Examining the Effects of Better Facial Landmark Localization Method on 3D Face Recognition:
A naïve 3D face recognition method was applied to faces that were aligned using facial landmarks found by both parties’ approaches. Although the face recognition performances were lower than the state-of-the-art, experiments showed us that the approach with better facial landmark localization resulted in more successful 3D face recognition.
Description of the main results obtained
Our experiments showed us that the approach proposed by the host had state-of-the-art 3D facial landmark localization performance while working with 10 and 22 most commonly used facial landmarks on Bosphorus 3D face dataset.
We have also realized that working with different features can affect the 3D landmark localization performance drastically (HOG vs. SIFT). To this extent we started on experimenting with different features, which may be more suitable for 3D data.
To examine the effects of having better facial landmark localization on 3D face recognition, we have experimented with a naïve 3D face recognition approach which used the facial landmark locations from both parties’ approaches to align faces. We have seen that better facial landmark localization results in a higher 3D face recognition accuracy, thus proving the 3D facial landmark localization being crucial for biometric applications that work on 3D data.
Future collaboration with host institution
The experiments during the STSM showed that the method in our hands has the state-of-the-art 3D facial landmark localization performance. Currently we are experimenting on ways to improve this method by trying out cascaded structures and new features which will exploit the 3D information. Our goal is to turn this work into a publication as our collaboration continues.