Extraction of soft-biometry traits from profile pictures in social networks

• Applicant: Jordi Gonzàlez, Computer Vision Center-Universitat Autònoma de Barcelona, WP3 member
• Host: Thomas Moeslund, VAP – University of Aalborg, MC member
• Period: 01/08/2015 to 31/08/2015

Purpose. This STSM aimed to establish a new collaboration between two IC1106 institution members. The visit established a new collaboration line between VAP and CVC‐UAB, which has allowed the applicant to learn new techniques related to soft‐ biometry extraction of static pictures, and to optimise and apply convolutional neural networks (CNNs) in the problem as defined by VAP.

Work carried out. The main aim of the research carried out during the STSM has been to apply the latest state‐of‐the‐art soft‐biometry recognition analysis algorithms developed by VAP and combine these with the auto key‐wording algorithms and software tools developed by CVC‐UAB (based on deep learning and convolutional neural networks) for facial pain recognition, in order to create a tailored solution that automatically classifies the levels of pain given an image.

Outcomes and future collaboration. The outcomes of the work carried out include an automatic classification procedure for extracting the pain description of the facial appearance of a user (VAP) using deep learning algorithms (CVC‐UAB). We plan to submit a collaborative journal (with impact factor) before 2016 on this topic. Also, we have collected social media data in Aalborg for further CNNs fine-tuning and enhance the description of a person with clothes and other soft-biometric characteristics of a human face. Towards this end, we have established a schedule of Skype meetings for finishing the work initiated during the STSM visit in terms of a joint ECCV2016 submission .

Additional activities. During the visit, I participated regularly in the research meetings of the VAP group, and I gave the following talk:

• Towards Visual Hermeneutics: From pixels to semantics, Aalborg University, August 11, 2015.