FaceNet implementation in Tensorflow

This is a TensorFlow implementation of the face recognizer described in the paper "FaceNet: A Unified Embedding for Face Recognition and Clustering". The project also uses ideas from the paper "A Discriminative Feature Learning Approach for Deep Face Recognition" as well as the paper "Deep Face Recognition" from the Visual Geometry Group at Oxford.

The accuracy measured on the LFW test set is ~0.98 when trained on a combination of FaceScrub and CASIA-WebFace.

Inspiration

The code is heavly inspired by the OpenFace implementation.

Training data

The FaceScrub dataset and the CASIA-WebFace dataset have been used for training. This training set consists of total of 536 685 images over 11105 identities.

Pre-processing

Face alignment using MTCNN

One problem with the above approach seems to be that the Dlib face detector misses some of the hard examples (partial occlusion, siluettes, etc). This makes the training set to "easy" which causes the model to perform worse on other benchmarks. To solve this, other face landmark detectors has been tested. One face landmark detector that has proven to work very well in this setting is the Multi-task CNN. A Matlab/Caffe implementation can be found here and this has been used for face alignment with very good results. Experimental code that has been used to align the training datasets can be found here. However, work is ongoing to reimplement MTCNN face alignment in Python/Tensorflow. Currently some work still remains on this but the implementation can be found here.

Running training

Currently, the best results are achieved by training the model as a classifier with the addition of Center loss. Details on how to train a model as a classifier can be found on the page Classifier training of Inception-ResNet-v1.

Pre-trained model

Inception-ResNet-v1 model

Currently, the best performing model is an Inception-Resnet-v1 model trained on a combination of FaceScrub and CASIA-Webface aligned with MTCNN. This alignment step requires Matlab and Caffe installed which requires some extra work. This will be easier when the Python/Tensorflow implementation is fully functional.

Performance

The accuracy on LFW for the model 20161030-023650 is 0.978+-0.008. A description of how to run the test can be found on the page Validate on LFW.