Light Field Based Face Recognition Via A Fused Deep Representation

Alireza Sepas-Moghaddam, Paulo Lobato Correia, Kamal Nasrollahi, Thomas B. Moeslund, Fernando Pereira

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

1 Citation (Scopus)

Abstract

The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.
Original languageEnglish
Title of host publicationIEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING
EditorsNelly Pustelnik, Zheng-Hua Tan, Zhanyu Ma, Jan Larsen
Number of pages6
PublisherIEEE
Publication date31 Oct 2018
Article number8516966
ISBN (Print)978-153865477-4
ISBN (Electronic)9781538654774
DOIs
Publication statusPublished - 31 Oct 2018
EventIEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING - Aalborg, Denmark
Duration: 17 Sep 201820 Sep 2018

Conference

ConferenceIEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING
CountryDenmark
CityAalborg
Period17/09/201820/09/2018
SeriesMachine Learning for Signal Processing
ISSN1551-2541

Fingerprint

Face recognition
Biometrics
Feature extraction
Classifiers
Fusion reactions
Textures
Cameras
Experiments

Keywords

  • Depth Map
  • Disparity Map
  • Face Recognition
  • Fine-Tuning
  • Lenslet Light Field Imaging
  • VGG-Face Descriptor

Cite this

Sepas-Moghaddam, A., Correia, P. L., Nasrollahi, K., Moeslund, T. B., & Pereira, F. (2018). Light Field Based Face Recognition Via A Fused Deep Representation. In N. Pustelnik, Z-H. Tan, Z. Ma, & J. Larsen (Eds.), IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING [8516966] IEEE. Machine Learning for Signal Processing https://doi.org/10.1109/MLSP.2018.8516966
Sepas-Moghaddam, Alireza ; Correia, Paulo Lobato ; Nasrollahi, Kamal ; Moeslund, Thomas B. ; Pereira, Fernando. / Light Field Based Face Recognition Via A Fused Deep Representation. IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING. editor / Nelly Pustelnik ; Zheng-Hua Tan ; Zhanyu Ma ; Jan Larsen. IEEE, 2018. (Machine Learning for Signal Processing).
@inproceedings{a5afe9515b47414baf96cfbd64a0c554,
title = "Light Field Based Face Recognition Via A Fused Deep Representation",
abstract = "The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.",
keywords = "Depth Map, Disparity Map, Face Recognition, Fine-Tuning, Lenslet Light Field Imaging, VGG-Face Descriptor",
author = "Alireza Sepas-Moghaddam and Correia, {Paulo Lobato} and Kamal Nasrollahi and Moeslund, {Thomas B.} and Fernando Pereira",
year = "2018",
month = "10",
day = "31",
doi = "10.1109/MLSP.2018.8516966",
language = "English",
isbn = "978-153865477-4",
series = "Machine Learning for Signal Processing",
editor = "Nelly Pustelnik and Zheng-Hua Tan and Zhanyu Ma and Jan Larsen",
booktitle = "IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING",
publisher = "IEEE",
address = "United States",

}

Sepas-Moghaddam, A, Correia, PL, Nasrollahi, K, Moeslund, TB & Pereira, F 2018, Light Field Based Face Recognition Via A Fused Deep Representation. in N Pustelnik, Z-H Tan, Z Ma & J Larsen (eds), IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING., 8516966, IEEE, Machine Learning for Signal Processing, IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING, Aalborg, Denmark, 17/09/2018. https://doi.org/10.1109/MLSP.2018.8516966

Light Field Based Face Recognition Via A Fused Deep Representation. / Sepas-Moghaddam, Alireza ; Correia, Paulo Lobato; Nasrollahi, Kamal; Moeslund, Thomas B.; Pereira, Fernando.

IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING. ed. / Nelly Pustelnik; Zheng-Hua Tan; Zhanyu Ma; Jan Larsen. IEEE, 2018. 8516966 (Machine Learning for Signal Processing).

Research output: Contribution to book/anthology/report/conference proceedingArticle in proceedingResearchpeer-review

TY - GEN

T1 - Light Field Based Face Recognition Via A Fused Deep Representation

AU - Sepas-Moghaddam, Alireza

AU - Correia, Paulo Lobato

AU - Nasrollahi, Kamal

AU - Moeslund, Thomas B.

AU - Pereira, Fernando

PY - 2018/10/31

Y1 - 2018/10/31

N2 - The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.

AB - The emergence of light field cameras opens new frontiers in terms of biometric recognition. This paper proposes the first deep CNN solution for light field based face recognition, exploiting the richer information available in a lenslet light field image. Additionally, for the first time, the exploitation of disparity maps together with 2D-RGB images and depth maps has been considered in the context of a fusion scheme to further improve the face recognition performance. The proposed solution uses the 2D-RGB central sub-aperture view as well as the disparity and depth maps extracted from the full set of sub-aperture images associated to a lenslet light field. After, feature extraction is performed using a VGG-Face deep descriptor for texture and independently fine-tuned models for disparity and depth maps. Finally, the extracted features are concatenated to be fed into an SVM classifier. A comprehensive set of experiments has been conducted with the IST-EURECOM light field face database, showing the superior performance of the fused deep representation for varied and challenging recognition tasks.

KW - Depth Map

KW - Disparity Map

KW - Face Recognition

KW - Fine-Tuning

KW - Lenslet Light Field Imaging

KW - VGG-Face Descriptor

UR - http://www.scopus.com/inward/record.url?scp=85057018492&partnerID=8YFLogxK

U2 - 10.1109/MLSP.2018.8516966

DO - 10.1109/MLSP.2018.8516966

M3 - Article in proceeding

SN - 978-153865477-4

T3 - Machine Learning for Signal Processing

BT - IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING

A2 - Pustelnik, Nelly

A2 - Tan, Zheng-Hua

A2 - Ma, Zhanyu

A2 - Larsen, Jan

PB - IEEE

ER -

Sepas-Moghaddam A, Correia PL, Nasrollahi K, Moeslund TB, Pereira F. Light Field Based Face Recognition Via A Fused Deep Representation. In Pustelnik N, Tan Z-H, Ma Z, Larsen J, editors, IEEE International Workshop on MACHINE LEARNING FOR SIGNAL PROCESSING. IEEE. 2018. 8516966. (Machine Learning for Signal Processing). https://doi.org/10.1109/MLSP.2018.8516966